How to Protect Your Data Using ZFS
A question often asked in the server industry is, ‘what’s better - software RAID or hardware RAID?’ If you research this topic, a lot of the information about RAID suggests that a hardware RAID card is preferable to using a Software RAID. But I don’t agree. In my opinion is that, for most applications, Software RAID is far better than hardware RAID. Hardware RAID does have a place under certain OS’s, but I’m going to tell you why Software is generally far superior.
Favoring hardware RAID over software RAID comes from a time when hardware was just not powerful enough to handle Software RAID processing, along with all the other tasks that it was being used for. Back then, the solution was to use a hardware RAID card with a built-in processor that handled the RAID calculations ‘offline’. It would present itself to the computer’s OS as a single disk, and internally it would process data moving in and out of the multiple storage devices (hard drives). This made things run smoothly while benefiting from the security of RAID.
Hardware RAID is still popular with some people and many of today’s hardware RAID cards offer kick-ass performance while lightening the load on the CPU, but there’s still some serious problems and disadvantage:
- Challenges recovering data when major failures happen
- Proprietary/nonstandard protocols which mean your array only works with the same hardware raid card. You can’t plug in your set of hard drives into any RAID card and expect it to read your data.
- Hardware RAID has the ability to take a group of drives and make it appear as a single drive. This architecture is elegant in its simplicity. However this also fundamentally precludes integration of features into the OS and file system. This integration is really what has allowed Software RAID to dramatically outpace hardware RAID.
Where I Believe We Are Today
Two things happened that benefited Software RAID over hardware RAID and allowed it to take the lead. Computing power grew so radically that the computing load presented by RAID is no longer significant. Secondly, strength, features, and integration of RAID Software has grown dramatically.
Hardware RAID continues to offer solid and simple architectural solutions for combining multiple drives into RAID arrays and presenting them to the OS as a single device. This is particularly useful with MS Windows which has a painfully slow implementation of software RAID. However, it continues to have a data security risk because you need to use an identical (or compatible) controller to recover data in the event of hardware failure.
On the software side, today’s software RAID is super-fast (at least with Linux and BSD), extremely flexible, and highly integrated into OS’s. It’s also much more capable and powerful in recovery situations than hardware RAID. To recover your data, all you need is another storage server with the same OS. After that the steps are simple, just plug the drives in and get to work – you should be able to recover from just about any situation where your data loss hasn’t exceeded fundamental limits.
It’s clear that computers and software have come a really long way and it’s becoming clearer that software-defined RAID is going to be more and more prevalent as time goes by.
ZFS File System and Volume Manager
Now we're going to dig into one of our favorite file systems and volume managers, ZFS. We’re going to talk about some of the features that make ZFS unique and then give you an example from one of our customers who saved a lot of money because he was using ZFS with software RAID.
We love ZFS because it can bypass a lot of the issues that might arise when using traditional RAID cards. For example, instead of a hardware RAID card getting the first crack at your drives, ZFS uses a JBOD card that takes the drives and processes them with its built-in volume manager and file system. This gives ZFS a greater control to bypass some of the challenges hardware RAID cards usually have. Because of this control and its functional features - ZFS can handle errors extremely well. These features including Copy on Write, Snapshots, practically infinite scaling, self-healing with check sums, and built-in virtualization for your storage pool, give ZFS an extra level of robustness.
ZFS has its own way to structure new writes called Copy on Write. This is different from the way most volume managers structure new writes. Normally when a modification or new data is written, it is written over the old data. However, if there is a power failure during that write - the data could be lost. Instead of writing over the old data, ZFS writes data to a new location and copies the pathway over. This means your data doesn’t get lost if you lose power while it is writing. It also gives ZFS the ability for another one of its most useful features; Snapshots.
Snapshots are essentially time stamps that show what the pathway for data was at a specific point in the past, these are stored in their own table separate from data. Because ZFS doesn’t overwrite data and instead writes it to a new location, you can revert back to a previous timestamp, it’s almost functioning as a backup. Snapshots have far less overhead than a real backup though, as a full backup requires copying your data. Snapshots can be taken quickly and easily in comparison.
ZFS also has an incredible ability to heal itself against write errors, data corruption or bit rot. It analyzes data stored in redundant locations using checksums and repairs itself based on inconsistencies. It has traditional RAID functionality as well - utilizing mirroring, striping and parity checks. When using a hardware RAID card, the disk health checker gets masked when the disks are plugged into it. This is a problem because you do not get to see the looming signs one of your disks is going to kick the bucket. ZFS gets around this because it doesn’t have any operations written to the drives before they are presented to the OS.
Customer Case Example
We all make mistakes, some can be more costly than others. One of our customers using ZFS on his Storinator had to move offices and did not take out the hard drive disks before moving and somehow damaged the disks during the move.
When the customer attempted to boot up the Storinator and import the ZFS pool - some of the drives were damaged and some were misplaced – there was even corrupted metadata. However, because the customer was using ZFS he was able to rebuild the pool reasonably easily.
Our support team here at 45Drives was able to manually import the pool as degraded. Once the pool was in we were able to add new drives, scrub the pool and things went back to normal.
We were only able to do this because of ZFS’s resiliency but this would not have been possible if our customer was using hardware RAID because it is much more sensitive to component failures. If they were using hardware RAID there was a good chance that their data would have been lost - without some sort of expensive recovery.
If you need the utmost in security, features, and performance, software RAID is the answer. Note, that you’ll need to use a high-performance OS like Linux or BSD; and if doing so, you really should consider ZFS. Its performance is excellent on today’s machines, it takes data security to an unprecedented level and as a bonus, and it is really easy to use once you come up the learning curve.
No comments:
Post a Comment