Why I Love Rockstor on our Storinators

As a NAS operating system, FreeNAS has been out there for a decade. Working on the beautiful marriage of FreeBSD and ZFS, it's been tried and tested by many users within the storage community, to great success. But if you're one of those people who is more into Linux and Btrfs, you may be yearning for an alternative.

Over the past few months here in the 45 Drives lab, we have been putting an exciting new NAS OS through its paces on our Storinators. After working through a few kinks, such as how to set up redundant boot drives, we are ready to introduce Rockstor as one of our recommended solutions to create a solid NAS appliance when installed on a Storinator (alongside the traditional OS option, FreeNAS). I am writing this post to talk about why I like Rockstor, its Linux+Btrfs roots, and to share a bit of the story of how we got here.

Welcome, Rockstor.

Some of you may already be familiar with, or at least have heard of, Rockstor, a Linux + Btrfs-powered NAS OS. I, and the rest of the 45 Drives team, first heard of Rockstor when founder and lead engineer Suman Chakravartula contacted us back in November 2014 as a prospective hardware partner. We were pretty excited with the idea of helping a new NAS appliance gets its footing – it’s just another example of how the storage community really is a collaborative environment, and one that we’re proud to be a part of.

Personally, I was extra excited, as this was a solid Linux alternative to the already proven FreeBSD-based FreeNAS. Nothing against the FreeBSD/NAS software – we have extensive experience with FreeBSD and FreeNAS, as well as many Storinator owners who are getting fast and reliable results – but personally I’m a Linux guy at heart.

I couldn’t wait to get my hands on not only a Linux-based NAS OS, but one that was powered by Btrfs, a (relatively) new file-system that has matured into a very reliable option for large data storage. (Note: the Linux preference is mine, with some others in the 45 Drives team feeling differently. I'm happy our company lets me express my views :) )

I also want to note that Suman and his team have been great to work with, and have recently put together some features that they and we agreed would be super helpful to those running RockStor on large NAS servers such as Storinators. This includes the ability to back up configurations, which I'll elaborate below. But I am not going to talk about these or any of the other wonderful features of Rockstor, as Suman and his team have put together a great website and documentation outlining their OS. Also don’t forget to check their blog and community forum to keep in the loop as they add more and more.

But what I really want to talk about is why Btfrs + Linux are winners in the NAS world.

Why I Love Btrfs

First developed by Oracle, Btrfs is a file-system that has since seen lots of work to move it into maturity. As of July 2014, it’s been included in the Linux 3.10 kernel as a stable filesystem. Due to the fast development speed, the file-system improves noticeably with every new kernel version, so it’s recommended to run the most modern kernel possible. Rockstor abides by this recommendation – you’ll notice your version of Rockstor is running a very recent kernel.

Btrfs’ big draw is its Copy on Write (CoW) nature of the filesystem. Essentially, when multiple users want to read/write a file, it does not make a separate copy until changes are made to the original file by the user.

This has a benefit of having a "memory" of past changes, allowing you to use snapshots of the volume to restore files from a past point of time should anything go wrong.

Note that a snapshot is not back up of your data and if your volume goes down, so will your snapshots. Snapshots are meant to be used if files accidentally get corrupted or deleted. One can simply backtrack a few hours (or whatever frequency you set them at) and restore the broken/missing files.

Another one of my favorite features of Btrfs is that it has its own native RAID support built in called Btrfs-RAID. One of the biggest benefits of how Btrfs does its RAID is that a RAID6 volume does not need any additional re-syncing upon creation of the RAID. Using Linux RAID, “mdadm”, the initial build of a 258TB RAID6 (that’s a Storinator S45 filled with 6TB drives), can take up to two days depending on your system. You can also add disks into a mounted Btrfs-RAID live with a single command, and not only that, but you can also convert non-raid mounted device into a Btrfs-RAID just as easily. Rockstor makes all of this painless through its management interface, but if you would like to learn more, check out this link on Btrfs-RAID.

Plus, as my benchmarks below will show, this OS+Filesystem is smokin' fast! 

Making Rockstor Perfect for the Storinator

Inspired by FreeNAS, we had one feature that we were dead set on having – an option to back up the current configuration of Rockstor, so that you could revert to a previous setup if anything went wrong. Just as Btrfs can take snapshots of your data, you can think of this as a snapshot of your NAS setup. 

We had asked Suman if this could be implemented back in late June 2015. Sure enough, within two weeks, the Rockstor team went out of its way to provide it for us. It’s just another example on how dedicated these guys are to their product.

Ultimately, the best part of all of this, for me, is that Rockstor is built on Linux. It’s like playing hockey on a Friday afternoon, it just feels right. This is how I feel, being a Linux guy – the BSD diehards are probably scoffing at their computer screens right now (we can agree to disagree though, right?).

Rockstor Key Benefits

So to sum up, Rockstor is a solid Btrfs+Linux-powered NAS appliance that has a wonderfully talented and hardworking development team behind it. 

     • Now that Btrfs is quite stable and mature, it has all kind of features to make large data storage painless. This includes snapshots, Btrfs-RAID, and even more! I’ve even heard rumors that native support for hybrid storage (automated tiers) is in the works.
     • The Linux base of this appliance will make any Linux user feel right at home, and give them the freedom to experiment with new ideas. 
     • The Rockstor Management Interface is pretty intuitive and the latest update has introduced the most attractive Rockstor yet. You can check out a demo and download via their site. Email info@45drives.com if you are interested in a Storinator NAS Appliance with Rockstor (we’ve dubbed it the Rockinator!).

Rockstor Benchmarks

I couldn’t finish this post without a graph or two, however. Given the impressive results I got from benchmarks I ran on a Btrfs volume created in Rockstor – over 3GB/s read/write speed! – how could I not share?

I used a base-level Storinator Q30 to generate these results, with an 18 disk Btfrs-RAID6 array configuration using WD Ae Enterprise-level drives.

This test consisted of using IOzone to read and write 1MB block sized sequential 64GB files on the volume. The Y-axis represents aggregate throughput in GB/s and the X-axis is the number of parallel threads, meaning that 1 thread will read/write 1 64GB file, 2 parallel threads will r/w 2 64GB files simultaneously, and so on through to 20 simultaneous transfers. This test was run on the volume locally.

This is the same data as above but plotted as Throughput/Thread (GB/s) vs Number of Thread.

According to theory, 18 drives in a RAID6, using HDDs that can do 170MB/s each, should give an expected throughput around 3060MB/s.

You can see at 1 thread we are pretty close to true disk speed. I have yet to see such agreement with theory with any other file-systems.

It is interesting to note that the write throughput is a bit higher than the read (~100MB/s). This doesn’t agree with theory as one would expect the RAID6 write penalty to significantly reduce the write performance. I made sure to run this test multiple times and average the results to ensure that this was not an anomaly.

Explaining the Discrepancies

In search of an answer, I came across this entry in the Btrfs Mailing List, explaining how RAID 5&6 work in Btrfs. The author of the post is seeing similar write performance to mine, about six times what you would expect out a MD-RAID.

It appears that, unlike MD-RAID, Btrfs-RAID does all its parity calculations at the time of IO submission rather than handing them off to helper threads which would cause some latencies and therefore decrease overall throughput.

Regardless, these numbers are quite impressive and really show off the performance capabilities of Btrfs and Rockstor+Storinator. I’m intrigued, and excited to see where this takes 45 Drives’ Storinator NAS Appliances.

Have you tried Rockstor yet? When it comes to Linux vs FreeBSD, where does your allegiance lie? Let us know your thoughts in the comments.