dmraid (fakeraid) mirror + striped

30Jun11

While some folk look down on fakeraid (that is BIOS based RAID-until-OS-takes-over) solutions, I think they are pretty neat: they let a user get many of the benefits of dedicated controller cards at a fraction of the cost. The benefits include the usual ones for RAID – more spindles to handle IO, tolerance of disk failures. And unlike pure LVM solutions, you can boot from a degraded RAID 1 / 5 / 10 set because the BIOS knows how.

In some ways this is better than dedicated cards, because we have the software take over, so we can change the algorithms for IO dispatch all the way down to the individual devices :)

However, these RAID volumes are in a pretty awkward spot for installers and bootloaders: inside a running Linux environment they look like software RAID which cannot be depended on for booting, but at boot time they look like hard disks which cannot be looked under the hood.

I recently got a new desktop machine which has one of these motherboards, and fortuitously my old desktop I was replacing had the same size disks – so I had 4 disks and the option of using a RAID setup. Apparently I’m a sucker for punishment because I went for a RAID 10 (that is two RAID volumes made up of two-disk mirrors (the RAID 1 component), and then those two volumes are combined via striping (the RAID 0 component). This has the potential for pretty nice performance: in principle any read can come from one of 2 disks, and every 64KB (the stripe size) of linear data will switch to the other mirror set, giving a nice boost. Writes need to write to 2 disks always, but every 64KB worth of data will alternate mirror sets, also giving a boost.

Sadly we (Ubuntu) aren’t ready for this yet: there are two key bugs that make this layout almost impossible to install into. This blog post is for my exo-memory, I want to be able to figure out what I did next time around :).

Firstly parted_devices, a helper used by Ubiquity and debian-installer to determine which block devices are actually disk drives that one can partition and install onto, has a confused heuristic – when dealing with dmraid it looks for devices which are not layered on other dmraid devices. This handily excludes partitions, but has the undesirable effect of excluding that striped device – because it is layered on the two mirrored devices. Bug 560748 was filed about that, and I’ve added a workaround to it – basically disabling the filtering, so its not suitable as a long term fix, but it will let one select the RAID volume correctly.

Secondly, grub2, which needs to figure out what the name at boot time of the RAID volume will be currently gets confused. I don’t know enough to really explain – and be correct in my explanation – but I do have a fugly patch which worked for me. Bug 803658 tracks this defect. The basic approach I took was to say that dmraid devices should be an abstraction layer we don’t peek under: if it claims to be a disk, well then its a disk. As grub does actually work that way  – it talks to INT 13h – the BIOS support for booting off of the RAID volume is entirely sufficient.

Sadly neither bug is at the point where the patches can be rolled into Ubuntu itself, but the workaround should let folk get up and running.

In both cases, build the package locally in the installer, install it, then after than run ubiquity and things should install.

After the install, you will need to reapply the patch in the resulting installed environment, or things like update-grub will die on you!

(huge thanks to cjwatson and ev for giving me some tips while I investigated this)

About these ads


5 Responses to “dmraid (fakeraid) mirror + striped”

  1. Fakeraid sucks ass. If you want raid and want it to actually work, get a card. They are cheap and work great. Fakeraid is ok for desktop smallfry but for anything even slightly more than this it just fails horribly.

  2. If you make sure that /boot (and if you’re as paranoid as me, /) are on RAID1, and install grub on _all_ disks, and once done, actually try booting off of each disk individually, and prove to yourself that the machine doesn’t care which disk it’s booting off of, or the order that they are in, then you get all the benefits you’ve described, with the added bonus that you’re not tied to using a crappy fakeraid with the associated danger that the on-disk format is incomprehensible to some other motherboard.

    • 3 rbtcollins

      I don’t quite follow – to have /boot and / on RAID1 and bootable implies manual recovery if/when something dies.

      The intel ICH*R series chips have a very stable format – I don’t have an expectation that the disk format would be portable to say an nvidia southbridge, but on the other hand, with a RAID adapter card, you also wouldn’t have that expectation. You’d expect that a matching card from the same vendor could recover your array, not that an arbitrary vendor could.

      Anyhow, I’ve run this method in the past, and yes it works. Its also about ten times as complex from the point of view of a small scale system integrator or small business owner: the very market that these entry-level RAID configs are aimed at.

  3. Fakeraid is pretty much worst of both worlds – dependency on firmware magic for boot and “disk format”, while providing same performance as software raid. Just, Dont, Use. It ;)

  4. 5 James

    I’d really like to be able to use the nice simple bios raid options, but unfortunately, linux dmraid RAID1 seems to be pretty much worthless (tested in Ubuntu Lucid).

    I tested a new install by pulling one disk; the machine hung for a few seconds, then recovered. But when I put the disk back in later, it thought everything was A-O-K immediately, with two active disks in the raid…Which seemed completely wrong, it should’ve been starting a resync. Just to make sure it was really lying to me about everything being all hunky-dory, I pulled out the other disk. And sure enough, that blew everything up. :) So much for reliability.

    I think BIOS-supported software raid would be pretty nice, if only the linux driver for it wasn’t so useless. But for now, I’d say to watch out if you ever actually *need* the reliability, and aren’t just using the second disk for decoration.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Follow

Get every new post delivered to your Inbox.

Join 1,056 other followers

%d bloggers like this: