Encrypted LVM on RAID1 - Failure testing, manual mount

Question

We have an Ubuntu 10.04 LTS server set up with LVM running on top of disk encryption, which runs on top of a RAID1 volume comprised of two physical SATA disks. The configuration looks like this:

             _______________
            |swap|     /    | - LVM volume for swap (10 GB)
            |____|__________| - LVM volume for root (rem. space ~ 490 GB)
            |______LVG0_____|  - LVM group  
           /d0fc97a3b3e8ae5a8\    - dm-crypt encrypted partition on RAID1
    ______/___________________\______
   |/boot|  > RAID1 | > RAID1  |/boot| - 250MB /boot partition on each disk
   |_____|__________|__________|_____| - remaining space on each for RAID
   |______500 GB____|____500 GB______|   - SATA HDD x 2

This works just great, however when I go to simulate a cold disk failure (shutdown, unplug one or the other of the disks, and power up), it fails to prompt for my boot passphrase and shows the error:

cryptsetup: LVM device name (dev/disk/by-UUID/[UUID String]) does not begin with /dev/mapper

After about a 5 minute wait, it will come up with a BusyBox (initramfs) prompt. My question is, why won't it ask for the boot passphrase when booted this way from either disk? I did choose the option to continue booting after a disk failure when I installed the RAID partition, but this obviously isn't happening. Second, how can I manually get it to unlock the remaining disk from the RAID array and boot properly using the BusyBox shell?

Thanks in advance.

EDIT: As requested, here's the output of pvdisplay run after it boots with both drive plugged in:

   --- Physical volume ---
   PV Name               /dev/mapper/[md disk]_crypt
   VG Name               [LVM group name]
   PV Size               465.53 GiB / not usable 2.93 MiB
   Allocatable           yes (but full)
   PE Size               4.00 MiB
   Total PE              119174
   Free PE               0
   Allocated PE          119174
   PV UUID               [UUID]

When only one drive is plugged in, it eventually drops me to a BusyBox shell where pvdisplay is not available.

UPDATE: I came across this debian bug report from the end of last year that matched my problem pretty well. It appears to have been fixed, so I manually installed the updated cryptsetup, libcryptsetup1, and libpop0 packages from upstream.

Now when I boot with either disk unplugged, I don't get the error anymore, and it asks for a passphrase properly. However, it won't accept the password I've configured. With both disks plugged in, it accepts the password and boots as normal, but with either one unplugged it gets to asking for the passphrase won't accept the correct password.

UPDATE 2: The symptoms now appear identical to the bug report filed here, though I'm running Lucid. As described in the link, I can get it to boot from a single disk -- if I generate enough failures it will eventually kick me out to the BusyBox shell again, and I can run cryptsetup luksOpen /dev/md1 md1_crypt, enter the password, and then hit ctrl-d to get it to boot. This seems like a bizarre procedure to get it to boot in a degraded state. I think I'm going to wipe everything and start over. Since I haven't run into this before I'm hoping a fresh install may fix things, and if not I at least know how to boot the degraded array if I need to.

Does passing `bootdegraded=true` as a kernel boot option fix it? — DerfK, Apr 19 '11 at 01:55
My understanding is that this is exactly what the mdadm option that I mentioned above does, but I'll check grub for this tomorrow. Thanks. — nedm, Apr 19 '11 at 03:25
What does `pvdisplay` say after a normal boot vs a single-drive boot? — DerfK, Apr 19 '11 at 19:07

score 3 · Accepted Answer · answered Apr 19 '11 at 00:19

3

I think you should believe the error that says that the device does not begin with /dev/mapper. I have cryptsetup here and it works fine, but all my devices are named /dev/mapper/something.

I know you want to use the UUID to mount it, but apparently that is not supported right out of the box.

answered Apr 19 '11 at 00:19

coredump

12,573
2
34
53

Take the error at face value?! Inconceivable! ;-) So, it's loading the RAID partition as /dev/mapper/* when it finds both of the mirrored disks, but since the individual devices aren't in /dev/mapper, it won't boot either individually? Hadn't considered that, thanks. – nedm Apr 19 '11 at 03:35
How is your `/etc/crypttab`? This one points to `/dev/mapper` too, but on other computer I have a `/etc/crypttab` uses UUIDS, but on this one I use dm-raid and on the other its plain disks, so I guess its a requirement from raid devices. – coredump Apr 19 '11 at 10:13
Thanks, @coredump, it was by 'UUID=' so I switched it and rebooted. However, again when I cold-unplug one disk and boot, I get the same cryptsetup error as above with the 'disk/by-UUID/' path. Where does cryptsetup get its device info from if it isn't using crypttab? – nedm Apr 19 '11 at 18:39
See update above. – nedm Apr 19 '11 at 22:46
This got me on the right track to direct my google-fu, so I'm marking as answer. Thanks for your help! – nedm Apr 19 '11 at 23:37

Encrypted LVM on RAID1 - Failure testing, manual mount

1 Answers1

Linked