1

After a hard reboot, my raid10 array (made of 4 disk partitions) failed to reassemble. After many trials, i attempted the "nuclear" option of zeroing the superblocks and re-creating the array with the --assume-clean option:

% cat /proc/mdstat
Personalities : [raid0] [raid1] [raid10] [raid6] [raid5] [raid4] [linear] [multipath]
md3 : active raid10 sda4[0] sdd4[3] sdc4[2] sdb4[1]
      951117824 blocks super 1.2 512K chunks 2 near-copies [4/4] [UUUU]
      bitmap: 0/8 pages [0KB], 65536KB chunk


% mdadm --zero-superblock /dev/sd[abcd]4 
% mdadm --create /dev/md3 --level=raid10 --raid-devices=4 --assume-clean /dev/sda4 /dev/sdb4 /dev/sdc4 /dev/sdd4

This "worked", but after rebooting LVM failed to find any of the LVM partitions that are on /dev/md3, which probably means that the array was not re-created correctly.

Since the order in which the disk partitions are presented to mdadm --create is important, i wrote a little script that tried all 24 possible permutations in the order of sda4, sdb4, sdc4 and sdd4, followed in each case by pvscan (rather than a reboot!) to see whether the LVM physical volume on md3 was actually detected, i.e. something like:

mdadm --stop /dev/md3
mdadm --create --assume-clean /dev/md3 --level=10 --raid-devices=4 /dev/sda4 /dev/sdb4 /dev/sdc4 /dev/sdd4
pvscan

mdadm --stop /dev/md3
mdadm --create --assume-clean /dev/md3 --level=10 --raid-devices=4 /dev/sda4 /dev/sdb4 /dev/sdd4 /dev/sdc4
pvscan

mdadm --stop /dev/md3
mdadm --create --assume-clean /dev/md3 --level=10 --raid-devices=4 /dev/sda4 /dev/sdc4 /dev/sdb4 /dev/sdd4
pvscan

......

Unfortunately, pvscan never detected the LVM volume on md3, though i am not sure that it is supposed to be able to do so for a suddenly activated RAID array... Also, there is no non-LVM partition on /dev/md3 that i could attempt to mount directly for testing.

I have found several cases where people have been able to reassemble arrays after zero-superblocking the disks, e.g. Recover RAID 5 data after created new array instead of re-using or mdadm rebooted, array missing? can't assemble? . This makes me hopeful that there is a way to resurrect this array unless it was really damaged during the hard reboot, in which case i would like to know it for sure. Thanks in advance for any suggestions!

orionbelt
  • 11
  • 1
  • 1
    Why do you believe that whatever erased your RAID metadata didn't also erase the contained LVM metadata? Your next step should be to go to your backup. – Michael Hampton Apr 19 '19 at 02:07
  • Well, i do not believe anything at this point, but what erased my RAID metadata was presumably `mdadm --zero-superblock`, and i would naively think that LVM metadata are stored at a different location since RAID != LVM ... – orionbelt Apr 19 '19 at 15:25
  • I’m not sure that —create implies —run. Check that your array is shown in /proc/mdstat after you create it each iteration. If the array itself is partitioned, maybe try a partprobe too. – Mike Andrews Apr 20 '19 at 13:18

0 Answers0