Reassemble RAID 1 array from old system

Question

I recently upgraded my OS from RHEL 5 to 6. To do so, I installed the new OS on new disks, and I want to mount the old disks. The old disks are listed as /dev/sdc and sdd in the new system, they were created as a RAID 1 array using LVM, using the default setup from the RHEL install GUI.

I managed to mount the old disks and use them for the last two weeks, but after a reboot, they did not remount, and I can't figure out what to do to get them back on line. I have no reason to believe there is anything wrong with the disks.

(I'm in the process of doing dd copy of the disks, I have an older backup, but I hope I don't have to use either of these...)

Using fdisk -l :

# fdisk -l

Disk /dev/sdb: 300.1 GB, 300069052416 bytes
255 heads, 63 sectors/track, 36481 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00042e35

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *           1       30596   245760000   fd  Linux raid autodetect
/dev/sdb2           30596       31118     4194304   fd  Linux raid autodetect
/dev/sdb3           31118       36482    43080704   fd  Linux raid autodetect

Disk /dev/sda: 300.1 GB, 300069052416 bytes
255 heads, 63 sectors/track, 36481 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00091208

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1       30596   245760000   fd  Linux raid autodetect
/dev/sda2           30596       31118     4194304   fd  Linux raid autodetect
/dev/sda3           31118       36482    43080704   fd  Linux raid autodetect

Disk /dev/sdc: 640.1 GB, 640135028736 bytes
255 heads, 63 sectors/track, 77825 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00038b0e

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1       77825   625129281   fd  Linux raid autodetect

Disk /dev/sdd: 640.1 GB, 640135028736 bytes
255 heads, 63 sectors/track, 77825 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00038b0e

   Device Boot      Start         End      Blocks   Id  System
/dev/sdd1               1       77825   625129281   fd  Linux raid autodetect

Disk /dev/md2: 4292 MB, 4292804608 bytes
2 heads, 4 sectors/track, 1048048 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/md1: 251.7 GB, 251658043392 bytes
2 heads, 4 sectors/track, 61439952 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/md127: 44.1 GB, 44080955392 bytes
2 heads, 4 sectors/track, 10761952 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

And then

# mdadm --examine /dev/sd[cd]
mdadm: /dev/sdc is not attached to Intel(R) RAID controller.
mdadm: /dev/sdc is not attached to Intel(R) RAID controller.
/dev/sdc:
          Magic : Intel Raid ISM Cfg Sig.
        Version : 1.1.00
    Orig Family : 8e7b2bbf
         Family : 8e7b2bbf
     Generation : 0000000d
     Attributes : All supported
           UUID : c8c81af9:952cedd5:e87cafb9:ac06bc40
       Checksum : 014eeac2 correct
    MPB Sectors : 1
          Disks : 2
   RAID Devices : 1

  Disk01 Serial : WD-WCASY6849672
          State : active
             Id : 00010000
    Usable Size : 1250259208 (596.17 GiB 640.13 GB)

[Volume0]:
           UUID : 03c5fad1:93722f95:ff844c3e:d7ed85f5
     RAID Level : 1
        Members : 2
          Slots : [UU]
    Failed disk : none
      This Slot : 1
     Array Size : 1250258944 (596.17 GiB 640.13 GB)
   Per Dev Size : 1250259208 (596.17 GiB 640.13 GB)
  Sector Offset : 0
    Num Stripes : 4883824
     Chunk Size : 64 KiB
       Reserved : 0
  Migrate State : idle
      Map State : uninitialized
    Dirty State : clean

  Disk00 Serial : WD-WCASY7183713
          State : active
             Id : 00000000
    Usable Size : 1250259208 (596.17 GiB 640.13 GB)
mdadm: /dev/sdd is not attached to Intel(R) RAID controller.
mdadm: /dev/sdd is not attached to Intel(R) RAID controller.
/dev/sdd:
          Magic : Intel Raid ISM Cfg Sig.
        Version : 1.1.00
    Orig Family : 8e7b2bbf
         Family : 8e7b2bbf
     Generation : 0000000d
     Attributes : All supported
           UUID : c8c81af9:952cedd5:e87cafb9:ac06bc40
       Checksum : 014eeac2 correct
    MPB Sectors : 1
          Disks : 2
   RAID Devices : 1

  Disk00 Serial : WD-WCASY7183713
          State : active
             Id : 00000000
    Usable Size : 1250259208 (596.17 GiB 640.13 GB)

[Volume0]:
           UUID : 03c5fad1:93722f95:ff844c3e:d7ed85f5
     RAID Level : 1
        Members : 2
          Slots : [UU]
    Failed disk : none
      This Slot : 0
     Array Size : 1250258944 (596.17 GiB 640.13 GB)
   Per Dev Size : 1250259208 (596.17 GiB 640.13 GB)
  Sector Offset : 0
    Num Stripes : 4883824
     Chunk Size : 64 KiB
       Reserved : 0
  Migrate State : idle
      Map State : uninitialized
    Dirty State : clean

  Disk01 Serial : WD-WCASY6849672
          State : active
             Id : 00010000
    Usable Size : 1250259208 (596.17 GiB 640.13 GB)

Trying to assemble:

# mdadm --assemble /dev/md3 /dev/sd[cd]
mdadm: no RAID superblock on /dev/sdc
mdadm: /dev/sdc has no superblock - assembly aborted

I've tried:

# mdadm --examine --scan /dev/sd[cd]
ARRAY metadata=imsm UUID=c8c81af9:952cedd5:e87cafb9:ac06bc40
ARRAY /dev/md/Volume0 container=c8c81af9:952cedd5:e87cafb9:ac06bc40 member=0 UUID=03c5fad1:93722f95:ff844c3e:d7ed85f5

And adding this to the /etc/mdadm.conf file, but it doesn't seem to help. I'm not sure what to try next. Any help would appreciated.

EDIT 1: Does "Magic : Intel Raid ISM Cfg Sig." indicate that I need to use dmraid?

EDIT 2: As suggested below, I tried dmraid, but I don't know what the response means:

# dmraid -ay
RAID set "isw_cdjaedghjj_Volume0" already active
device "isw_cdjaedghjj_Volume0" is now registered with dmeventd for monitoring
RAID set "isw_cdjaedghjj_Volume0p1" already active
RAID set "isw_cdjaedghjj_Volume0p1" was not activated

EDIT 2b: So, now I can see something here:

# ls /dev/mapper/
control  isw_cdjaedghjj_Volume0  isw_cdjaedghjj_Volume0p1

but it doesn't mount:

# mount /dev/mapper/isw_cdjaedghjj_Volume0p1 /mnt/herbert_olddrive/
mount: unknown filesystem type 'linux_raid_member'

EDIT 2c: Ok, maybe this might help:

# mdadm -I /dev/mapper/isw_cdjaedghjj_Volume0
mdadm: cannot open /dev/mapper/isw_cdjaedghjj_Volume0: Device or resource busy.

# mdadm -I /dev/mapper/isw_cdjaedghjj_Volume0p1
#

The second command returns nothing. Does this mean anything or am I way off track?

EDIT 3: /proc/mdstat:

# cat /proc/mdstat
Personalities : [raid1]
md127 : active raid1 sda3[1] sdb3[0]
      43047808 blocks super 1.1 [2/2] [UU]
      bitmap: 0/1 pages [0KB], 65536KB chunk

md1 : active raid1 sda1[1]
      245759808 blocks super 1.0 [2/1] [_U]
      bitmap: 2/2 pages [8KB], 65536KB chunk

md2 : active raid1 sda2[1]
      4192192 blocks super 1.1 [2/1] [_U]

unused devices: <none>

md1 and md2 are raid arrays on sda and sdb, which are used by the new OS.

Have you tried `mdadm --assemble /dev/md3 /dev/sd[cd]1`? Also, could you edit into your question the output of `cat /proc/mdstat` - that `md127` looks like it might be the right thing already. — MadHatter, Jul 08 '14 at 11:19
Yes: the message is `mdadm: cannot open device /dev/sd[cd]1: No such file or directory` — RD Miles, Jul 08 '14 at 11:21
Also tried the same thing using sd[cd] without the 1, and get: `mdadm: no RAID superblock on /dev/sdc` — RD Miles, Jul 08 '14 at 11:23
That's weird, because you've shown us those partitions. Could you try it without the glob, just do `mdadm --assemble /dev/md3 /dev/sdc1 /dev/sdd1`? Also, still waiting for the `cat /proc/mdstat`. — MadHatter, Jul 08 '14 at 11:27
Same response: `mdadm: cannot open device /dev/sdc1: No such file or directory`... I don't really understand what Volume0 means in the --examine output: I think it indicates that there is a VolumeGroup, but I'm not sure how to set it up. — RD Miles, Jul 08 '14 at 11:34

Dani_l · Answer 1 · 2014-07-08T05:17:20.933

1

A bit confused here - is it mdadm raid or lvm raid? In the question you mention lvm raid, yet keep trying to use mdadm raid.

for lvm - first use

pvscan -u

possibly

pvscan -a --cache /dev/sdc /dev/sdd

would be enough to recreate your device. if not, use

vgchange -ay VolGroup00

or

vgcfgrestore VolGroup00

The other possibility is that you used dmraid - can you try

dmraid -ay

but the disks must be connected to the intel fakeraid controller (make sure raid is enabled in bios for the ata slots the disks are connected to)

edited Jul 08 '14 at 05:17

answered Jul 08 '14 at 00:13

Dani_l

498
2
8

I believe it is a LVM over a RAID1... but I could be wrong. pvscan -u _UUID_ produces no results, nor do the subsequent suggestions. Do I need to rebuild the physical volumes from scratch? – RD Miles Jul 08 '14 at 01:37
Using vgscan -vvvv, I find this phrase "_#filters/filter-partitioned.c:45 /dev/sdd: Skipping: Partition table signature found_" ... does this help? Should I delete the partition table? – RD Miles Jul 08 '14 at 02:08
Quite possibly you had a dmraid setup using the bios "raid" capability. – Dani_l Jul 08 '14 at 05:19
I added the output from `dmraid -ay`, @Dani_l, can you make sense of it? – RD Miles Jul 08 '14 at 11:44
looks like dmraid is catching the already active sd[ab] devices. A stupid question - what's the result of mdadm -I /dev/md/Volume0 ? That's a capital i, btw, not lowercase L – Dani_l Jul 08 '14 at 15:26
`mdadm: stat failed for /dev/md/Volume0: No such file or directory.` – RD Miles Jul 08 '14 at 15:28
The message at the top of the `--examine` output, `mdadm: /dev/sdc is not attached to Intel(R) RAID controller.` ... does that indicate that it _should_ be attached to a Intel(R) RAID controller? And if so, why did it work before my reboot? – RD Miles Jul 08 '14 at 15:49
actually, I thought it did show attachment to intel controller - hint the metadata on /dev/sdc is imsm – Dani_l Jul 08 '14 at 15:52
True. Then what is mdadm complaining about? – RD Miles Jul 08 '14 at 15:54
but seems like /dev/sdd might not be. can you check in bios or during boot if you have interactive intel management at POST? – Dani_l Jul 08 '14 at 15:54
The same message appears for both sdc and sdd. I'll look into the bios. – RD Miles Jul 08 '14 at 15:55
@Dani_I - The motherboard does have LSI Logic Embedded SATA RAID and Intel® Matrix Storage Manager. However, the jumpers have been set to LSI, and LSI has been turned on in the BIOS. – RD Miles Jul 08 '14 at 19:19
Can you invoke the LSI console during reboot, before OS? and make sure the disks are configured as mirrored devices there? you might have to press ctrl-m for lsi (or ctrl-i for intel console). try to enter both and see which configurations exists there – Dani_l Jul 09 '14 at 20:16

score 0 · Accepted Answer · answered Jul 14 '14 at 17:46

It appears that I had a conflict between the dmraid setup and the mdadm setup. I don't understand the details, but what I finally did to fix it was stop the dmraid

dmraid -an

and then assemble the drives to a whole new md device:

mdadm --assemble /dev/md4 /dev/sdc /dev/sdd

When I did this, /dev/md126 and /dev/md126p1 mysteriously appeared (mysterious to me, but I'm sure someone can explain it), and I mounted md126p1:

mount /dev/md126p1 /mnt/olddrive

And voilà: my data reappeared! There were a couple of corrupted files, but no data loss.

Thank you @Dani_l and @MadHatter for your help!

Reassemble RAID 1 array from old system

2 Answers2