2

After setup, cat /proc/mdstat output looks like this:

proxmox:~# cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : active raid1 sdc2[1] sdb2[0]
      293024832 blocks [2/2] [UU]

unused devices: <none>

Also, after I setup raid1 fresh, i got the following:

proxmox:~# mdadm --examine --scan
ARRAY /dev/md0 level=raid1 num-devices=2 UUID=fbda4051:61cbc27f:7f2b1f39:e153e83f

But, after reboot, cat /proc/mdstat outputs:

proxmox:~# cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : active (auto-read-only) raid1 sdc[1]
      293024832 blocks [2/1] [_U]

unused devices: <none>

Why is it using sdc1 now?

Also, now I get:

proxmox:~# mdadm --examine --scan
ARRAY /dev/md0 level=raid1 num-devices=2 UUID=fbda4051:61cbc27f:7f2b1f39:e153e83f
ARRAY /dev/md0 level=raid1 num-devices=2 UUID=fbda4051:61cbc27f:9822ee23:9b948649


proxmox:~# dmesg | grep md0
md/raid1:md0: active with 1 out of 2 mirrors
md0: detected capacity change from 0 to 300057427968
 md0: p1 p2
md0: p2 size 586049840 exceeds device capacity, limited to end of disk

Where did the two partition on /dev/md0 come from? I never made them. Also, sdc1 and sdc2 aren't listed in the /dev tree.

Here is the fdisk output:

proxmox:~# fdisk -l /dev/sdb

Disk /dev/sdb: 300.0 GB, 300069052416 bytes
255 heads, 63 sectors/track, 36481 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x3bd84a48

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1           2       10240   83  Linux
Partition 1 does not end on cylinder boundary.
/dev/sdb2               2       36482   293024920   fd  Linux raid autodetect
Partition 2 does not end on cylinder boundary.

proxmox:~# fdisk -l /dev/sdc

Disk /dev/sdc: 300.0 GB, 300069052416 bytes
255 heads, 63 sectors/track, 36481 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x371c8012

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1           2       10240   83  Linux
Partition 1 does not end on cylinder boundary.
/dev/sdc2               2       36482   293024920   fd  Linux raid autodetect
Partition 2 does not end on cylinder boundary.

A bit of info: Server running Proxmox v1.9, which is debian lenny 64bit. sda is the system hard drive (hard ware RAID). sdb and sdc are 300GB brand new Raptor drives.

Sparctus
  • 29
  • 1
  • 2
  • 5

2 Answers2

2

First off, check the physical hardware like connections, cables, and correctly-seated cards. For the disk itself, check SMART data on /dev/sdb to make sure the disk itself is not failing out periodically. Western Digital Raptors are fast but prone to failure, I've had one fail on me out of nowhere (not even SMART data predicted it). Use smartctl to read the SMART data and run tests. It comes in the smartmontools package:

apt-get install smartmontools

Pull the data and look for anomalies or errors logged:

smartctl -a /dev/sdb

Finally, run a manual self-test, which will take about 2 minutes. long can be substituted for short, and is more thorough, but it takes far longer ("tens of minutes"):

smartctl -t short /dev/sdb

Once the test is done, review the results: 

smartctl -l selftest /dev/sdb

If it all comes back clean, you can move on to debugging the mdadm stack.

Your partition arrangement is a bit strange on the RAID devices. If those devices will be dedicated to RAID, you don't need a partition table at all. Assuming there's no data, you'd be well advised to keep it simple and use the block devices directly. In fdisk they would show up like this:

Disk /dev/sdb: 300.0 GB, 300069052416 bytes
255 heads, 63 sectors/track, 36481 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00000000
Disk /dev/sdb doesn't contain a valid partition table

To eliminate any issues with partitioning and start from scratch, just dd some zeros at the beginning of each disk;

dd if=/dev/zero of=/dev/sdb count=128 bs=4096k

Repeat for /dev/sdc. Create the array using those two devices:

mdadm --create --raid-devices=2 --level=raid1 --bitmap=internal --assume-clean --name=RAID1 /dev/sdb /dev/sdc

Don't forget to check dmesg for any disk-related output!

Joel
  • 191
  • 1
  • hi Joel, so I did check both the disks (sdb and sdc) using smartmontools as you suggested. Both came clean. I can setup RAID1 yet again, as I've done it few times already from scratch. When I do it, it's perfect. Everything is fine as evident by cat /proc/mdstat but it's only after reboot, it doesn't build it properly. It may be of interest. The OS installation is on /dev/sda (which is provided by Adaptec Hardware RAID). When the system reboots, the bios doesn't look at /dev/sda for bootcode, it searches it in sdb/sdc. So I've to manually boot system using grub2 cmd line. – Sparctus Nov 28 '11 at 21:10
  • A simple script at boot to stop the array, then reassemble and mount it could be all that you need. – Joel Nov 28 '11 at 22:14
  • hmmm... there's an idea... although i've little idea how i'd go about scripting such a script. I'll look into it. Thanks for your help though, appreciate it. – Sparctus Nov 28 '11 at 22:43
0

Proxmox does not support software RAID:

For production only hardware raid is possible. if you use an custom softraid you will run now and in future versions into problems means it will NOT work. It´s quite interesting that people does not believe us that Proxmox VE will not work with soft raid - again: we are the developers and we should know it!?

P.S. The developers never seem to explain "why" it doesn't work... That just say that it won't.

http://forum.proxmox.com/threads/838-Install-on-Soft-Raid?p=4406#post4406

Even if "this" problem isn't due to Proxmox, it's generally not going to be worth the effort, because it won't work in the end anyway...

Soviero
  • 4,306
  • 7
  • 34
  • 59
  • proxmox is debian lenny beneath the hood. sofware RAID is integral part of linux. Of course you can have raid with proxmox, it's just that you've to do it manually yourself. Have a look at proxmox forums. People are doing it. Thanks for your input though. – Sparctus Nov 28 '11 at 21:07
  • I did look at Proxmox's response regarding software RAID and it does not seem like a good reason at all. Even so, they may be aware of some aspect of it that they have broken themselves. It is "based" on Lenny but it is not just Lenny underneath and the Proxmox stack on top. It is almost certainly a heavily-modified Lenny. If you do have hardware RAID capability, is it possible to use that for the boot array as well as the data (I assume) array? – Joel Nov 28 '11 at 22:14
  • Proxmox discourages software RAID for they system partition and I understand why. I'm using software RAID for additional storage hard drives - data only. So no problem. – Sparctus Nov 28 '11 at 22:45
  • I am looking at Proxmox's responses, and some of their forums posts give me the impression that they are incompetent idiots. OTOH, that is a 4 year old forum thread, and there does seem to be some disagreement between the advice various people the forum identifies as 'Proxmox staff'. – Zoredache Nov 29 '11 at 00:56
  • Agree completely with Zoredache – Sparctus Nov 29 '11 at 15:11