6

Using MegaRAID Storage Manager 13.04 on an LSI 9260 RAID card, which is the correct way to create RAID 10 using 8 drives with maximum redundancy?

There appears to be multiple ways to reach the same amount of expected disk space.

Spanned Drive Group 0, RAID 10
 Virtual Drive 0, VD_0, 2.181 TB, Optimal
   Drives:
     Span 0:
      Slot 0 SAS 558.912 GB
      Slot 1 SAS 558.912 GB
      Slot 2 SAS 558.912 GB
      Slot 3 SAS 558.912 GB
     Span 1:
      Slot 4 SAS 558.912 GB
      Slot 5 SAS 558.912 GB
      Slot 6 SAS 558.912 GB
      Slot 7 SAS 558.912 GB

Reports as RAID 10 with 2.181 TB usable

Spanned Drive Group 0, RAID 10
 Virtual Drive 0, VD_0, 2.181 TB, Optimal
   Drives:
     Span 0:
      Slot 0 SAS 558.912 GB
      Slot 1 SAS 558.912 GB
     Span 1:
      Slot 2 SAS 558.912 GB
      Slot 3 SAS 558.912 GB
     Span 2:
      Slot 4 SAS 558.912 GB
      Slot 5 SAS 558.912 GB
     Span 3:
      Slot 6 SAS 558.912 GB
      Slot 7 SAS 558.912 GB

Reports as RAID 10 with 2.181 TB usable

Which is the correct configuration to allow one of each mirrored pair to fail before the array fails?

lunitin
  • 161
  • 1
  • 1
  • 2
  • UPDATE: After visiting the colo and pulling drives, these setups are functionally equivalent. Pulling every other drive (1, 3, 5, 7) left a working setup in both configurations. – lunitin Jun 24 '13 at 16:09
  • 1
    This is wrong... The main issue is what happens if *two* drives fail. – ewwhite Jun 24 '13 at 20:48
  • No it's not - I concur with the original poster. In both set-ups, slot 0 & 1 are the mirror, 2 & 3 next mirror pair, 4 & 5 the next. I really am perplexed why it works like this as it doesn't make sense. Span 1 is NOT the mirror set for span 0. I know it feels like it should be but it's not! Somebody on Spiceworks said that he felt LSI Logic really confuses the set-up of RAID-10. I don't know why you need two spans as they are nothing to do with mirroring... they are to do with striping AFAIK – Rob Nicholson Jun 25 '13 at 23:53

3 Answers3

8

First off all, see this post: What are the different widely used RAID levels and when should I consider them

Notice the difference between RAID 10 and RAID 01.

Match this to your setups (both of them labeled as RAID 10 in your text). Look carefully.

Read the part in the link I posted where it states:
RAID 01 Good when: never Bad when: always

I think your choice should be obvious after this.

Edit: Stating things explicitly:

Your first setup is a mirrored pair of 4 drive stripes. 

  Span 0:  4 drives in RAID0  \
                               }  Mirror from span 0 and span 1
  Span 1:  4 drives in RAID0  /

If any drives fails in a stripe then that stripe of lost.
In your case this means:

1 drive lost -> Working in degraded mode.

2 drives lost. Now we use some math.
If the drive fails in the the same span/stripe then you have the same result as 1 drive lost. Still degraded. One of the spans is off-line

If the second drive happens in the other span/stripe: Whole array off line. Consider your backups. (You did make and test those, right?)

The chance that a second drive fails in the wrong span is 4/7th (4 drives left in the working span, each of which can fail) and only 3/7th that is fails in the span which is already down. Those are not good odds.


Now the other setup, with a stripe of 4 mirrors.

1 drive lost (any of the 4 spans): Array still works.

2 drives lost: 85% chance that the array is still working. That is a lot better then in the previous case (which was 4/7th, or 57%).


TL:DR: use the second configuration: It is more robust.

Community
  • 1
Hennes
  • 4,772
  • 1
  • 18
  • 29
  • I'm not sure you are correct here about the first setup. When you add disks to the span, they have to added in pairs as that is the mirror pair. I have confirmed this today on a configuration with 2 spans of 16 disks. Slot 0 & 1 the first two disks in span 0. Pull disk from slot 0 and then rebuild and slots 0 & 1 flash. If the mirror for slot 0 in span 0 was from the other span, a disk further down would flash – Rob Nicholson Jun 25 '13 at 23:43
  • IMO my guess is first set-up is two spanned drives of ~1GB each. In span #0, slot 0 & 1 are mirrors, then 2 & 3 are mirrors. Striping only occurs across two spanned disks. In the 2nd set-up, striping occurs across four disks and therefore should be higher performance. However, I must admit I'm guessing as all the documentation I've read today suggests going for the first set-up. It's not clear at all in the documentation – Rob Nicholson Jun 25 '13 at 23:48
  • I've currently building a new MegaRAID SAN right now so will do some tests with writing data in the first set-up. If all drives are accessed, then it suggests the some other striping is going on - maybe there is striping across a span? If so, why create two spans. It's all very confusing! – Rob Nicholson Jun 25 '13 at 23:50
  • I'm not sure Hennes, if I have a Virtual Disk 0 with 0,1,2,3 and add a Virtual Disk 1 with 4,5,6,7, then each drive group has 1 disk mirroring a disk in the other drive group. So if VD0-1 goes bad, now I need to worry if VD1-5 goes bad since it holds the remaining data. The controller would only need to mark whole thing bad if the same, corresponding sector goes bad. If it is one of the other 3 sectors, that shouldn't be a problem. Now if a vendor card uses some algorithm that fails the way you describe, they should be shot. They key is to keep track of which disks are mirrors. – Astara Apr 30 '20 at 04:10
  • Additionally, if I have a 24-disk Bay, seems it would make more sense to pair disk 0+12, 1+13...10+22. That way you spread out heat and vibration across half the disk when writing to same span. If you allocate at 0+1 you are likely going to hit both disks every time you hit 0. To rewrite both of those disks, seems like you'd have the two of them as a heat point, vs. spreading the R0 pairs away from each other. (disks11+23 left as spares). – Astara Apr 30 '20 at 04:15
  • A bit more thinking -- if you have 1 disk go bad on the 4 disk raid 0, the controller doesn't _really_ mark the hold strip as bad, just the 1 disk that went bad -- i.e. a rebuild doesn't need to rebuild the whole stripe (4 disks), but will focus efforts on the 1 disk that went bad. – Astara May 01 '20 at 23:39
4

The second configuration is what you want. Multiple RAID1 mirrors striped together.

I don't understand why some controllers give people non-viable options.

ewwhite
  • 194,921
  • 91
  • 434
  • 799
1

It seems that LSI decided to introduce their own crazy terminology: When you really want a RAID 10 (as defined since ages), you need to choose RAID 1 (!). I can confirm that it does mirroring and striping exactly the way I would expect it for a RAID 10, in my case for a 6 disk array and a 16 disk array.

Whatever you configure as RAID 10 in the LSI meaning of the term, seems to be more something like a "RAID 100", i.e., every "span" is it's own RAID 10, and these spans will be put together as a RAID 0. (Btw that's why it seems you can't define RAID 10 for numbers of disks other than multiples of 4, or multiple of 6 when using more than 3 disks.) Nobody seems to know what the the advantage of such a "RAID 100" could be, the only thing that seems to be for sure is that it has significant negative impact on performance compared to a good old RAID 10 (that LSI for whatever reason calls RAID 1).

This is the essence of the following very long thread, and I was able to reproduce the findings I mentioned above: http://community.spiceworks.com/topic/261243-raid-10-how-many-spans-should-i-use

realMarkusSchmidt
  • 540
  • 1
  • 7
  • 16