15

Suppose you have two RAID arrays, one with N disks and one with N+1 disks. The array with N disks was formatted as a RAID 5 and left alone, while the other array was formatted as a RAID 6 before one of its disks was removed. Now both arrays have N disks, N-1 disks worth of usable storage, and can survive the loss of one (more) disk.

Besides whatever metadata the RAID controller uses, are there any differences between these two arrays (in terms of data layout, performance, and reliability)? Could I convert a RAID 6 array with one disk missing to a RAID 5 of one less expected disk with minimal "reshaping"/"rewriting"?

I know that there are different "policies"/"alignments" within RAID 5 and RAID 6, but that's probably beyond the scope of this question. Perhaps it should be assumed that both arrays use a policy that is common to both RAID levels.

shodanshok
  • 44,038
  • 6
  • 98
  • 162
ATLief
  • 299
  • 2
  • 12
  • Keep in mind that RAID5 is a [dumpster fire](https://serverfault.com/q/598955/221656) with consumer grade drives - there's not enough redundancy for it to reliably be recoverable in the event of errors. Just have a read around here for the number of folks who come crying when their RAID5 array fails a rebuild and the only answer to give is *"Sorry - your data is gone"*. Why would you ever want to downgrade RAID6 to RAID5? If you're desperate enough to need to keep this online then surely the easy answer is to just swap a good drive in and let it rebuild. Why do you not want to do this? – J... Mar 02 '22 at 19:38
  • @J... I wanted to replace a drive using a (software) RAID controller that I wasn't sure would would remain resilient to drive failure during the replacement process. If the performance of a degraded RAID 6 was similar, I might have even kept it that way since I want to eventually move to RAID 6 anyway. – ATLief Mar 02 '22 at 19:48
  • @J... I think your other remarks support the idea that RAID is not a backup, rather than RAID 5 being unsuitable for general use. With per-drive integrity checking, RAID 5 provides error correction for otherwise silent data corruption with failure resilience being a free bonus. – ATLief Mar 02 '22 at 19:50
  • Unless you're using enterprise drives, RAID5 is unsuitable - full stop. It's a coin toss whether or not it will rebuild with a failed drive due to the URE rate, so it's a coin toss whether or not you actually get any usable redundancy out of it. It's about uptime, not backups. If it rebuilds then you haven't lost uptime - just performance. If it fails, you need to pull backups and lose anything in between. That sucks, so depending on RAID5 for uptime redundancy is pointless. You might as well not use RAID at all. You need RAID6 so that UREs during rebuild don't sink you. – J... Mar 02 '22 at 19:53
  • @J... Please see my last comment as I recently updated it. Anyway, if you want to store something that is larger than a (single) available drive, you'll need some kind of striping solution. – ATLief Mar 02 '22 at 19:57
  • If you want to resilver weekly, yeah, maybe. By the time you realize there's a problem, though, it's often too late. And you're likely to still fail a rebuild if a whole drive fails. – J... Mar 02 '22 at 20:08

1 Answers1

23

Besides whatever metadata the RAID controller uses, are there any differences between these two arrays (in terms of data layout, performance, reliability)?

Yes. RAID5 uses a single, rotating parity while RAID6 uses two. You may be visualizing dedicated parity disks, but they're in fact rotating.

RAID5 should be rather straight-forward:

enter image description here

With RAID6, imagine the last disk missing:

enter image description here

diagrams courtesy of Wikipedia [*1]

The layouts obviously differ, and a degraded RAID6 requires significant extra fetching and recalculation to compensate for the missing disk in contrast to an intact RAID5 array.

For instance, instead of reading B3 directly, any three chunks out of [B1, B2, Bp, Bq] need to be read to reconstruct B3 - note that this data reconstruction is only temporary and needs to be done each time B3 is accessed. Also note that this 'read amplification' for random reads increases with the number of disks. For longer, sequential reads, the other chunks would have to be read anyway.

The degraded RAID6 array is expected to perform much worse, depending on the workload and the RAID controller implementation and caching. RAID implementations tend to be optimized for normal operation and stability, so a degraded array may perform below expectations, even with a large cache.

Could I convert a RAID6 array with one disk missing to a RAID5 of one less expected disk with minimal "reshaping"/"rewriting"?

Yes, in theory, since all data can be reconstructed. In practice that depends on the capabilities of the RAID controller at hand. Since it requires a special migration algorithm, implementation is not too likely. Personally, I don't think I've ever seen that option but then again I haven't been looking for it either.

[*1] You should note that there are different ways to map the data in RAID arrays. The above diagrams show the simplest, not optimized schemes. Other schemes would optimize specific workloads and a RAID5 scheme for sequential read access could be

A1-A2-A3-Ap
B2-B3-Bp-B1
C3-Cp-C1-C2
Dp-D1-D2-D3
Zac67
  • 8,639
  • 2
  • 10
  • 28
  • 4
    Note that the read amplification only occurs for random reads touching the missing drive without reading the rest of the stripe (and without having that part in the stripe cache from past accesses). A sequential read that hits the whole stripe won't be affected (will have to read from three drives no matter if the array is clean or degraded). The CPU overhead of reconstructing from parity is nowadays entirely negligible unless your array is built on top of fast storage like NVMe SSDs. – TooTea Mar 01 '22 at 09:56
  • @TooTea Yes, good point - I've integrated that into the answer. – Zac67 Mar 01 '22 at 10:07
  • 5
    And rotating parity is of course used so that read-only workloads can use all disks, instead of leaving the parity disk idle and it balances writes if the system can / often does write just small blocks and the associated parity change, instead of rewriting full stripes. Anyway, RAID4 is the one that uses a dedicated parity disk, and the idea of the question might work with an N-disk RAID0 and an N+1-disk RAID4 with the parity disk lost: they might well be very much alike. Of course, if you pull a random disk from a RAID4, you'd probably hit a data disk and then it's not at all the same. – ilkkachu Mar 01 '22 at 11:15
  • 5
    Some enterprise-grade RAID controllers can indeed do such a migration, but it's generally frowned upon. Transforming a degraded volume isn't usually recommended since it involves doing a *lot* of I/O and in every case except single-degraded RAID6, you're a single URE away from losing the volume. It's likely much safer to replace the drive, get the volume healthy, and *then* do the migration. – bta Mar 01 '22 at 18:51
  • 1
    Shouldn't `A1-A2-A4-Ap` be `A1-A2-A3-Ap` in your footnote? I imagine that 4 instead of 3 was a typo? – Andrew Mar 02 '22 at 18:51
  • 1
    @Andrew Yes, of course - corrected! – Zac67 Mar 02 '22 at 18:52
  • Usually, when a RAID-6 has a single failure, it recalculates using only parity, so the degradation speed is not as bad. It's when 2 drives fail that there is a lot of overhead as the Galois fields(Q) need calculating when it becomes seriously slow. – Mike Mar 02 '22 at 20:34
  • No problem, I'd have fixed it myself but I have to follow the char minimum for edits on this SE site – Andrew Mar 03 '22 at 14:38
  • LVM has a variant of RAID6 called "raid6_ls_6" which is identical to RAID5, except with an extra disk on the end that only holds parity calculated by a different function. I couldn't find this layout described anywhere else, so it may be non-standard. As long as you don't later convert that layout to a standard one, the performance will be slightly worse, but you can revert the array to RAID5 with minimal rewriting (as long as the array is not degraded, or if the missing disk was the dedicated parity). – ATLief Apr 26 '22 at 22:25