20

My understanding is that an SSD has a limited amount of writes. RAID5 performs many writes due to parity information across the drives. So reasoning states that RAID5 would kill and lower the performance of Solid State Drives at a faster rate.

The following statement from This Article, makes me think I don't fully understand or might be incorrect with my above reasoning.

Another niche for high-endurance SSDs is in parity RAID arrays. SLC, due to its inherently superior write latency and endurance, is well suited for this type of application.

Damainman
  • 995
  • 5
  • 14
  • 26
  • 2
    You have to remember how many writes it takes to kill an SSD.... Something like 1 or 1.5 Million for consumer grade SSDs. – Chad Harrison Jun 06 '13 at 20:09
  • @hydroparadise Current (June 2013, 20nm MLC) consumer grade SSDs have flash rated at around 3000 write/erase cycles. They don't stop working immediately after 3000 full writes, but they will fail much sooner than a million writes. – Daniel Lawson Jun 07 '13 at 06:09

4 Answers4

13

Your reasoning is correct, though you're missing the scale of the problem.

Enterprise SSDs are being made with higher endurance MLC cells, and can tolerate very high write-rates. SLC still blows high-endurance MLC out of the water, but in most cases the lifetime write-endurance of HE-MLC exceed the expected operational lifetime of a SSD.

These days, endurance is being listed as "Lifetime Writes" on spec-sheets.

As an example of this, the Seagate 600 Pro SSD line has a listing of this, roughly:

Model   Endurance
100GB       220TB
200GB       520TB
400GB      1080TB

Given a 5 year operational life, to reach the listed endurance for that 100GB drive, you need to write 123GB to that drive per day. That may be too little for you, which is why there are even higher endurance drives on the market. Stec, OEM provider for certain top-tier vendors, has drives listed for "10x full-drive writes for 5 years". These are all eMLC device.

Yes, R5 does incur a write amplification. However, it doesn't matter under most use-cases.


There is another issue here, as well. SSDs can take writes (and reads) so fast that the I/O bottleneck moves to the RAID controller. This was already the case with spinning metal drives, but is put into stark light when SSDs are involved. Parity computation is expensive, and you'll be hard pressed to get your I/O performance out of a R5 LUN created with SSDs.

sysadmin1138
  • 131,083
  • 18
  • 173
  • 296
  • What are SLC, MLC, HE-MLC and eMLC? – mdpc Jun 06 '13 at 20:25
  • 1
    Thanks for the example and reasoning. Makes sense. My boss keeps telling me never use RAID5 with SSD, instead use RAID0, RAID1, or RAID10. Just don't understand enough to properly debate. – Damainman Jun 06 '13 at 20:36
  • 1
    @Damainman The main reason to not use R5 with SSD is you leave a lot of performance on the floor when you do that. It isn't because of wearing things out faster (anymore). – sysadmin1138 Jun 06 '13 at 20:38
  • @sysadmin1138 you leave a lot *more* performance on the floor if you use RAID1 or RAID10 though, particularly if you're talking about a decent size array (eg, 8 disks per RAID5 set). – Daniel Lawson Jun 07 '13 at 06:11
  • 1
    Also, don't use RAID0 if you care about your data. SSDs don't fail as often as spinnig rust, but they do fail. The most common failure mode in SSDs that I've seen is complete controller failure of one form or another, which means total loss - your RAID0 is now broken. – Daniel Lawson Jun 07 '13 at 06:13
  • Thank all of you for the information. I am Definitely behind on SSD technical details. – Damainman Jun 07 '13 at 19:49
  • 2
    If you care about your data, back it up.. RAID is only there to keep everything running during a disk failure. – John Hunt Oct 15 '15 at 14:49
  • Another reason to use SSD with RAID, even R5, can be much lower power-consumption, depending on drives though. Must SSD's use less than 100mv, burst 2-4 watt (SATA SSD), which is was a HHDs use in idle alone. Exmample, 4xdisk RAID5, idle: SATA-SSD:320mv, HDD:16.000mv. – MrCalvin Jun 06 '20 at 05:59
10

I found 2 research papers about this topic:

  1. Parity update increases write workload and space utilization

    Introduction

    [...] The results from our analytical model show that RAID5 is less reliable than striping with a small number of devices because of write amplification.

    Conclusion

    [...] Different factors such as the number of devices and the amount of data are explored, and the results imply that RAID5 is not universally beneficial in improving the reliability of SSD based systems

    Source: Don’t Let RAID Raid the Lifetime of Your SSD Array
    (Published 02/2012)

  2. Equal aging of all SSDs imposes risk of simultaneous failure (RAID1 & RAID6 affected too!)

    Abstract

    [...] Redundancy solutions such as RAID can potentially be used to protect against the high Bit Error Rate (BER) of aging SSDs. Unfortunately, such solutions wear out redundant devices at similar rates, inducing correlated failures as arrays age in unison. [...]

    5. Simulation Results

    [...] Conventional RAID-5 causes all SSDs age in lock-step fashion, and conventional RAID-4 does so with the data devices; as a result, the probability of data loss on an SSD failure climbs to almost 1 for both solutions as the array ages, and periodically resets to almost zero whenever all SSDs are replaced simultaneously. [...]

    Source: Differential RAID: Rethinking RAID for SSD Reliability
    (Published 03/2012)

    To protect from this the paper proposes a new RAID level called Diff-RAID that does automatically age-driven shuffling on device replacements).

    You can protect from this by manually checking the SSD wear out indicator and replacing drives proactively with spare discs so that at no time multiple discs have the same critical age.

Rowan Hawkins
  • 590
  • 2
  • 18
TegtmeierDE
  • 301
  • 1
  • 3
  • 6
8

Parity RAID will thrash your $300 desktop SATA SSD. It will not even put a dent in a $3000 enterprise grade SSD.

It's all about what you're shopping for and what your use case is. SSD is a much more mature technology than it used to be. On the high end, their MTBF and max writes are approaching the same sort of reliability as mechanical HDDs.

One reason that you may not want to use parity RAID on SSD is that you can quickly saturate a backplane or controller bus with a large many-member SSD RAID group. There are diminishing returns very quickly with the read speed of high end SSDs and the bus/backplane bandwidth of current RAID controllers. Not to mention that if these are hosting data that is dished out over the network, it's entirely possible that your network interfaces will be the bottleneck before the disk IO is when you're talking about large SSD RAIDs.

Basically, write lifetime isn't that big of a deal unless you're building your "server" from Newegg, but there are some other reasons why you may be wasting money putting SSDs into large parity RAID sets.

MDMarra
  • 100,183
  • 32
  • 195
  • 326
  • 3
    It's quite easy to hit raw sequential throughput bottlenecks when using SSDs, even on latest generation RAID controllers. However, it's a lot harder to hit random IO bottlenecks. Even if you aren't able to saturate all your SSDs sequentially, you'll still get increased performance in random workloads. – Daniel Lawson Jun 07 '13 at 06:42
  • 1
    @MDMarra thank you for the response, I did an upvote on your response due to the details provided :). – Damainman Jun 07 '13 at 19:50
0

The question does not specify operating system or controller hardware. My answer is only relevant to the Linux kernel and the software RAID system related to "mdadm".

By default it will configure the RAID with a large chunk size 128k which was originally tuned for spinning disks where seek time is a major concern. IMHO this is way too large for SSD, and especially bad when the absolute minimum write of only 1 byte will end up being a whole chunk of 128k, and in order to get redundancy you need to write a second time also 128k.

That's where the worst part of the write multiplier comes from. I would recommend a much smaller chunk, e.g. 8k. That's probably the major problem you will run into if you assemble a RAID5 array with default settings.

Make sure you have "discard" enabled (in some places called trim) because that's by default disabled on the Linux software RAID drivers.

cat /sys/module/raid456/parameters/devices_handle_discard_safely

That should be saying Y to indicate the feature is enabled.

Dave M
  • 4,494
  • 21
  • 30
  • 30
Tel
  • 1