-2

Drives are very cheap these days. Would it be worth it to increase the group size from 2 to 3. I found 1TB drives for $60. I feel as if the array is very vulnerable to failure in a 2 group setup since drive failures are tightly correlated and it would be worth adding another drive to each RAID 10 group for the added redundancy (all 3 drives per group have to fail). Am I better off going this route, or sticking to a 2 drive per group setup. What would be considered "best practice".

Basically:

1TB
1TB

1TB
1TB

1TB
1TB

-vs-

1TB
1TB
1TB

1TB
1TB
1TB

I like RAID 10 setups due to their simplicity.

oooooo3333
  • 182
  • 2
  • 7
  • What is the general use for this host? This will have an effect on the responses offered. – AngryWombat Aug 15 '13 at 22:08
  • Just for regular storage such as user profiles or backups. What I'm trying to say is that yes I can get get 1 more TB by doing a 2x3 setup, yet drives are so cheap that who cares I can get the same by doing a 3x3 setup. There is a point where it becomes a waste. Obviously 5 drives in a group is way overkill and a waste of $. I'm asking what that point is. – oooooo3333 Aug 16 '13 at 12:52
  • 1
    "since drive failures are tightly correlated" -- Needs reference – Chris S Aug 21 '13 at 14:13

2 Answers2

3

If you're looking to be able to withstand double-drive failures to any disks in the array (not possible with two-member-per-mirror RAID 10s), you may want to consider RAID 6 instead.

As for what is "best practice" I'll say that you almost never see a triple mirror anywhere.

MDMarra
  • 100,183
  • 32
  • 195
  • 326
  • 1
    ZFS triple-mirrors. Sometimes we do it for read performance. – ewwhite Aug 16 '13 at 22:09
  • Triple mirror seems like such a good strategy. RAID 6 is a more complicated setup, has long rebuild times, is taxing on the hardware, and there are hardly any drive recovery tools for it. Why doesn't anyone tipple mirror? – oooooo3333 Aug 21 '13 at 13:25
  • 2
    Almost no one triple mirrors because it's wildly inefficient. If you need such high availability, usually people start clustering storage nodes at that point. RAID 6 isn't any more complicated to configure than RAID 10 and it is not taxing to a dedicated RAID controller - they have built-in XOR processors *just* for RAID 5 and RAID 6 parity calculations and the write penalty is negated by properly sizing your write cache. As far as drive recovery tools goes - they should not matter. You should be using proper backups and not relying on dodgy software-based data recovery tools. – MDMarra Aug 21 '13 at 13:46
  • @oooooo3333 (proper, useful) triple mirroring is more complex to set up than RAID 6. If your application isn't speed-critical the performance difference will be negligible (and you're using $60 1TB disks, which says "consumer-grade SATA" to me, so I'm going to go out on a limb and say whatever you're doing can't be speed-critical). – voretaq7 Aug 21 '13 at 13:56
  • I mean less complex and taxing from a computer perspective. Tripple mirroring seems like a very elegant and efficient setup. No parity to calculate, just write the same data to three dumb disks. Read is also sped up because B1, B2, and B3 can be read off of each disk simultaneously. It seems as if it would write quicker than RAID 6 (no parity) and there is no penalty to rebuild the array. The only thing you trade off is disk space. – oooooo3333 Aug 21 '13 at 14:04
  • You forget that data is striped in RAID 6, so you get read IOPS of (n-2)*IOPS_per_disk, which will usually outperform a RAID 10 in similar sized sets. RAID 10 write performance is better to the raw disks because parity calculations are unnecessary, but as I said earlier, this doesn't matter with a proper RAID controller. They have built in parity processors that handle the XOR calculations so that the CPU doesn't have to and they have battery/flash-backed write caches that acknowledge writes when they hit the cache before they are flushed to disk, so unless you can fill a 1GB+ cache [cont] – MDMarra Aug 21 '13 at 14:09
  • faster than it can be flushed to disk, write performance will be very similar between a RAID 6 and RAID 10, and I'm assuming that since you're talking about consumer SATA disks and not enterprise NL/ML-SAS disks that you don't care about performance, so your workload likely won't be able to outpace the cache of a proper RAID controller, unless of course you're a) not profiling your workload or b) not using a proper RAID controller, in which case this sounds more like a hobby project than something in a professional setting. – MDMarra Aug 21 '13 at 14:12
  • Of course, you're free to use RAID 10 with three-mirror sets. No one is going to hunt you down and un-configure your array, but you seem to have some fundamental misunderstandings and/or knowledge gaps about disk IO, RAID levels, caching, etc. – MDMarra Aug 21 '13 at 14:18
  • 1
    Not a hobby project I'm helping someone out and I don't think they NEED to spend thousands of dollars to get a good setup (it's a win server with roaming profiles). I also wanted to use this for their backup NAS. I want a nice cheap redundant setup, that's all. This isn't a database that is getting hit millions of times per day. I'm trying to help them out and save them $. I was just under the assumption a RAID 10 has pretty good performance and redundancy. I thought I could boost the redundancy a little more by triple mirroring. – oooooo3333 Aug 21 '13 at 14:23
  • Since you haven't told @ewwhite or myself what your actual hardware is, there's no way for us to tell you whether or not RAID 10 w/ triple mirror or RAID 6 makes sense. If you provide your redundancy requirements, IOPS requirements, and what hardware/software you're using we can give you an actual mathematical answer. Anything else is going to get you generalizations which you are apparently not satisfied with. Please provide these requirements, otherwise any further conversation is a waste. – MDMarra Aug 21 '13 at 14:32
  • But I will say that you get what you pay for. Do the cheapo $60 disks that you're buying support TLER? If not, are you willing to have a single failed disk potentially [hang your entire disk array](http://serverfault.com/q/331499/10472). Really, there's a reason that enterprise hardware costs as much as it does. If there weren't a benefit to enterprise class hardware, do you think people would continue blindly paying 2-10x more for it than consumer hardware? – MDMarra Aug 21 '13 at 14:35
  • My two words : Triple mirror is wildly efficient for me, in read AND write contexts, with one or MUCH MORE simultaneous users (real life ?). It makes sense : you have 3 branches to send your I/O requests. You can lose one disk per branch at the same time (which makes it 3 disks fault tolerant in best case luck :]), but 2 disks of the same group at the same time and you're done. Anyway, better than any simple stripping. And as simple as any other raid type to create, when you work with zfs... Imagine, it's even possible under a good and stable centos7, thanks to zfsonlinux ! – tisc0 Mar 21 '18 at 15:05
1

"Triple mirroring" is what you're speaking of.

The ability to execute this is a function of your RAID controller setup and/or software capabilities.

sysadmin1138
  • 131,083
  • 18
  • 173
  • 296
ewwhite
  • 194,921
  • 91
  • 434
  • 799
  • I'm not asking if it can be done. I'm asking if it's worth it from a price perspective. Obviously 5 drives in a group is way overkill and offers no more benefits than 4 drives on a price perspective. There is diminishing marginal utility to adding new drives to a group. – oooooo3333 Aug 16 '13 at 12:52
  • @oooooo3333 You haven't provided enough detail as to what equipment you're using... Disk type/make/model, controller type, hardware RAID, software RAID, operating system... So um, there's not much to say without that information. – ewwhite Aug 16 '13 at 13:03