3

To prefix, I am rather new to ZFS. My goal is to create an internal NAS system, with minimal chance of losing all my data.

I've currently acquired four 4TB disks, and I plan to run them in a single zpool with two mirrored vdevs, like so:

  pool: data
 state: ONLINE
  scan: none requested
config:

    NAME                STATE     READ WRITE CKSUM
    data                ONLINE       0     0     0
      mirror-0          ONLINE       0     0     0
        <disk 0>        ONLINE       0     0     0
        <disk 1>        ONLINE       0     0     0
      mirror-1          ONLINE       0     0     0
        <disk 2>        ONLINE       0     0     0
        <disk 3>        ONLINE       0     0     0

When extending the zpool later with more disks, would it make sense to just keep adding mirror-2, mirror-3 with two paired disks each?

MechMK1
  • 306
  • 1
  • 9

2 Answers2

2

Using multiple mirror vdevs is not an issue for zfs - at all. They provide much higher performance than raidz vdevs and faster resilver. An all-mirrors pool is easier to extend and, with recent zfs versions, even to shrink (ie: vdev removal is supported).

raidz has better space efficiency and, in its raidz2 and raidz3 versions, better resiliency. A raidz pool can be extended with at least 3 disks at time - leading to a new raidz1 vdev. However, such a configuration is not ideal: with big disks you should always used raidz2. This means that expanding a raidz2 vdev requires 4 disks, with no better space efficiency compared to mirrors (ie: 50%), but somewhat better resiliency (any two disks can fail without data loss).

All things considered, when needing good random IO performance I always use mirrors. I only use raidz2 when space efficiency is a bigger concern, and only with reasonably wide stripe (6+ disks).

As you plan to only use 4 disks and to expand later, I suggest you to use plain mirrors.

shodanshok
  • 44,038
  • 6
  • 98
  • 162
  • With my current setup, my motherboard can only handle 4 internal drives, so I will use 4x4TB as mirrors. If 8 TB storage won't do, then I can always replace single drives with larger ones, as they get cheaper. – MechMK1 Oct 03 '21 at 18:49
  • Like SSD never existed, eh ? "Good random I/O performance". World is changed. – drookie Oct 04 '21 at 02:36
  • @drookie An all-SSD pool is less IOPS-constrained, right, but I am under the impression that OP is using HDD rather than SSD. Anyway, the points remain: a raidz vdev has the IOPS of a single drive and it is trickier to expand. Moreover, with only 4 disks a single raidz2 vdev has no space efficiency advantage over plain 2x 2-ways mirrors. – shodanshok Oct 04 '21 at 11:37
  • So don't use the raidz2, use single redundancy raidz. Didn't quite get the phrase "raidz vdev has the IOPS of a single drive". Nope, it does not. – drookie Oct 06 '21 at 06:25
  • @drookie single-parity array with big drives is not good idea. Moreover, as stated before, a raidz vdev only provides the IOPS of a *single* drive (but the throughput of multiple drives). You can check [here](https://www.reddit.com/r/zfs/comments/i9fh1x/why_does_raidz_limit_a_vdev_to_the_iops_of_the/) and [here](http://nex7.blogspot.com/2013/03/readme1st.html) for more details, or even testing you array with fio (and a cold cache). – shodanshok Oct 06 '21 at 18:06
  • These articles only state that when reading or writing data chunks, somewhere in the chain of these transfers you'll get maximum speed slowed down to the speed of the slowest disk. They don't say that the overall speed will be just it. You're misinterpreting what they say. – drookie Oct 07 '21 at 09:05
  • 1
    @drookie first link: "Why does RAIDZ limit a vdev to the **IOPS of the slowest device on it?**" - second link: "1. Virtual Devices Determine IOPS. **IOPS (I/O per second) are mostly a factor of the number of virtual devices (vdevs) in a zpool. They are not a factor of the raw number of disks in the zpool.** This is probably the single most important thing to realize and understand, and is commonly not." But hey - feel free to try yourself (short story: I already did that). – shodanshok Oct 07 '21 at 11:49
  • You still don't understand, and, what's even more important, confuse the question comprising an answer (wrong in this case) with actual answer, where the author explains things. That discovery about vdevs still stands for conventional layered raids, 10 and 50. Glad that you're exploring the IT world and you want to share. The thing is - others already did. – drookie Oct 08 '21 at 20:07
1

Depends on. What you are showing is in fact a raid 10. So yeah - you can continue to grow it with more mirror vdevs. Although you should really consider using raidz, because raid10 is merely a waste of disks.

drookie
  • 8,051
  • 1
  • 17
  • 27
  • [This article](https://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs-not-raidz/) states that mirrors should be preferred over raidz. [This answer](https://serverfault.com/a/650120/66501) explains that raidz1 is really not a good idea, as resilvering can become an issue. And raidz2 has the same efficiency as mirroring. Due to hardware constraints, I can't use more than 4 data disks. – MechMK1 Oct 03 '21 at 18:51
  • 1
    The article you referring to is alarmistic. Once there was a load of those, staing that starting from some volume space, your arrayas already contain disk errors, and you should start replacing disks faster that you're insering them in the tray. These articles jongle facts and make incorrect probability theory maths. 50% of storage efficiency is awful. Considering that with zfs you only got only 80% of total space, because after that performance degrades, 50% (which is in fact even less) is an inacceptable level of storage inefficiency. I'm using raridzs in production. For years. Still there. – drookie Oct 04 '21 at 02:35