23

I have an OpenSolaris server with a zpool backupz comprised of four SCSI drives:

-bash-3.2# zpool status backupz
  pool: backupz
 state: ONLINE
 scrub: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        backupz     ONLINE       0     0     0
          raidz1    ONLINE       0     0     0
            c7t0d0  ONLINE       0     0     0
            c7t1d0  ONLINE       0     0     0
            c7t2d0  ONLINE       0     0     0
            c7t3d0  ONLINE       0     0     0

errors: No known data errors

I want to add a fifth drive... but zpool add backupz raidz c7t4d0 isn't working...

-bash-3.2# zpool add backupz raidz c7t4d0
invalid vdev specification: raidz requires at least 2 devices

Can I not have a raidz config with 5 devices? Do I have to add two devices at once? or am I doing something incorrect altogether here?

Josh
  • 9,001
  • 27
  • 78
  • 124

2 Answers2

33

See this thread from one of the FreeBSD lists: http://lists.freebsd.org/pipermail/freebsd-fs/2009-June/006327.html

To quote Dan Naumov,

To reiterate, you cant just add a single disk drive to a raidz1 or raidz2 pool. This is a known limitation (you can check with SUN ZFS docs). If you have an existing raidz and you MUST increase that particular pool's storage capabilities, you have 3 options:

1) Add a raidz of the same configuration to the pool (think 3 disk raidz + 3 disk raidz or 5 + 5, for example)

2) Replace each (and every) disk in your raidz pool one by one, letting it resilver after inserting each upgraded disk

3) Backup your data, destroy your pool and create a new raidz pool with a bigger amount of disks.

brent
  • 3,481
  • 3
  • 25
  • 37
  • 3
    +1: Thanks @brent! Your answer was more detailed than Chris' however he was providing me with detailed info in chat, so I accepted his answer. I appreciate the information! – Josh Oct 13 '10 at 22:33
  • 1
    Why must one add a RAIDZ of the same configuration and can't e.g. add a RAIDZ with 8 disks to one with 3? Isn't data striped across all VDEVs aka RAIDZs? – Thorsten Schöning Jun 15 '17 at 14:44
  • 1
    @Thorsten Schöning The answer is performance, ZFS performs best with VDEVs of the same size: https://serverfault.com/a/564436/333397 – Thorsten Schöning Jun 15 '17 at 14:53
14

You can't expand an existing raidz vdev, you have to blow it away and create it again with the new drive(s). See the other answer for better details.

Side note: Someone actually worked out that it's technically possibly to add drives to a raidz, but the functionality hasn't been implemented. The same is true of removing a disk.

Chris S
  • 77,337
  • 11
  • 120
  • 212
  • 5
    Uh oh. If that's the case I've made a terrible decision somewhere! =o – Josh Oct 12 '10 at 18:59
  • 2
    -1 Of course a zpool can be expanded. What do you think the `zpool add` command does? The limitation which Josh is running into, is that you can't change the composition of a raidz vdev ("raidz1" in his example). You can add new devices to the pool itself just fine. This is one of the reason why mirror vdevs are preferred: http://serverfault.com/questions/63809/zfs-mirror-vs-raid-z – Wim Coenen Oct 13 '10 at 13:36
  • 9
    @Wim, I'm well aware that you can add additional vdevs to a pool. It was a minor typo, and there's no reason to jump all over me like that. – Chris S Oct 13 '10 at 14:01
  • 1
    Cool, I didn't know someone found a way to work around the architecture to allow adding drives! Do you have a link where I can read more on it? – lindhe Jan 31 '16 at 22:19