ZFS as JBOD with removable drives?

Question

Is it possible to set up a ZFS pool such that it looks something like JBOD but drives are removable from that pool?

I have read that, with the usual ZFS configurations, this isn't possible.

It seems to me that, if the used space is small enough, it would be possible to shuffle things around until one of the drives in a pool is essentially unused. Once that happens, it should be possible to remove that drive from the pool without messing things up. (It seems like it would even be possible to do this with out making the pool unavailable.)

Is there a configuration that would allow this? Maybe one that uses more than just ZFS?

Answer 1

At least ZFS On Linux, which given the tags on the question is probably what you are referring to, does not support your proposed usage at present.

As I have discussed elsewhere, ZFS does not support removing constituent devices from a striped set.

The pool itself is a striped set of one or more vdevs, where the vdevs may be single device, mirror sets or raidzN redundant sets. (In the case you describe (JBOD), the vdevs would consist of single underlying storage devices.) For brevity, the vdev level is elided in the zpool status and similar commands' output if there is only a single vdev in the pool, but it's still there and can be viewed for example using the ZFS data structures dumper/debugger zdb.

The effect of this is that once a vdev has been added to a pool, it cannot be removed except by recreating the pool from scratch and transferring the data from the old pool to the new pool.

The subject of ZFS "block pointer rewrite" (which in ZFS is needed for what you want to do) comes up every now and then, and I think it's slated to be implemented "at some point", but there are many issues that are higher priority and considerably easier to implement.

Remember that ZFS is designed and implemented as an enterprise file system. Because in enterprise settings doing what you propose would be seen as very nearly insane, I wouldn't think of it as likely that what you want to do will ever be officially supported; it might however at some point in the future become possible to do, much like using deleted sparse files as backing devices is a not-really-supported-but-still-often-suggested way of gradually increasing the effective redundancy level of raidzN vdevs.