Restore ZFS pool after creating over it

2

1

I reinstalled my Ubuntu 14.04 system and tried to mount my existing zfs drive. It's a single drive with all data on it. The OS is on a different drive, the zfs drive only holds data.

The pool name is crystal.

In my haste (yes.) I did zpool create crystal /dev/disk-by-id/...etc...

The pool is now empty.

Is there any way to restore the original pool and/or retrieve the data?

remmelt

Posted 2015-10-11T22:33:45.663

Reputation: 23

Answers

1

Unfortunately: No, not really.

There are, somewhat unintuitively, primarily two destructive operations in ZFS: zfs destroy and zpool create. Specifically, the command zpool destroy is not destructive, and in fact is quite easily reversible if you only realize your mistake quickly; it simply sets a flag, in which case zpool import -D is your friend.

These commands are destructive for similar, but slightly different reasons:

  • zfs destroy is destructive because it removes the references to the dataset that is being destroyed, so there's no way to find it on disk using any standard tools. (But see also this question.)
  • zpool create is destructive because it overwrites all of the überblocks of the pool, which means there is no way for the ZFS code to find the beginnings of the on-disk Merkle tree data structures. Once the root of the tree cannot be found, the rest of the data becomes inaccessible.

Both of these are exacerbated by ZFS' complex on-disk format, which further complicates any recovery attempts. With most other file systems files will generally be contiguous, or in a small number of distinct locations. With ZFS, it's perfectly possible for data to be spread literally all over the disk, with nothing left after either of the above commands to indicate which parts belong where. To make matters even worse, you are certainly looking for compressed blocks of data, and possibly for things like deduplicated data as well. (Deduplication is not likely in your scenario, but it's certainly a possibility in the general case.)

It might be possible to enlist some ZFS experts in recovering your data, but ordinary data recovery services are unlikely to be able to help you at this point, and commonly available file system recovery software don't even understand ZFS. Assuming you can find people willing to work with you, it will also be very expensive; I have seen mentions of prices starting in the five-digit dollars range for similar scenarios.

Your best choice at this point is probably to restore a recent backup of your data and move on with life.

I'm honestly a bit puzzled why you were able to overwrite the pool with a command resembling the one stated in your question. On my system, with ZFS On Linux 0.6.4, having constructed a similar scenario, zpool refuses the create operation with an "invalid vdev specification" error stating that the backing device is part of exported pool '...'. You should have got a similar error, unless you explicitly passed -f to zpool to force the operation, in which case you are telling ZFS that yes, you actually really truly want to do this. Which especially in the case of ZFS is usually not what you want to do.

a CVn

Posted 2015-10-11T22:33:45.663

Reputation: 26 553

I may or may not have added -f (yes, yes I did. Sometimes I shouldn't be allowed near computers.) – remmelt – 2015-10-20T14:24:29.947

@remmelt Well, in that case, I'm sorry. I hope you had a recent backup. – a CVn – 2015-10-20T17:43:00.263

Try zpool import -D at least to see if any metadata can still be recognized. If not, you're out of luck unless you enjoy spending months coding your own ZFS data recovery software. – qasdfdsaq – 2015-10-20T17:51:27.290

@qasdfdsaq The problem is that the OP's situation is a single disk pool that has been zpool created on top of. Any metadata that the standard tools will be able to find at that point will relate to the newly created pool, not the old pool. If the OP had multiple disks in the original pool and at least one of them hadn't had the pool metadata overwritten on, there might have been some small bit of hope of being able to use the standard tools to get something back. Remember -D to zpool import just says to list pools flagged as destroyed; it doesn't make ZFS go hunting for old pools. – a CVn – 2015-10-20T17:54:19.900

It's possible the partition offsets were different for example. – qasdfdsaq – 2015-10-20T18:48:15.100