zfs remove wrongly added device

Question

I accidentally added a device as normal pool device instead of cache device. Is there a way undo this? (I basically forgot to type cache when adding the device)

I can't remove it

root@dmi-vault2:~# zpool remove tank ata-INTEL_SSDSC2BA100G3_BTTV24540538100FGN-part2
cannot remove ata-INTEL_SSDSC2BA100G3_BTTV24540538100FGN-part2: only inactive hot spares, cache, or log devices can be removed

I can't detach it

root@dmi-vault2:~# zpool detach tank ata-INTEL_SSDSC2BA100G3_BTTV24540538100FGN-part2
cannot detach ata-INTEL_SSDSC2BA100G3_BTTV24540538100FGN-part2: only applicable to mirror and replacing vdevs

Also I can not just add it as cache

root@dmi-vault2:~# zpool add tank cache ata-INTEL_SSDSC2BA100G3_BTTV24540538100FGN-part2 -f
invalid vdev specification
the following errors must be manually repaired:
/dev/disk/by-id/ata-INTEL_SSDSC2BA100G3_BTTV24540538100FGN-part2 is part of active pool 'tank'

This is how my pool currently looks. The line of interest is ata-INTEL_SSDSC2BA100G3_BTTV24540538100FGN-part2 DEGRADED 0 0 0 external

root@dmi-vault2:~# zpool status
  pool: tank
 state: DEGRADED
status: One or more devices has experienced an unrecoverable error.  An
    attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
    using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://zfsonlinux.org/msg/ZFS-8000-9P
  scan: scrub repaired 0B in 22h29m with 0 errors on Mon Jun 29 05:17:12 2020
config:

    NAME                                                STATE     READ WRITE CKSUM
    tank                                                DEGRADED     0     0     0
      raidz3-0                                          ONLINE       0     0     0
        wwn-0x5000cca098c6e83f                          ONLINE       0     0     0
        wwn-0x5000cca098c6ddb1                          ONLINE       0     0     0
        wwn-0x5000cca098cadb96                          ONLINE       0     0     0
        wwn-0x5000cca098c9bdeb                          ONLINE       0     0     0
        wwn-0x5000cca098ca921b                          ONLINE       0     0     0
        wwn-0x5000cca098cadb0f                          ONLINE       0     0     0
        wwn-0x5000cca098cab2aa                          ONLINE       0     0     0
        wwn-0x5000cca098ca7e77                          ONLINE       0     0     0
        wwn-0x5000cca098ca9275                          ONLINE       0     0     0
        wwn-0x5000cca098ca9116                          ONLINE       0     0     0
        wwn-0x5000cca098cad20c                          ONLINE       0     0     0
        wwn-0x5000cca098ca91b3                          ONLINE       0     0     0
        wwn-0x5000cca098cab248                          ONLINE       0     0     0
        wwn-0x5000cca098c2c3d9                          ONLINE       0     0     0
      raidz3-1                                          ONLINE       0     0     0
        wwn-0x5000cca23de10b98                          ONLINE       0     0     0
        wwn-0x5000cca23de204aa                          ONLINE       0     0     0
        wwn-0x50014ee20898b856                          ONLINE       0     0     0
        wwn-0x5000cca23ddf44a5                          ONLINE       0     0     0
        wwn-0x5000cca23de15a2c                          ONLINE       0     0     0
        wwn-0x50014ee208947306                          ONLINE       0     0     0
        wwn-0x5000cca23dec3f07                          ONLINE       0     0     0
        wwn-0x5000cca23de1b8d3                          ONLINE       0     0     0
        wwn-0x5000cca23ddf91c1                          ONLINE       0     0     0
        wwn-0x5000cca23ddd65f1                          ONLINE       0     0     0
        wwn-0x5000cca23ddf0a14                          ONLINE       0     0     0
        wwn-0x5000cca23ddd65d2                          ONLINE       0     0     0
        wwn-0x5000cca23ddf424d                          ONLINE       0     0     0
        wwn-0x5000cca23ddd6109                          ONLINE       0     0     0
      raidz3-2                                          ONLINE       0     0     0
        wwn-0x5000cca098c1f945                          ONLINE       0     0     0
        wwn-0x5000cca098c20eb4                          ONLINE       0     0     0
        wwn-0x5000cca098c20909                          ONLINE       0     0     0
        wwn-0x5000cca098c24661                          ONLINE       0     0     0
        wwn-0x5000cca098c20d3d                          ONLINE       0     0     0
        wwn-0x5000cca098c1f2ef                          ONLINE       0     0     0
        wwn-0x5000cca098c208f3                          ONLINE       0     0     0
        wwn-0x5000cca098c23e41                          ONLINE       0     0     0
        wwn-0x5000cca098c2063c                          ONLINE       0     0     0
        wwn-0x5000cca098c20784                          ONLINE       0     0     0
        wwn-0x5000cca098c1f6b5                          ONLINE       0     0     0
        wwn-0x5000cca098c20087                          ONLINE       0     0     0
        wwn-0x5000cca098c20925                          ONLINE       0     0     0
        wwn-0x5000cca098c257bd                          ONLINE       0     0     0
      ata-INTEL_SSDSC2BA100G3_BTTV24540538100FGN-part2  DEGRADED     0     0     0  external device fault
    logs
      ata-INTEL_SSDSC2BA100G3_BTTV24540538100FGN-part1  ONLINE       0     0     0
    spares
      sdaj                                              FAULTED   corrupted data
      sdak                                              FAULTED   corrupted data
      sdaq                                              FAULTED   corrupted data

errors: No known data errors

Sucks when this happens. You just have be extra careful. – ewwhite Jun 30 '20 at 10:15 — ewwhite, Jun 30 '20 at 10:15

score 3 · Answer 1 · answered Jun 29 '20 at 12:52

Unfortunately the short answer is no, you can't remove such a device.

ZFS < 0.8.x does not support removing a non-cache/slog vdev at all, while 0.8.x does support a limited vdev removal which would not suffice for your needs because it does not work with RAIDZ vdevs.

Example on a test pool:

[root@localhost test]# zpool status zzz
  pool: zzz
 state: ONLINE
  scan: none requested
config:

        NAME                  STATE     READ WRITE CKSUM
        zzz                   ONLINE       0     0     0
          raidz1-0            ONLINE       0     0     0
            /root/test/disk1  ONLINE       0     0     0
            /root/test/disk2  ONLINE       0     0     0
            /root/test/disk3  ONLINE       0     0     0
          /root/test/disk4    ONLINE       0     0     0

errors: No known data errors

[root@localhost test]# zpool remove zzz /root/test/disk4
cannot remove /root/test/disk4: invalid config; all top-level vdevs must have the same sector size and not be raidz.

Sorry, I can feel the pain...

zfs remove wrongly added device

1 Answers1