How to reduce memory usage after setting dedup=off (after having it on for a while)?

Question

I know that setting dedup=on causes high memory usage and that it's almost never worth it (see e.g. https://serverfault.com/a/569099/208661). However I did turn it on in order to see how it works and put 20 GB on a 1.4 TB pool with a cache. The memory consumption (difference of memory usage after boot with and without zfs) is approx. 8 GB higher than with btrfs. After setting dedup=on it was noticable higher than with btrfs, but not 8 GB. Now, I turned dedup off and rebooted, but the memory consumption is still high (I'd say the same).

Will it remain high, i.e. do I have to destroy and recreate the pool in order to avoid this memory consumption, will it take time until it does down, can I do something (some sort of re-duplication) in order to speed up the process?

Answer 1

Assuming your problems stem from dedup, it will not help much, as only new data is unduplicated, old data remains deduplicated and the dedup table also remains, see http://list.zfsonlinux.org/pipermail/zfs-discuss/2013-March/007668.html :

This is a very commonly-asked question. You can technically set dedup=off on a pool, and newly-written data will not be deduped. The pool will continue to work perfectly. However, the data structures required for all the existing deduped data will not go away, and nor will any of their performance implications.

The bottom line is that if you are disabling dedup to fix slowness caused by dedup, it won't work. In that case, you will need to send the data to another storage device, destroy your pool, create a new pool with dedup off, and then restore the data. There's currently no other way.

This would be quite quick in your case, as you only have 20GB to move.

Other good resources for the details behind dedup are:

• http://constantin.glez.de/blog/2011/07/zfs-dedupe-or-not-dedupe
• http://nex7.blogspot.de/2013/03/readme1st.html (section 2)
• https://blogs.oracle.com/bonwick/entry/zfs_dedup