1

Does the Windows Server 2012 Data Deduplication feature work with 3rd party defrag programs, in particular Auslogics Disk Defrag? I recently enabled the Data Deduplication feature on a drive to test it out and noticed that the drive is being reported in Auslogics as being 91% fragmented (YIKES!). I'm wondering if it's possible (and worthwhile) to defragment it in Auslogics without a high risk of data loss.

bbosak
  • 119
  • 4
  • I'm quite sure that you won't loose any data, the bigger question is whatever the data will remain deduplicated. Deduplication usually causes quite high fragmentation... – Hubert Kario Jan 06 '13 at 18:00
  • It will. Because on the file system level it is stored in separate files - one for the file, others for the fragments. Any sensible modern defrag FOR server 2012 should deal with it at least correctly. – TomTom Jan 06 '13 at 18:03
  • 3
    This question really should be "Do 3rd party defrag programs work with the Windows Server 2012 Data Deduplication feature?" – joeqwerty Jan 06 '13 at 19:06
  • 1
    @joeqwerty or "is defrag still important on a windows 2012 volume that's using the data deduplication feature" - maybe there's little point in running defrag if the volume is going to fragment easily due to the nature of the data dedup feature? – Rob Moir Jan 06 '13 at 22:03

1 Answers1

1

If it uses the Microsoft defragmentation APIs it should be able to, as the deduplication chunks and metadata are stored as plain files on the disk. If you're paranoid about data loss, just disable the dedup jobs on the volume before running it. I asked Ran Kalach, part of the dedup team at Microsoft about this, and he stated that there were no known data integrity issues with 3rd party defragmentation programs which use the Microsoft defragmentation APIs. Although there could be performance issues due to large sparse files utilized by dedup.

I've been using MyDefrag because it is highly configurable and allows you to write scripts to determine file placement and other actions. The deduplication chunks and metadata are stored in ?:\System Volume Information\Dedup. Security permissions on this directory are set to only allow NT AUTHORITY\SYSTEM access, so if you want to be able to defragment these files you will need to run your defragmentation program under the NT AUTHORITY\SYSTEM account. This can be acomplished with Microsoft/SysInternal's psexec program. Just run psexec.exe -i -s -d C:\YourDefrag.exe

To address the comments in your question regarding defragmenting a deduplicated volume is of little use, I would have to disagree. To start off not all files and directories are always deduplicated. In a default configuration several file types are excluded, see the ExcludeFolder, ExcludeFileType and ExcludeFileTypeDefault properties for the Get-DedupVolume cmdlet. This can be further configured by the administrator, for instance I exclude .MKV video files because of the low duplication rates in my environment. Also files in excess of 1TB will not be deduplicated even in Server 2016, and files 32KB or smaller will not be deduplicated either. Secondly, free space fragmentation can decrease write performance, and can increase the chance future files are fragmented. Thirdly even if a deduplicated file is inherently fragmented, a fragmented deduplication chunk will further decrease performance. And finally by grouping dedup chunks together with a program like MyDefrag you can reduce the time it takes to perform garbage collection and scrubbing jobs by reducing the amount the amount of time spent as the disks are seeking.

Also the data itself will not be rehydrated if defragmentation is ran as the user visible deduplicated files are stored as reparse points on disk - a special type of file similar to a junction or directory mount point.

Muh Fugen
  • 309
  • 1
  • 4
  • 14