0

I've copied a folder using robocopy by using the command switches /mt /s /e /copyall /np /nfl /ndl

When I navigate to -> Right Click -> Properties I can see a significant difference in size on disk

The source folder shows 1.1GB size on disk and the destination folder much less Both directories reside in a REFS formatted system using windows software mirroring.

Should that be alerting me of an issue during copy?

Size on disk difference: Size on disk difference

Alexander Tolkachev
  • 4,513
  • 3
  • 14
  • 23
gabtzi
  • 103
  • 1
  • 5

2 Answers2

2

The 'size on disk' specifies how much disk space is used by the file. It heavily depends on the file system and especially the cluster size.

A file with a length of 8000 bytes occupies two clusters with 4Ki cluster size but only a single cluster with 8Ki or 64Ki clusters (sizes on disk 8192, 8192 and 65536 bytes respectively).

With completely random file sizes, each file wastes half a cluster in the file system. However, smaller and larger numbers of clusters increase file system overhead and may increase file fragmentation.

The 8000 byte file wastes unused 192 bytes with 4Ki or 8Ki clusters but 57536 bytes with 64Ki clusters.

Zac67
  • 8,639
  • 2
  • 10
  • 28
  • OK, so If I understand correctly since the two directories are on the exact same type of filesystem and therefore using the same cluster size and they contain the same files, fragmentation could be the reason this vast difference appears? – gabtzi Jan 21 '18 at 11:38
  • No - apparently D: and E: use different cluster sizes. 1,186,070,528 is a multiple of 64Ki (65,536) while 402,231,296 is a multiple of 4Ki (4,096), so I'm assuming these cluster sizes. 13,968 files multiplied by (64-4)Ki/2 gives ~429 MB which very roughly corresponds to the difference in size on disk (statistically, each file wasted 4Ki/2 before copying and 64Ki/2 after copying). `fsutil fsinfo ntfsinfo D:` should show you the exact cluster size. – Zac67 Jan 21 '18 at 11:55
  • Since the average file size (362,700,777 / 13,968) is significantly less than 64Ki you're wasting more space than statistics predict. – Zac67 Jan 21 '18 at 12:11
  • 1
    I just checked and indeed they hare difference cluster size. Thank you very much for the detailed explanation. I was under the assumption they have the same cluster size because of the way they were formatted from disk management using defaults but apparently in between insider builds the default cluster size changed for REFS and I didn't notice it. – gabtzi Jan 21 '18 at 12:13
1

It should also be noted (although it's probably not relevant to this particular case) that Robocopy does not preserve NTFS compression, so if there are compressed files in the source folder (they usually appear in blue color), the destination folder will be significantly larger. It may or may not be the same for “sparse” files.

SynchronizeIt is a file copy tool which I use a lot : like Robocopy, it preserves all timestamps (including directory timestamps, which is very rare among Windows tools), and it preserves the C attribute, meaning, the NTFS compression. But there's one caveat : it first copies the file uncompressed, then applies the NTFS compression, which increases the copy time. The current official version also has a bug : if an input file has the “sparse” attribute, the copy gets corrupted (only the first clusters are actually copied, the rest ends up empty). It can be a problem even for someone not knowingly using the “sparse” NTFS feature, as many download managers or file sharing applications set it by default to minimize the allocated size of partial files. In 2015 the author provided me with a corrected version, but for some reason never made it public (the available version hasn't been updated since 2009).

WinHex has a function called “Replicate directory” which can directly copy compressed files as compressed, but it does not preserve timestamps. Applying Robocopy /CREATE /DCOPY:T afterwards on the same directory corrects all timestamps very quickly without re-copying the files.

GabrielB
  • 111
  • 2