Why does certain files and collection of files take so long to copy?

Question

I was copying the windows "users" folder for a friend of mine and I noticed that the 6.5GB file took unusually long to copy. The transfer rate never exceeded 5Mb/s. Also, I noticed there was an unusually large amount of folders: 30,000-40,000. Does a high number of files, folders and subfolders have a bad effect on transfer speed?

The transfer mentioned was done between an SSD and a USB 3.0 flash drive.

Answer 1

When you copy data, the ideal case is having one sequential file. Instead, having a myriad of small files kills performance. A mechanical disk has to keep seeking for the right sector, and this is very time-consuming; an SSD is much faster, but still it suffers. If you check an SSD benchmark (I recommend Anandtech) you'll see that sequential read is often around 500 MB/s (assuming a SATA 3 connection, which is the most common), whereas reading chunks of 4KB is much slower, less than 100 MB/s.

Moreover, you are copying from an SSD to a USB drive. The latter is probably your bottleneck: consider that a USB 3.0 interface doesn't guarantee that the device will operate at that speed, but rather that it could transfer data up to that speed. Many cheap USB drives have a low quality memory and/or controller, and they are slow, so slow that USB 3.0 is actually useless as they couldn't even saturate USB 2.0.

In your case you are probably seeing a combination of the 2 things, the many files and the slow USB drive. The result is that the operation takes longer than other file copies.