Why do my external USB hard drives become slower when I have filled them up only to a third?

Question

The read/write speeds of my external USB3 hard drives (*) are significantly higher when the disks are empty (no normal files; less then 10MB used) compared to when they are moderately full (2.5 million files; ca. 1TB used).

Here a table with speed results for the Seagate 3TB hard drive measured with the Xbench app and using the dd (**) command. In Xbench higher scores means faster disk speeds. In dd output (seconds) lower means faster:

╔═════════════╦════════╦══════╗
║ Disk config ║ XBench ║  dd  ║
╠═════════════╬════════╬══════╣
║ Empty       ║    140 ║  25s ║
║ 1/3 full    ║    100 ║  56s ║
╚═════════════╩════════╩══════╝

Therefore the performance goes down at least by 30%! Why? I use the drives to have cloned backup of OSX system on the iMac, so they contain lots of files, but all files are written in one big write task (using Carbon Copy Cloner), which means the disks should not be very fragmented. Additionally the filled drives are only filled to 30%, so there is plenty of continues free space left.

Is this speed degradation normal? What is the explanation for this? Is there a formula which tells me the speed reduction as a function of how filled up the drive is?

---

(*) The hard drives I use are Seagate Expansion 2TB and WD My Book Studio 3TB. They are directly connected to my iMac via USB3.

(**) Here the dd command I'm testing the (write) speed with: dd bs=1024k count=4096 if=/dev/zero of=/Path/To/Volume/test

Answer 1

The performance degradation is probably due to filesystem overhead since you have so many small files.

Try replacing your 1TB of data comprised of 2.5 million files with 1TB of data comprised of only a few very large files and then run the same benchmark.

As @Josh Vazquez mentioned, HDDs do lose performance as they fill up, but based on what I've seen it should only be roughly around a 10% decrease when the drive is about 1/3rd full, not 30% or more like you're seeing.