6
1
I've noticed that whenever I copy or move large files from my SSD, which I use as the system drive, to my HDD or to an external hard or flash drive, the speed graph shown by Windows always looks the same: the transfer speed starts at around 450 MB/s, and after a few seconds drops down to somewhere between 90 and 130 MB/s and remains stable until the end of the copy/move operation.
This sparked my curiosity, so I decided to figure out what is the cause of this. Some of my thoughts were these:
Maybe that is the actual speed at which the transfer happens
Doubtfully. While the 450 MB/s speed matches the rated speed of my SSD, considering I also have some other disk reads/writes going on in the background, there is no way a 7200 rpm hard drive is capable of keeping up with it, as the 130 MB/s speed I get later on is also the most I can expect from it. So, where does the extra data go?
The extra data is being stored in the hard drive's cache memory
This makes a bit more sense, but if I take into account the duration of the higher transfer speed, my hard drive's cache would have to be over 3 GB in size, which it definitely isn't. What else could it be?
The extra data is being stored in the RAM
This makes sense. My RAM is the only other part of my system which can match the speed of my SSD, and I have plenty of it. Let's check this theory!
I open up the Task Manager, and take a look at the Performance tab. Memory usage is stable at 3.7 GB. Then I start another 15 GB file transfer. Memory usage starts rising, and stops at 5.3 GB just as the transfer speed drops to 130 MB/s. It remains the same until the end of the file transfer(the transfer dialog closes), and then slowly drops back to the 3.7 GB level it was before the transfer.
So, my last theory is true. Further confirmation is the fact that the extra used memory is marked as Modified
.
What's the point?
My question is, what is the purpose of doing this? While I don't mind having some of my RAM used by file transfers, as even during the heaviest of my multitasking sessions I've never seen its usage go over 70%, what is the benefit of storing 1.6 GB of data which you won't be doing any kind of processing on in your RAM?
I don't see any benefit from the data integrity standpoint, as you're merely copying the files, and in the case of a power failure neither the RAM or the HDD will be particularly successful in retaining the data in transfer.
I could see the benefit being that the source disk(the SSD) is quickly freed up, so that if another process needs to perform lots of read/write operations on it, it can do so without the file transfer impeding it, but if that is the case, why not go ahead and load all 15 GB at max speed into the memory?
Also, this process misleads the user, as the file transfer keeps going even after the transfer dialog closes, because some of the data is still being copied from the memory to the hard drive. This could cause a user to plug out a removable drive while data is still being written to it, possibly corrupting the removable drive, cause not everyone bothers with safely removing hardware.
Keep in mind I haven't thoroughly tested this with removable drives, as Windows might be handling them differently, making my last point invalid.
3The point is a speed up by caching disk information in free memory. Should you need to file again then there is no need to read it from a slow device (and yes, even an SSD is still slow compared to RAM). So almost any modern OS reads and writes to a cache, and that cached info get flushed (for writes) or free'd (for reads) whenever the OS needs more free memory. – Hennes – 2015-10-18T15:40:12.200
Oh, and unplugging an external drive can be done with cached but unwritten data. 'safely remove this device' might take a while though (e.g. until all information is written). This is very noticeable when I copy from my laptops internal SSD to an external USB HDD. But as long as you follow the normal method (which should always involve safely remove this device) all is safe. – Hennes – 2015-10-18T15:42:30.827
@Hennes But the OS(Windows in this case) completely removes all of the data from the RAM as soon as the file transfer is complete, and what is the chance that you will need to access the file in the middle of a transfer operation, and that you will be accessing the exact part which is loaded into the memory at that moment? Doesn't really make sense to me. – akukas – 2015-10-18T15:49:05.880
"I've seen SCSI host adapters that can perform drive to drive transfers (without the involvement of CPU and RAM)...These are the exceptions, not the rule or in common use." Exactly, they'll be adapters with their own CPU and RAM to handle those transfers. :) – Ƭᴇcʜιᴇ007 – 2015-10-18T17:14:02.897
So, where does the extra data go?
- there is no "extra data", what you see is read/write speed, not amount of data. No need/use to read more data than can be accepted by the destination device. Copy data to /dev/null for read speed from a single device, copy data from /dev/zero for writing to a single device. – Hannu – 2015-10-18T17:20:42.700@Hannu Which read/write speed? If I'm seeing the read speed of my SSD, and my HDD can't write at that speed, since time is constant there is more data being read than is being written. So there is some extra data, and it is being read and temporarily stored in the RAM, despite the destination device not being able to accept it at that speed. – akukas – 2015-10-18T17:36:35.083
there is more data being read than is being written.
- false assumption. – Hannu – 2015-10-18T17:39:39.720The disk driver might be caching the content, the COPY-COMMAND is not reading it at that rate though – Hannu – 2015-10-18T17:45:30.307