Why does HDD activity slow the whole System down (on Windows 7)?

If I copy a large amount of data from on drive to another (physical) drive my whole system is running slow.

I am using Windows 7 64bit version on an Intel Core 2 Duo 3 GHz.

This is the situation:

Very fast copying between two SATA disks.
Resource Monitor shows 80MB/s read, 70MB/s writes and 150MB/s total
The total CPU load is about 3%
There is plenty (>2GB) of free physical RAM left
All partitions have enough (>10gb, >10%) of free space left and show low fragmentation
Anti-Virus software is not active

This is the problem: No matter what I want to do while copying, it takes forever. Even tasks that do not need a lot of disk access (as they should work from RAM only), are extremely slow. The moment I stop the copying process, all the unfinished tasks finish at once. For example trying to open a new browser windows or explorer window will not work. If the copying is stopped 5 explorer windows popup at once.

My question is: why? What resource is exhausted, if CPU and memory are idle? And what part of my computer hardware I need to upgrade to get better behavior? Also read Psycogeek's comment for further questions in this direction.

If your answer is, that everything needs HDD Access, which is blocked by such an activity, then my question is: Is there a way to copy something on your system without rendering it unusable until its done? (which can be hours, if >500g have to be copied).

user4514

Posted 2011-10-20T12:24:01.330

Reputation: 1 172

4Always wondered that myself, and if there are differances depending on the motherboards/chipsets. If high data communications on the busses exist, it can choke down the whole system. It is high use of "resources" but ones they dont give us a percentage meter for ? – Psycogeek – 2011-10-20T12:31:00.547

Just a quick fact you might find useful - transfer rate @ response rate. HDD: 100 MB/s @ 10ms. SSD transfer rate: 500 MB/s @ 0.1ms. RAM transfer rate: 20,000 MB/s @ 60ns. CPU Cache: 100,000 MB/s @ 0.5ns (assumed clock speed of 2 GHz, word length 32 bits). See the bottleneck? – cp2141 – 2011-10-21T12:48:52.717

I don't think write caching helps on long writes. – ThunderByrd43 – 2011-10-25T22:14:49.073

This is just a wild hypothesis, but perhaps you need to update your chipset drivers? Also, in the Device Manager, make sure that write-caching is enabled for each of your drives. – Bigbio2002 – 2011-10-24T19:19:33.997

Answers

If you're using multiple resources on the same hard drive, then this is normal behaviour. Hard drives are big, mechanical devices which are good at one thing - only doing one thing at a time. You can only read/write to a single sector at a time, so attempts to use the hard drive simultaneously usually result in thrashing. This isn't a side effect of anything other than the hardware working as designed.

If you're doing multiple things on a mechanical hard drive at once, you might find better results by performing these actions one at a time. If you're talking about file transfers, considering replacing the Explorer file copy handler with another program which supports transfer queuing (such as TeraCopy, which will also allow you to pause the transfer if you need fast disk access temporarily).

You can also help to mitigate these effects with the use of a solid-state drive, but it is no guarantee - it only helps because of the drastically reduced random access time. You can still have thrashing with an SSD, and an SSD still has the same limitation - it can only read or write to a single sector at a time.

If an SSD isn't the route you want to go, consider using a RAM disk, performing any data processing in-place in memory, add additional drives (not partitions) for each task, or performing I/O intensive tasks on non-OS drives.

Breakthrough

Posted 2011-10-20T12:24:01.330

Reputation: 32 927

1There has to be more to it than the hard drive(s). I have 7 hard drives, and 4 are on a seperate PCI-E controller. The chokedown can occur from high disk activity unrelated to the system drive I am running a program from. . . When I had "real" raid board in PCI slot (long ago) it was not "as bad". I wish I knew If there is different chipsets in MB or something that sidebars the task better?? – Psycogeek – 2011-10-20T16:44:57.583

it acts like . . . remember when working with a CD-Rom device in a PATA system, and the system would just park waiting for a return from the device? like that. I donno. – Psycogeek – 2011-10-20T16:51:14.593

Thanks for your thoughts, but not really helping, as I said: " Even tasks that do not need a lot of disk access (as they should work from RAM only), are extremely slow" – user4514 – 2011-10-20T17:19:02.840

2@Psycogeek that depends on your particular hardware. I have 4 HDDs in my desktop, and using the latter 3 (i.e. non-system drives) rarely has an impact on my main drive, holding my OS and programs. I frequently backup gigabytes of data in the background (between those secondary drives) with no adverse effect, even if I'm playing a game. – Breakthrough – 2011-10-21T12:40:14.177

@user4514 I believe I explained that effect in my answer, but I'll expand a bit for you. Not every program is loaded entirely into memory when you run it. And even if it loaded just the executable, what about any additional library functions it needs to call? Dynamically-linked libraries? Underlying operating system function calls and interrupts? This is just the way computers work, and aside from adding more drives or upgrading what you have, again, everything is working as designed. There are physical limitations in place here... – Breakthrough – 2011-10-21T12:44:02.347

Yes you are right. I plugged in a quite fast (30mb/s) 8GB usb flash stick and activated ReadyBoost to use it. Now the whole system is much more responsive. (I also turned SuperFetch Back on.). While copying something I still can do other things with almost no limitation – user4514 – 2011-10-27T02:01:16.170

There are ways to copy, that can be throttled.

"FastCopy" can be used to replace the systems copy operations, and it can be set to throttle its own operations. The throttle can be set "globally" or set manually while it is copying. Reverse that :-) allowing fastcopy to go full speed, it can copy way faster, but can choke the system down worse, use a whole core of processor and make high use of ram to do the copy. it allows for either , but I have not tested it in Win7 yet.

"TerraCopy" a popular replacment for the systems copy , allows for pausing of the copy operations. with it you can get something else going, then un-pause to continue. Terracopy is known to be working in win7.

I Have used both the above for copying, but I avoid having it takeover all operations, each of them can have things that it cant do exactally the same as the system. so I make sure that the original system copy is still available somehow.

Psycogeek

Posted 2011-10-20T12:24:01.330

Reputation: 8 067

Are you running anti-virus software with on-access scanning enabled? If so, disable the on-access scan when copying large files around. Put it back on when you're done.

I have seen on-access scan very serious performance problems a few times. It's basically because the disks end up waiting for the AV engine, and that can take a while to scan big files.

mauvedeity

Posted 2011-10-20T12:24:01.330

Reputation: 745

1See my question: "Anti-Virus software is not active". Thanks anyway – user4514 – 2011-10-22T18:41:07.880

I had a very similiar problem: Copying a file from HDD via to a slow network (only about MBit/s) share via robocopy. Whole computer was quite unusable. The operating system (Win10) is on a SSD! Even browsing (Firefox is installed to the OS partition C:) was very slow because opening websites generates IO.

Via ProcessHacker https://processhacker.sourceforge.io/ I've reduced IO priority. So the PC stays usuable and copying speed doesn't seem reduced so far.

Hannes

Posted 2011-10-20T12:24:01.330

Reputation: 9