How to limit HDD write speed for chosen programs

My HDD is slow 5400RPM and when working with 10GB size files it is impossible to use the computer until that big file finish re-creating a new copy/muxing/writing/etc. depends on process, the thing is it eats up all HDD speed and no other program can be used while this process runs.

So I was thinking maybe I could limit HDD write speed for programs I work with, so then it would reserve free HDD leftover speed to use for browsing internet or playing mp3s while the big files are being processed.

user3108268

Posted 2014-08-11T16:23:56.137

Reputation: 574

2I've never heard of a throttled HDD IO in Windows, at least natively. Have you tried setting the process priority?: wmic process where name="calc.exe" CALL setpriority "idle" – Marcelo – 2014-08-11T16:33:05.307

@Marcelo Priority dictates CPU time, not disk. When processing a large file, the disk is what is slowing the computer down, not the CPU. – Keltari – 2014-08-11T19:09:06.533

– Cees Timmerman – 2014-08-11T19:19:46.680

@Keltari Yes, I'm aware. I just thought that maybe modifying the priority for CPU could have some impact towards the final objective in the question. – Marcelo – 2014-08-11T19:27:31.250

@Marcelo It would have an impact, true. But it might not necessarily be for the better. If the computer was also swapping memory, then it would heavily decrease the performance. – Keltari – 2014-08-11T19:29:32.150

Things you need to know about having multiple programs using the disk at once: http://stackoverflow.com/a/7640319/103167 and http://stackoverflow.com/a/1240473/103167

– Ben Voigt – 2014-08-11T19:42:32.243

2If the computer is "unusable" when copying huge files, is it possible that you have forgotten to enable DMA? Normally, whether or not the disk is busy makes little or no difference since that happens without much (or any) CPU intervention. – Damon – 2014-08-11T19:45:19.187

It is possible by setting the Background priority, according to dsulimano

– niutech – 2014-08-11T21:45:38.657

@Marcelo: Some programs (like Diskeeper) can throttle their own I/O, so I suppose this can be done by writing a filter driver. But it's a difficult thing to get right, and even more difficult to get working well. – user541686 – 2014-08-12T06:55:16.330

@Mehrdad I would think that such throttling would happen above the I/O, in the application. As in scheduling the buffers/blocks to be written to disk at the application level. – Marcelo – 2014-08-12T12:33:18.493

Answers

Unfortunately, you can't do that. Hardware is abstracted from applications and even parts of the OS. In essence, the application says "write this to disk" and the OS handles it. The application doesn't handle how or even when it gets written. All of this is handled at the driver and kernel level and there is no real way to change this.

Even if you could do this, the fact is you are using a spinning magnetic disk - and a very slow one at that - you wouldn't see much gain. The drive is constantly spinning and the heads are moving along the disk. Even if you wrote to your file less, the disk will constantly be seeking the right areas on the disk to read/write to/from.

Your best option would be to get another drive and do your work on that drive, leaving your first drive for your internet, MP3s, etc.

P.S. You didn't mention how much RAM you have. Swapping could be a factor in the slow speed. Depending on how much RAM you have, adding more will only help.

Keltari

Posted 2014-08-11T16:23:56.137

Reputation: 57 019

2Wouldn't a Windows version of Linux' ionice come close to being an answer? – Daniel Beck – 2014-08-11T19:08:43.750

@DanielBeck There may be software out there that might work. However, who knows what complications might come from using it? It would be so much easier to just slap in another drive and be done with the issue. – Keltari – 2014-08-11T19:12:04.523

You can do it. See the dsulimano's answer to a similar question

– niutech – 2014-08-11T21:47:48.577

1I don't see any principled argument to why an application could not be throttled. Here's a very simple throttle: Suspend the process for 1sec every second. That throttles IO by about 50% and lets other processes slip in to use the disk. Seek times are on the order of 10ms. – usr – 2014-08-12T00:09:53.093

@usr: It depends on the type and frequency of I/O. Your approach works in some cases but not others -- e.g. if the I/O is happening entirely in kernel mode, I don't think you can always pause it by pausing the user-mode process. – user541686 – 2014-08-12T06:57:17.780

@usr A reasonably written application on a system with disk DMA I/O enabled will need almost no CPU time for itself during the work - the copying itself is handled not even by the kernel, but by the actual hardware. The only point where you need any CPU time at all is at the buffer boundaries, which might or might not happen often enough for suspends to be effective. – Luaan – 2014-08-12T07:29:07.633

@Luaan yes, but a suspended app cannot issue more IO requests. The old ones complete, and then there is a pause. This was just an example to show that throttling IO is possible in principle. It's just not built-in to Windows, but many techniques for doing that exist. You can even extend Windows with a driver to do that. – usr – 2014-08-12T10:12:42.327

Here's a link to an article in Russian, the author has built an utility which integrates into Explorer context menu: https://habr.com/ru/post/317802/ The sources is .REG file (i.e. the utility is a simple registry tweak), it is posted in the article.

– Alexandr Zarubkin – 2019-03-25T11:18:23.480

@DanielBeck you are correct. The Sysinternals Process Explorer can perform the same functionality as the nice and ionice utilities under *nix variants. This answer is plain wrong. Setting the process to background 4 priority is the correct way to achieve the desired result of low I/O and low CPU priority. Both combined significantly reduce the system impact of these processes. This is the real way to change this. The low memory priority should also throttle any disk swapping taking place as well, another hidden factor that can easily affect system responsiveness. – Ian Macintosh – 2020-01-30T20:53:43.947

Process Lasso allows you to prioritize I/O. From their FAQ:

New in v4.1 (v4.09 beta), Process Lasso has explicit current and default I/O priority settings available those who require them. This only applies to NT6+ (Vista,Win7,...).

HOWEVER, Vista's I/O prioritization is based on the priority of the thread doing the I/O, which is dependent on the priority class of the process doing the I/O. There are actually only two levels of priority usable by applications, Normal and Very Low. Although there are technically 5, only these two are implemented or usable by processes.

Therefore, adjustment of the thread priorities will propagate to the I/O of that thread. In this way, Process Lasso already does I/O prioritization. Microsoft recommends applications adjust their I/O priority based by setting the calling thread's priority (directly or by adjustment of the process priority base class).

niutech

Posted 2014-08-11T16:23:56.137

Reputation: 763

What I do when I have a process that keeps eating all my disk I/O on Windows is bringing up Process Explorer and pausing the offending process. After I'm done with my business and am ready to wait a bit I resume it. It's definitely not the most productive way of doing it since it's all-or-nothing way of controlling disk I/O and your process will have to wait while you use your computer, but so far I haven't seen any better alternative.

On the other hand, if you can do your process on Cygwin, you can install pv on your computer and use it as the first or final step with an option (I think it's -r) to throttle pv's output rate.

RAKK

Posted 2014-08-11T16:23:56.137

Reputation: 345

Use the Sysinternals "Process Explorer" to set the priority to 'Background: 4 (Low I/O and Memory Priority)'

Find the process that is using Disk I/O by right-clicking the Column Headings and under Process I/O add the column I/O Delta Writes. Then click on that column to sort by it, and the offender is likely to be the top process. Whatever way you use to find the process, then right-click the process and set it to Background: 4 priority.

Subsequently right-click the process again, go to Properties and look at the Performance tab and you should see I/O Priority set to Very Low.

This would be the equivalent of using nice and ionice under Linux.

You should find a significant improvement to the system performance and responsiveness after doing this. You will have to re-do this after rebooting and/or after the process exits and restarts.

Ian Macintosh

Posted 2014-08-11T16:23:56.137

Reputation: 101