1
I'm trying to run a very long and slow (but important) background process on my computer. It took all weekend and is still running.
The problem is that this process scans dozens of GBs of disk files (some local, some on the network), and Windows apparently likes to page out all other programs so that it can use almost every megabyte (out of 4GB RAM) as disk cache for this one program. This occurs despite the fact that I used task manager to set its priority to "low". Not only does Windows page out everything else, this low-priority process seems to be given "first dibbs" on the hard disk, so other apps can take a few minutes to be paged back in. This disrupts all the other work I want to do.
I'm on Win7 x64, but I have observed the same phenomenon in WinXP (I'd say it's worse on XP... at least in Win7 when I use a single program for awhile it becomes responsive again.)
Other than disabling the paging file entirely, which requires a restart anyway, is it possible to limit the disk cache used by a single process, or by the system as a whole?
Qwertie
Posted 2011-07-25T17:29:19.307
Reputation: 151
There's really nothing you can do. If you squeeze the disk cache, it will just mean that the process needs to do even more disk I/O. The OS can't stop disk I/O once it has issued it, so needing to do even more disk I/O, the latency for other processes will just get worse regardless of priority. You can only fix this by changing the design of the program or by moving the files it needs to analyze to their own physical disk. – David Schwartz – 2016-05-01T19:23:06.237
@DavidSchwartz IIRC it would have been perfectly fine not to allocate any disk cache to the program, since it scanned all the files serially (and there was not enough memory to hold all the files). The problem was not the I/O itself but rather the OS's bad decision to page out DLLs and data of other programs. – Qwertie – 2016-05-08T09:06:15.883
@Qwertie It's the program's responsibility to tell the OS how it's going to use the data it's reading. Otherwise, all the OS can do is make the best guess it can. You may be able to work around a badly written program, but put the blame where it belongs. – David Schwartz – 2016-05-08T22:10:28.823
3
Just FYI, you can use Process Explorer (from Microsoft Sysinternals) to pause/suspend the execution of a process. You might find this useful if you absolutely need to stop the program but don't want to just terminate the process (and in most cases, its progress).
– Breakthrough – 2011-07-25T17:34:35.060That's really useful, thanks! I could resume it when I go home at night. – Qwertie – 2011-07-25T18:23:32.800