1

I had problem with a process spawning very many threads due to a bug, eating up memory, causing heavy swapping to the swap partition. Thus, I switched off swap – which is recommended in server clusters anyway – to let the buggy program fail early. To my surprise, this didn't improve things at all! Similarly, setting vm.swappiness to 0 didn't help. iotop revealed heavy reading from disk (almost no writing).

I suspect that the Linux kernel is still swapping code pages in memory, and re-reads overwritten code pages from disk when they are needed. When RAM is very low, this happens very frequently, causing thrashing, making the computer almost unresponsive.

How can one prevent this unfortunate and potentially dangerous situation? I tried switching off swap, cgroups for memory and CPU constraints for that process, and ulimit for max. 30 processes. Nothing even improved the situation. (Well, this is not entirely true: a RAM limit which left hundreds of MB permanently unused did help.)

In particular: Can I prevent swapping code segments in memory, and let the RAM-requesting process simply get a NULL for their next malloc?

Torsten Bronger
  • 226
  • 1
  • 2
  • 10

1 Answers1

1

How about tuning the memory overcommit accounting and setting it to option 2 (do not overcommit) instead of the default option 0 (heuristic overcommit)? The description for that states

Don't overcommit. The total address space commit
for the system is not permitted to exceed swap + a
configurable amount (default is 50%) of physical RAM.
Depending on the amount you use, in most situations
this means a process will not be killed while accessing
pages but will receive errors on memory allocation as
appropriate.

The above snippet is from kernel documentation

The policy can be set with vm.overcommit tunable.

Janne Pikkarainen
  • 31,454
  • 4
  • 56
  • 78