7

My server has 8Gigs of RAM and 8Gigs configured for swap file. I have memory intensive apps running. These apps have peak loads during which we find swap usage increase. Approximately 1 GIG of swap is used.

I have another server with 4Gigs of RAM and 8 Gigs of swap and similar memory intensive apps running on it. But here swap usage is very negligible. Around 100 MB.

I was wondering what are the exact conditions or a rough formula based on which Linux will do a swapout of a process memory in RAM to the swap file. I know its based on swapiness factor. What else is it based on? Swap file size? Any pointers to Linux kernal internal documentation/source code explaining this will be great.

Zenil
  • 281
  • 1
  • 2
  • 6

4 Answers4

3

The VM subsystem of a Linux kernel is a very complicated beast, the kernel uses heuristics and algorithms to determine which pages to swap in/out and when to do that. I don't think there is a simple formula which is able to describe how and when pages are being put to disk. Maybe this LWN article is of use to you:

http://lwn.net/Articles/83588/

pfo
  • 5,630
  • 23
  • 36
2

As you probably know, the linux kernel uses as much memory as possible to cache things. That's why the output of free looks like I'm almost out of memory, according to the top line: 

 $ free -m
              total       used       free     shared    buffers     cached
 Mem:          3168       2963        205          0        155       1263
 -/+ buffers/cache:       1543       1624
 Swap:          494         86        407

but the important part is the amount of RAM that buffers and cache aren't using

 -/+ buffers/cache:       1543       1624

Still, all but 205 MB of memory are used in that example, even if it is by cache. As it was explained to me (or at least, as I understood it), if a program suddenly requests > 205 MB of memory, that chunk comes out of swap, so that the process can have the memory chunk, then it comes out of swap once it gets a chance to free memory being used by cache.

As pfo mentioned, the underlying mechanisms are complex, and far beyond my understanding. I asked Ted Ts'o nearly this identical question at a class he was teaching at LISA, and I've given you essentially the same answer he gave me, or at least as I understood it.

Incidentally, I suspect that the memory "used" in swap isn't actually used by anything at this point, but that it just hasn't been freed yet.

Matt Simmons
  • 20,218
  • 10
  • 67
  • 114
  • 1
    ``swapoff -a && swapon -a``should be used to free that swap space. – pfo Jan 08 '10 at 17:14
  • If a program suddenly asked for > 205 MB of memory, Linux would *not* have to get a chunk of memory "out of swap". Instead, the system could simply decide that some of its disk cache pages were not as important as supplying this request for RAM, mark those pages as unused and wipe their content, then hand them to the process. After all, everything in the disk cache can just be read back in later if someone looks at the cached file again! Swap wouldn't even be involved. – Brandon Rhodes Apr 21 '10 at 11:50
  • That's not the way that it was explained to me (or at least, how I understood the explanation) – Matt Simmons Apr 21 '10 at 13:27
2

Expanding upon pfo's answer, a vendor can also pre-tune their deployed kernel(s) to handle the swappiness in a certain fashion. On a Red Hat system you can configure much of this in /etc/sysctl.conf -- Google up settings such as:

vm.swappiness
vm.vfs_cache_pressure

By carefully tuning the system's swappiness to your application's specific needs you may be able to reduce your swap usage on that 8GB machine. I have a note in my files that says for vm.swappiness "use of the swap memory by default on Kernel 2.6.xx is set to 60%" but didn't save the link where I read that, so take it with a grain of salt. :)

1

Ran into this very issue on production JBoss app servers on AWS.

I would diff sysctl -a both systems.

1) For servers set vm.swappiness to near 0. This prefers to keep stuff in memory.

2) Enable a small swap file on all servers so you can monitor actual memory load. (Check used swap %).

3) Monitor page faults (nFLT) with Cacti or similar.

4) Check ulimits.

5) Tune based on measureable app performance.