4

We are running a server with 23 cores (cpuinfo shows 22 as the highest process number) and 66 Gig RAM. The development team here is running some apps, which hogs memory.

I have observered that: 1. Even when the memory runs very low (free memory = 167 MB), there is no swap usage. Wondering why the server refuses to use swap. Any ideas on this? 2.The system load sometime increases to numbers like 70+, I can see that these apps are I/O intensive.. could this result in in such high load averages?

Here is an output from VMstat

procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 3  0      0  28932   1049  31757    0    0     6   145    6   33  1  0 98  0  0
..
..
12  2      0    167   1876  57749    0    0   932 49604 2306 96856 26  8 58  7  0

Output from meminfo

MemTotal:     66000044 kB
MemFree:      11824808 kB
Buffers:       1498068 kB
Cached:       48842440 kB
SwapCached:          0 kB
Active:        5350660 kB
Inactive:     46810284 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     66000044 kB
LowFree:      11824808 kB
SwapTotal:    131074324 kB
SwapFree:     131074324 kB
Dirty:         2641608 kB
Writeback:          96 kB
AnonPages:     1819092 kB
Mapped:         766660 kB
Slab:          1753100 kB
PageTables:     131908 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  164074344 kB
Committed_AS:  9460100 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    336508 kB
VmallocChunk: 34359401823 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB
Mohammed Riyaz
  • 51
  • 3
  • I voted this up because it's a common question and shows signs of influence from old "canonical" guides saying that swap should be twice what memory is. The reason for the question is well-founded and it would be good to spread that knowledge around a bit. Constructive, answerable, and showed signs of research. – Jeff Ferland Feb 02 '12 at 15:44
  • 2
    http://www.linuxatemyram.com/ – Zoredache Feb 02 '12 at 18:36

1 Answers1

10

You're reading this wrong, and barely using your memory.

MemTotal:     66000044 kB = 63GB available to the system
MemFree:      11824808 kB = 11GB with nothing at all used.
Buffers:       1498068 kB = 1.4GB
Cached:       48842440 kB = 46GB of data from the drive that is kept in otherwise unoccupied memory
SwapCached:          0 kB
Active:        5350660 kB = 5.1GB of memory that is actually used by applications.

When you read data from disk, it goes into memory. If nothing else is competing for the memory, that file is left in memory until an application tries to use the space. Whenever a file is read from disk, the cache is checked first.

You're barely using your memory for applications, and even with the cache you've got a lot of leftover memory to work with. You server shouldn't be utilizing swap space with that light of a memory load.

Jeff Ferland
  • 20,239
  • 2
  • 61
  • 85
  • And at that, you probably have bigger problems if you hit swap. It's virtually never a good idea to let it get to that point. – Publiccert Feb 02 '12 at 15:11
  • I would say that using swap is just fine, experiencing page faults is bad. A few gigabytes of swap that has really low activity is fine on a server that size. Having a swap partition in the 130GB range is a waste -- it should never be used like that. – Jeff Ferland Feb 02 '12 at 15:15
  • Exactly. I should have elaborated but you did a good enough job of it. This setup is just strange. – Publiccert Feb 02 '12 at 15:20
  • @Publiccert It's a "back in the day" artifact when the recipe was "make your swap twice your RAM". That was something meaningful in the days of 8MB systems. The "rule" hasn't fully gone away yet, and a lot of folks find that when they're trying to figure out how much swap to partition. – Jeff Ferland Feb 02 '12 at 15:25
  • I remember those guides and tutorials quite fondly. I still run into system admins that follow that rule with blind faith. What are your recommendations for modern swap sizes? I still use the rule of 2-4Gb on a desktop and ~8Gb on a modern server. – Publiccert Feb 02 '12 at 15:32
  • Sounds fair to me. You want enough to get rid of junk that never runs or needs to be displaced by another large job, but not enough that it always sits free. I suppose a massive swap partition might be useful on terminal servers with many users running intermittently with Firefox open all day, but otherwise your numbers seem reasonable. Benchmark & measure. There's never one true rule, but 2-4/8 is a good starting point. – Jeff Ferland Feb 02 '12 at 15:41