0

We have an Hadoop cluster of CentOS 6 (kernel version 2.6.32-504.el6.x86_64) and CentOS 7 (kernel 3.10.0-327.el7.x86_64) servers .

All the nodes are configured with a swapinness of 1. All nodes have plenty of RAM availables.

The CentOS 6 nodes never swap any processes but the CentOS 7 nodes swap processes even if there is available memory. This triggers warning in our monitorig tools.

Can someone explain me why my CentOS 7 nodes behave differently than the CentOS 6 nodes and what I can do to prevent swapinness? Apparently, configuring swapinness to 0 can lead to issues so I'm not confident to do it.

Thanks in advances, Loïc

loicmathieu
  • 111
  • 4
  • And are you sure, that there is never high memory usage situation? Ordinary state can be low memory usage, however there can be some peaks too. Do you have system monitoring in place(systat/sar) with like 1s granularity? – Jaroslav Kucera Oct 11 '17 at 07:53
  • From https://www.percona.com/blog/2014/04/28/oom-relation-vm-swappiness0-new-kernel/, it seems the way vm.swapiness has changed from kernel version 3.5 and has been backported, but maybe there are still some differencies between 2.x and 3.5+ versions. – Pierre-Alain TORET Oct 11 '17 at 08:06
  • @JaroslavKucera I check the monitoring I have (via Cloudera Manager) and my host never use more than 80GB on 256GB the rest is used by OS file cache. – loicmathieu Oct 11 '17 at 08:17
  • We experience a similar situation on RHEL7 with Cloudera Hadoop. Under load the system will use swap before it clears buffer cache, even with swappiness set to 1 and vfs_cache_pressure at 200. – Shaun Dewberry Aug 14 '18 at 08:08

0 Answers0