1

We are receiving alerts for kafka memory utilization in our kubernetes cluster and analysing it seems that most of the memory is page cache. I am aware that kafka uses loads of page cache for IO performance and reads/writes are cached before flushing to disk.i am trying to see ways to fine tune this page cache and lot of articles direct to update vm.dirty_ratio and vm.dirty_background_ratio kernel parameters for earlier flushing of data to the disk from memory.

When i checked i see that the Dirty cache already seems to be low and these parameters need not be updated.

cat /proc/meminfo|grep Dirty Dirty: 70416 kB

cat /proc/vmstat | egrep "dirty|writeback" nr_dirty 24873 nr_writeback 0 nr_writeback_temp 0 nr_dirty_threshold 34518467 nr_dirty_background_threshold 11506155

Any ideas on what else can be done to fine tune page caches?

Putharekulu
  • 21
  • 1

1 Answers1

0

I think you misunderstand a bit how page cache works in Linux systems.

Having full memory cache is something normal and in fact it is very much desired on any systems that read / write to disk a lot (kafka, elasticsearch, databases). You shouldn't fire alarms when page cache is full because it's not something harmful or something that is causing any trouble to system. It's quite the opposite.

Dirty pages are related only to writing to disk. Even when the dirty pages are flushed (written) to disk it doesn't mean they leave the page cache. They are just marked as clean (= up to date).

In page cache data can be only replaced (by dirty page mechanism) or new data added.

The details have been explained many times on internet. For example on this resource: https://www.linuxatemyram.com/

TLDR: disable your alerts that are triggered if page cache is full - it is normal and desired behaviour of healthy linux system and you absolutely WANT to have it as full as possible. It is not a real problem and it's not badly affecting your system performance. It's exact opposite. Full cache means you are using resources of your Linux system at full.

Ondrej Bardon
  • 101
  • 1