Database is going to recovery mode whenever the OOM killer is invoked on the postgres process.
/var/log/messages output with respect to the issue
oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/user.slice/user-0.slice/session-137004.scope,task=postgres,pid=4193383,uid=1001 Mar 25 15:55:21 kernel: Out of memory: Killed process 4193383 (postgres) total-vm:13226668kB, anon-rss:172912kB, file-rss:4kB, shmem-rss:10194068kB, UID:1001
postgres logs output with respect to the issue
postgresql log FATAL error : postgresql-2021-03-25_000000.log:2021-03-25 15:55:26 IST [54678]: [1-1] [unknown] 10.10.10.10. db=dbname ,user=username FATAL: the database system is in recovery mode
sysctl -a command output
vm.drop_caches = 3 vm.extfrag_threshold = 500 vm.hugetlb_shm_group = 0 vm.laptop_mode = 0 vm.legacy_va_layout = 0 vm.lowmem_reserve_ratio = 256 256 32 0 0 vm.max_map_count = 65530 vm.memory_failure_early_kill = 0 vm.memory_failure_recovery = 1 vm.min_free_kbytes = 67584 vm.min_slab_ratio = 5 vm.min_unmapped_ratio = 1 vm.mmap_min_addr = 4096 vm.mmap_rnd_bits = 28 vm.mmap_rnd_compat_bits = 8 vm.nr_hugepages = 0 vm.nr_hugepages_mempolicy = 0 vm.nr_overcommit_hugepages = 0 vm.numa_stat = 1 vm.numa_zonelist_order = Node vm.oom_dump_tasks = 1 vm.oom_kill_allocating_task = 0 vm.overcommit_kbytes = 0 vm.overcommit_memory = 0 vm.overcommit_ratio = 50 vm.page-cluster = 3 vm.panic_on_oom = 0 vm.percpu_pagelist_fraction = 0 vm.stat_interval = 1 vm.swappiness = 30 vm.user_reserve_kbytes = 131072 vm.vfs_cache_pressure = 100 vm.watermark_scale_factor = 10 vm.zone_reclaim_mode = 0
Need to understand what exactly is causing the issue here?