1

After a reload, old haproxy process remains there and doesn't release memory (sometime it does but behaviour is not consistent). After killing the old process (several days old) the memory get released.

Expected behavior: We have long lived connections but these connections should be cleared after 4-7 hours max from old process after reload. 

ubuntu@ip-10-0-5-193:~$ uname -a
Linux ip-10-0-5-193 3.13.0-48-generic #80-Ubuntu SMP Thu Mar 12 11:16:15 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

ubuntu@ip-10-0-5-193:~$ haproxy -v
HA-Proxy version 1.5.14 2015/07/02
Copyright 2000-2015 Willy Tarreau <willy@haproxy.org>

ubuntu@ip-10-0-5-193:~$ date
Tue Apr  5 00:44:29 UTC 2016

ubuntu@ip-10-0-5-193:~$ ps -ef | grep [h]aproxy
haproxy   7494     1 15 Apr01 ?        13:26:20 /usr/local/sbin/haproxy -f     /etc/haproxy/haproxy.cfg -p /var/run/haproxy.pid -D -sf 30065
haproxy  13357     1 18 Apr04 ?        02:40:15 /usr/local/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /var/run/haproxy.pid -D -sf 7494

ubuntu@ip-10-0-5-193:~$ free -g
total       used       free     shared    buffers     cached
Mem:          29         26          3          0          0          0
-/+ buffers/cache:       25          4
Swap:            0       0          0

ubuntu@ip-10-0-5-193:~$ sudo kill 7494
ubuntu@ip-10-0-5-193:~$ free -g
            total       used       free     shared    buffers     cached
Mem:          29         10         19          0          0          0
-/+ buffers/cache:       9         20
Swap:            0       0          0

Compiling info: 

$ wget http://www.haproxy.org/download/1.5/src/haproxy-1.5.14.tar.gz
$ wget ftp://g.oswego.edu/pub/misc/malloc.c
$ apt-get install -y libssl-dev make libpcre3-dev
$ tar zxvf haproxy-1.5.14.tar.gz; cd haproxy-1.5.14/
$ make TARGET=linux2628 CPU=native USE_STATIC_PCRE=1 ADDLIB=-lz USE_OPENSSL=1  DLMALLOC_SRC=../malloc.c
$ make install

Edit: added compile time info

Jagbir
  • 157
  • 1
  • 2
  • 7
  • 1
    Obviously a connection (or more) is keeping the old process alive, and until said connection(s) closes it won't terminate. Find what's causing that, and you'll get your RAM back after the 4-7 hour mark, as you claim you should. – GregL Apr 05 '16 at 12:06
  • 1
    Are you *sure* that the connections only last up to 7 hours? – GregL Apr 05 '16 at 12:06
  • @GregL Yes, connections keeping old process live and we're not sure how to properly close/kill such connection(s). About your second question, yes, we are _sure_ that 99% connections drops if we remove hosts from DNS (stops taking new connections) within 4-5 hours itself. The issue here is that even if small no. of connections still linger around, process should not hold huge amount of RAM. – Jagbir Apr 06 '16 at 04:32
  • This usually happens to me as well (service haproxy reload). Instead of reload, have you tried a graceful restart? http://serverfault.com/a/580630/101643 – Ianthe the Duke of Nukem Apr 06 '16 at 08:49
  • Not an option here as connections are long lived (can stay idle for few hours), since a restart (even graceful one IMO) will clear existing connections. Anyhow, thanks for link, interesting read. – Jagbir Apr 11 '16 at 07:27

0 Answers0