2

I am using HAProxy to load balance app servers. Each app server has a max connections limit, once that limit is exceeded the queue is exceeded. When that happens, I will be adding another app server to HAProxy, and using a "graceful reload" to hold incoming requests until after the service has started again. My question is, when it's restarted, what will happen to the existing queue - will it be retained? If not, how am I able to retain it? I obviously don't want the HTTP requests to be dropped on reload.

Zach Russell
  • 277
  • 1
  • 3
  • 13
  • Have a look on [this](http://engineeringblog.yelp.com/2015/04/true-zero-downtime-haproxy-reloads.html) and [this](http://serverfault.com/questions/580595/haproxy-graceful-reload-with-zero-packet-loss). – gxx Jan 05 '16 at 23:28
  • @gf_ I have seen both of these articles. This doesn't answer my specific question about the http queue being persistent on reload - it only applies to new incoming requests. – Zach Russell Jan 05 '16 at 23:36
  • I see...AFAIK, established connections (like these in your queue) aren't dropped while doing a reload. – gxx Jan 05 '16 at 23:38
  • Not sure if you saw [this](http://serverfault.com/questions/740419/haproxy-1-5x-reload-configuration-without-terminating-layer-7-sessions) as well, but it may be of help. – gxx Jan 05 '16 at 23:45

1 Answers1

3

From the official docs: http://www.haproxy.org/download/1.7/doc/management.txt

HAProxy supports a graceful and a hard stop. The hard stop is simple, when the SIGTERM signal is sent to the haproxy process, it immediately quits and all established connections are closed. The graceful stop is triggered when the SIGUSR1 signal is sent to the haproxy process. It consists in only unbinding from listening ports, but continue to process existing connections until they close. Once the last connection is closed, the process leaves.

The hard stop method is used for the "stop" or "restart" actions of the service management script. The graceful stop is used for the "reload" action which tries to seamlessly reload a new configuration in a new process.

Both of these signals may be sent by the new haproxy process itself during a reload or restart, so that they are sent at the latest possible moment and only if absolutely required. This is what is performed by the "-st" (hard) and "-sf" (graceful) options respectively.

So in particular:

It [the graceful stop] consists in only unbinding from listening ports, but continue to process existing connections until they close.

Emphasis: but continue to process existing connections

You may see the backend queue drop to 0, because those are the stats for the newly formed backend queue of the new process. The old one, that was filled to max, is not shown anymore as it belonged to the previous process. But it is being processed, according to this link, which also includes a test scenario to verify this yourself:

http://comments.gmane.org/gmane.comp.web.haproxy/7815

JayMcTee
  • 3,763
  • 12
  • 20