My current setup is nginx as a load balancer, in front of a bunch of upstream servers.
I want to be able to deploy with no change in user experience. This means no 502s and no increase page times.
My understanding is that when nginx -s reload
is issued, nginx will gracefully spawn new threads to handle new connections for the new configuration, but will still finish handling requests that were in-flight at the time of reload.
Moreover, I have read somewhere that marking an upstream as down
in the upstream block, and reloading, will take that upstream from the rotation, but will still finish any in-flight connections to it.
Is any/all of this correct?
The current plan is to have a small process in the nginx host, that will basically mutate nginx configuration and issue these graceful reloads when asked to do so through an internal REST API.
My deployment tool will then connect that process and add/remove my upstreams from nginx config in sequence, after graceful shutdown and boot up/warm-up for each upstream is complete.
Does this make sense? How are other people doing this? Are there any already available tools for this?
I haven't had a lot of luck searching for info on this...