nginx automatic failover load balancing

Question

I'm using nginx and NginxHttpUpstreamModule for loadbalancing. My config is very simple:

upstream lb {
    server 127.0.0.1:8081;
    server 127.0.0.1:8082;
}

server {
    listen  89;
    server_name localhost;

    location / {
            proxy_pass      http://lb;
            proxy_redirect  off;
            proxy_set_header        Host            $host;
            proxy_set_header        X-Real-IP       $remote_addr;
            proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

But with this config, when one of 2 backend server is down, nginx still routes request to it and it results in timeout half of the time :(

Is there any solution to make nginx to automatically route the request to another server when it detects a downed server.

Thank you.

score 36 · Accepted Answer · edited Jun 27 '18 at 22:30

36

I think that it's because nginx is not detecting that the upstream is down because it's on the same machine.

The options that you're looking for are: proxy_next_upstream and proxy_connect_timeout.

Try this:

location / {
        proxy_pass              http://lb;
        proxy_redirect          off;
        proxy_next_upstream     error timeout invalid_header http_500;
        proxy_connect_timeout   2;
        proxy_set_header        Host            $host;
        proxy_set_header        X-Real-IP       $remote_addr;
        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
}

edited Jun 27 '18 at 22:30

Elie Saad

103
3

answered May 12 '10 at 18:59

Guillaume Filion

967
1
10
13

I was having problems with an incomplete Apache/PHP setup (missing PEAR library files) that threw http 500 errors. proxy_next_upstream just retried on the other machine - that did return OK. – Alister Bulman Jul 24 '10 at 16:08
5

Why is proxy_redirect off? – Seun Osewa Mar 15 '12 at 09:38

score 2 · Answer 2 · answered May 12 '10 at 08:03

Hey, please see the wiki: http://wiki.nginx.org/NginxHttpUpstreamModule#server

Basically if a failure is detected the backend will be marked as down for x seconds and it will try again. So if you keep seeing connections it's probably nginx that keeps checking if the backend has become available.

It should, however, try the next entry in the upstream block, so you shouldn't actually see that no backends are available if only one is down.

nginx automatic failover load balancing

2 Answers2

Linked