Nginx ip_hash does not load-balance connections to meteor backend

Question

I cannot get nginx to load balance internal connections with ip_hash enabled. I need sticky sessions as I use meteor in the backend with sockets but all requests always hit the same backend.

The nginx access log file shows the following IP addresses:

192.168.0.20 - - [xx/xxx/2017:xx:xx:xx +xxxx] "GET /favicon.ico HTTP/1.1"  404 5 "http://xxxx.lokal/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36"
192.168.0.11 - - [xx/xxx/2017:xx:xx:xx +xxxx] "GET /sockjs/602/dpkl6lfe/websocket HTTP/1.1" 101 55045 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36"
192.168.0.208 - - [xx/xxx/2017:xx:xx:xx +xxxx] "GET /sockjs/031/cx1kml79/websocket HTTP/1.1" 101 1146677 "-" "Mozilla/5.0 (iPad; CPU OS 10_2_1 like Mac OS X) AppleWebKit/602.4.6 (KHTML, like Gecko) Version/10.0 Mobile/14D27 Safari/602.1"

Is it because they are all coming from the same 192.168.0.* subnet? If so, how can I change that behaviour?

Here is my config file:

user  www;
worker_processes  4;
error_log  /var/log/nginx/error.log;

events {
 worker_connections  1024;
}

http {
    map $http_upgrade $connection_upgrade {
      default upgrade;
      ''      close;
    }

    upstream demo {
      ip_hash;
      server 127.0.0.1:5000;
      server 127.0.0.1:5001;
    }

    include       mime.types;
    default_type  application/octet-stream;
    access_log  /var/log/nginx/access.log;

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
    ssl_prefer_server_ciphers on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    types_hash_max_size 2048;

    gzip on;
    gzip_disable "msie6";

    server_tokens off; # for security-by-obscurity: stop displaying nginx version

     server {
          listen       80;
              server_name xxxx.lokal;

          location / {
            proxy_pass http://demo;
            proxy_redirect      off;
            proxy_set_header    Host              $host;
            proxy_set_header    X-Real-IP         $remote_addr;
            proxy_set_header    X-Forwarded-For   $proxy_add_x_forwarded_for;
            proxy_set_header    X-Forwarded-Proto $scheme;
            proxy_http_version 1.1;
            proxy_set_header    Upgrade           $http_upgrade; # allow websockets
            proxy_set_header    Connection        "upgrade"; 
            proxy_buffering     off;
            proxy_connect_timeout 43200000;
            proxy_read_timeout    43200000;
            proxy_send_timeout    43200000;

            if ($uri != '/') {
                expires 30d;
            }
    }
}

score 8 · Accepted Answer · answered Mar 18 '17 at 21:38

8

Yes, this happens because ip_hash uses only the first three octets of the IP address for selecting the backend node.

You can use hash $remote_addr; directive to make nginx use the complete remote IP address for hash. Downside of this is that if a node goes down, all hash mappings change, and sessions will be lost.

More information on upstream selection methods can be found at nginx upstream module documentation.

answered Mar 18 '17 at 21:38

Tero Kilkanen

34,499
3
38
58

1

thanks replacing `ip_hash` for `hash $remote_addr` did the trick – user263367 Mar 20 '17 at 11:40
does hash $remote_addr evenly distribute clients among servers or is it premapped? – ninjaneer Jul 10 '20 at 02:27
It cannot do exact even distribution by definition, since hash value is alwys the same for a particular IP address. However, the has is calculated in a way that the distribution is practically even. – Tero Kilkanen Jul 10 '20 at 06:31

Nginx ip_hash does not load-balance connections to meteor backend

1 Answers1