Nginx upstream keepalive with SNI

Question

I've got Nginx setup to proxy two subdomains. SNI is used so each subdomain has a different SSL certificate. Nginx setup is roughly:

upstream a_example_443 {
    server 1.2.3.4:443;
    keepalive 128;
    keepalive_timeout 180s;
}
upstream b_example_443 {
    server 1.2.3.4:443;
    keepalive 128;
    keepalive_timeout 180s;
}
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_ssl_server_name on;
server {
    listen 443 ssl;
    server_name aproxy.example.com;
    location / {
        proxy_pass https://a_example_443;
    }
}
server {
    listen 443 ssl;
    server_name bproxy.example.com;
    location / {
        proxy_pass https://b_example_443;
    }
}

This works, the SNI names are a_example_443 and b_example_443 and the subdomains have aliases for those. However, is it bad that I use two upstreams?

I tried configuring it to use one upstream. After quite some effort, this works:

upstream example_443 {
    server 1.2.3.4:443;
    keepalive 128;
    keepalive_timeout 180s;
}
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_ssl_server_name on;
proxy_ssl_session_reuse off;
server {
    listen 443 ssl;
    server_name aproxy.example.com;
    location / {
        proxy_set_header HOST a.example.com;
        proxy_ssl_name a.example.com;
        proxy_pass https://example_443;
    }
}
server {
    listen 443 ssl;
    server_name bproxy.example.com;
    location / {
        proxy_set_header HOST b.example.com;
        proxy_ssl_name b.example.com;
        proxy_pass https://example_443;
    }
}

First I had to set the HOST and proxy_ssl_name to the SNI name. This is fine, except it seems when I added proxy_set_header HOST, I lose all the proxy_set_header I had at the http configuration level (not shown here). I'd love to know why, but OK fine, I put those in a file and include it in each server. Edit, found out why, proxy_set_header docs:

These directives are inherited from the previous level if and only if there are no proxy_set_header directives defined on the current level.

Next I had to set proxy_ssl_session_reuse off. What are the ramifications of using this? I understand it disables abbreviated handshakes, but when is that needed? My guess is when using keep alive, not very often. Is that right?

Keepalive is where things become unclear for me about exactly how the upstream connections work. Nginx gets a request, opens an SSL connection to the upstream, sends the an SNI name. The upstream routes it to the right subdomain, uses that subdomain's certificate, etc. Later Nginx receives another request -- can it reuse the previous SSL connection that is still because of keepalive? If so, what if the second request is for the other SNI name? Does it just send the request and let the upstream use the Host header to route it?

Ultimately, should I use two upstreams or one?

Nginx can handle this, but at this level of complexity it's time to start looking at more full-featured load balancers, like haproxy, which handles this scenario without blinking. — Michael Hampton, Jun 18 '20 at 21:36

score 0 · Answer 1 · answered Jun 19 '20 at 11:54

0

It seems that the above configuration is wrong: the SSL connections are reused and if the connection doesn't match the SNI name then the upstream rejects it. I suppose this means the answer is that two upstreams are required, one for each SNI name.

answered Jun 19 '20 at 11:54

Nate

184
1
2
6

Nginx upstream keepalive with SNI

1 Answers1