How to proxy multiple tcp streams on one port with nginx

Question

With nginx http directive, you can have multiple servers on the same port with different names:

server {
    listen       80;
    server_name server1.example.com;
    location / {
      proxy_pass http://server1.example.com;
    }
}
server {
    listen       80;
    server_name server2.example.com;
    location / {
      proxy_pass http://server2.example.com;
    }
}

Is it possible to have nginx proxy multiple mysql servers on the same port with different names like you can with http? They aren't part of a cluster or anything. The have different, unrelated tables.

stream {
  upstream db1.example.com {
    server db1.example.com:3306;
    #server_name db1.example.com; "server_name" directive is not allowed here
  }
  upstream db2.example.com {
    server db2.example.com:3306;
  }

  server {
    listen 3306;
    proxy_pass db1.example.com;
  }
  #duplicate "3306" address and port pair
  #server { listen 3306; proxy_pass db2.example.com; }


}

2ps · Answer 1 · 2016-12-14T15:55:54.543

Yes, if nginx was compiled with --with-stream. https://www.nginx.com/resources/admin-guide/tcp-load-balancing/

Here is one of the examples with the same port:

upstream stream_backend {
    hash $remote_addr;

    server backend1.example.com:12345;
    server backend2.example.com:12345;
    server backend3.example.com:12346;
}

Second update:

You obviously can’t do anything like this:

stream {
  upstream db1.example.com {    
    server db1.example.com:3306;
    #server_name db1.example.com; "server_name" directive is not allowed here
  }
  upstream db2.example.com {
    server db2.example.com:3306;
  }

  server {
    listen 3306;
    proxy_pass db1.example.com;
  }
  #duplicate "3306" address and port pair
  #server { listen 3306; proxy_pass db2.example.com; }    
}

Because the nginx proxy for upstream db1.example.com is competing with db2.example.com for packets on port 3306. So you have to have the proxy for db1.example.com listening on another port than the proxy for db2.example.com. Otherwise nginx wouldn’t know how to route packets to and from the two upstreams. Apologies for misunderstanding your original post. server_name isn't allowed in stream definitions because, unlike http headers, there is no additional metadata in the tcp/udp packet that identifies which DNS was used to address the packet over to nginx.

I did, I posted one of the examples in my response. I know it’s a lot to read, but the documentation does have a lot of good stuff! — 2ps, Dec 14 '16 at 15:41

score 4 · Answer 2 · answered Jan 17 '20 at 22:11

It is possible to direct connections based on the SNI header using the nginx ssl preread module.

This depends on the client specifying an SNI header when opening the connection.

In the example below, nginx listens for connections on port 443. A connection to presence.myglance.org:443 is forwarded to port 4443 (that's an http server, not shown). A connection to presence-s.myglance.org:443 is forwareded to a stream server listening on port 5502, which terminates the ssl connection and forwards to port 6502.

stream  {

    ############################################################
    ### logging
    log_format log_stream '$remote_addr [$time_local] $protocol [$ssl_preread_server_name] [$ssl_preread_alpn_protocols] [$internalport] '
        '$status $bytes_sent $bytes_received $session_time';

    error_log   /home/Logs/error.log debug;
    access_log  /home/Logs/access.log log_stream;

    ### https://nginx.org/en/docs/stream/ngx_stream_ssl_preread_module.html

    #########################################################################
    # Connections on 443 could be raw socket or https / wss
    # Distinguish between the two based on the SNI (preread server name)
    # Server names with -s are raw socket connections
    # Server names without -s are https or wss connections
    map $ssl_preread_server_name $internalport {
        presence-s.myglance.org      5502;
        presence.myglance.org        4443;
        default                      glance-no-upstream-instance;
    }

    # Note this does not terminate the ssl connection.  It just reads the SNI
    # header and forwards to the appropriate port
    server {
        listen                  443;
        ssl_preread             on;
        proxy_connect_timeout   20s;  # max time to connect to pserver
        proxy_timeout           30s;  # max time between successive reads or writes
        proxy_pass              127.0.0.1:$internalport;
    }    

    server {
        listen                 5502 ssl;
        ssl_preread            off;
        proxy_pass             127.0.0.1:6502;
    }
}

score 2 · Answer 3 · edited Oct 21 '18 at 12:23

I have been in a great search for this, unfortunately, it is not possible as TCP has no concept of server names, so this is not possible. It only works in HTTP because the client sends the hostname it is trying to access as part of the request, allowing NGINX to match it to a specific server block.

Reference
Found on this thread

Very upsetting as I really would've liked to direct TCP traffic based on url

How to proxy multiple tcp streams on one port with nginx

3 Answers3