How to set websocket reverse proxy with both proxy_set_header and SSL allowed?

Question

I get a project where the outer websocket reverse-proxy is implemented by nginx stream.

However, steam only means "TCP" and it lost http features like writing the IP's alone the route.

Here is the config of the outer layer in encrypted SSL:

stream{ 
        upstream mysvr {
            server 10.3.3.7:1111;
    }
        server {
            listen 3331; 
            ssl_preread on;
            # proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_pass mysvr;
    }
......

the proxy_set_header clause has to be commented out since nginx does not allow it here if not: "proxy_set_header" directive is not allowed here

And here is the configuration of nginx on 10.3.3.7 machine:

http{   
     server {
                listen 1111 ssl;
                server_name localhost;
                ssl_certificate /etc/nginx/cert/server.crt;
                ssl_certificate_key /etc/nginx/cert/server.key;
                ssl_session_timeout 5m;
                ssl_verify_client off;
        location / {
            proxy_pass http://10.3.3.5:8888;
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
        }
......

So I tried to rewrite the first and outer reverse proxy to http 1.1 type, the real websocket proxy instead of the TCP or "stream" connection. I moved everything out of the stream bracket and put it in http bracket and remove the comment before proxy_set_header, finally add in the the three websocket specific clauses at the end of the second and inner reverse proxy:

http {
       upstream mysvr {
            server 10.3.3.7:1111;
    }
       server {
           listen 3331; 
           ssl_preread on;
           proxy_set_header Host $http_host;
           proxy_http_version 1.1;
           proxy_set_header Upgrade $http_upgrade;
           proxy_set_header Connection "upgrade";
           proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
           proxy_pass mysvr;
    }
......

But this time the nginx throw out an error:[emerg] "ssl_preread" directive is not allowed

So the conflict is:

Under steam I cannot use proxy_set_header; while under http 1.1 I cannot use ssl_preread.

But they are both important features in this scenario:

The real backend server need to know the real client's IP thus need to stamp the IP at the outer server into the header; The outer proxy server is prohibited from reading the transparent traffic but allow the inner reverse-proxy to decoded it with SSL. Thus the traffic between the real backend 10.3.3.5 and the inner reverse-proxy 10.3.3.7 is transparent while the traffic from inner reverse-proxy through the outer reverse-proxy to the client is encoded with the same SSL by the nginx on inner reverse-proxy.

Additional feature is that the real backend 10.3.3.5 routes via 10.3.3.7 the inner reverse-proxy , which routes via the outer reverse-proxy. So the proxy vpn and reverse proxy routes are coherent.

Under this setting, is there a way to achieve the 2 features at the same time? Thanks for reading.

How to set websocket reverse proxy with both proxy_set_header and SSL allowed?

0 Answers0