Nginx WebSocket reverse proxy keeps return 200 instead of 101

Question

I'm currently trying to have a hack.chat on my personal server working.

Long story short, it consists of two servers. The first is a simple httpd server serving javascript and CSS. The second one, the chat system, is a node.js server which the javascript connects to using websocket. And here comes the problems.

I want it all to use port 80, with a different domain name on a single IP, using a separate server block in Nginx.

I followed the Nginx websocket doc but this is not working. When the websocket tries to connect, it always gets a 200 return code whereas, if I understood well, it should get 101 (switching protocol).

My Nginx version is 1.8.0 and my server is running on gentoo with linux 4.0.5

Here is a dump of the relevant nginx conf files :

nginx.conf:

user nginx nginx;
worker_processes 1;

error_log /var/log/nginx/error_log info;

events {
    worker_connections 1024;
    use epoll;
}

http {
    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    log_format main
        '$remote_addr - $remote_user [$time_local] '
        '"$request" $status $bytes_sent '
        '"$http_referer" "$http_user_agent" '
        '"$gzip_ratio"';

    client_header_timeout 10m;
    client_body_timeout 10m;
    send_timeout 10m;

    connection_pool_size 256;
    client_header_buffer_size 1k;
    large_client_header_buffers 4 2k;
    request_pool_size 4k;

    gzip on;
    gzip_min_length 1100;
    gzip_buffers 4 8k;
    gzip_types text/plain;

    output_buffers 1 32k;
    postpone_output 1460;

    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;

    keepalive_timeout 75 20;

    ignore_invalid_headers on;

    include /etc/nginx/sites-enabled/*;
}

sites-enabled/chat :

map $http_upgrade $connection_upgrade{
    default upgrade;
    ''  close;
}

server{
    listen 0.0.0.0:80;
    server_name chat.axellink.fr;

    location / {
        proxy_pass http://127.0.0.1:6060;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
    }

    access_log /var/log/nginx/chat_access;
    error_log /var/log/nginx/chat_error debug;
}

When I take a look at the access_log, it effectively shows the 200 response and there is no error in the error_log. Unfortunately, the node.js server does not give me any log (or I don't know how to view it).

Thank you in advance for any response.

EDIT

Thx to mc0e, I managed to have the hack.chat server respond 101. By comparing what actually happens between nginx and node.js with what happens on a direct connection, I saw that on direct an header Upgrade: websocket is set but nginx does not. So I corrected my sites-enabled/chat to :

server{
    listen 0.0.0.0:80;
    server_name chat.axellink.fr;

    location / {
        proxy_pass http://127.0.0.1:6060;
        proxy_http_version 1.1;
        proxy_set_header Upgrade "websocket";
        proxy_set_header Connection "Upgrade";
    }

    access_log /var/log/nginx/chat_access;
    error_log /var/log/nginx/chat_error debug;
}

I also removed the map block as it used to set the Connection header to close instead of Upgrade.

Still doesn't work though. hack.chat gives back a 101 with Connection: Upgrade and Upgrade : websocket but somehow nginx gives back a 101 with Connection: keep-alive (as what I see in firefox) :/

EDIT

Did a ngrep on nginx communications, it does send the packet hack.chat gives back to him and firefox is complaining about cross-origin. I'll try to avoid cross origin.

LAST EDIT

Ok now that I'm home I did a test and it worked like a charm. Pretty sur that my last issue was because of my work proxy, let's hope it's cache or else everything would have been useless.

Here is the last of my conf that avoids cross origin :

server{
        listen 0.0.0.0:80;
        server_name chat.axellink.fr;

        location / {
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header Host $http_host;
                proxy_set_header X-NginX-Proxy true;
                proxy_pass http://127.0.0.1:8081;
        }

        location /chat/ {
                proxy_pass http://127.0.0.1:6060;
                proxy_http_version 1.1;
                proxy_set_header Upgrade "websocket";
                proxy_set_header Connection "Upgrade";
        }

        access_log /var/log/nginx/chat_access;
        error_log /var/log/nginx/chat_error debug;
}

Answer 1

From my reading of the nginx websocket dock, when your nginx proxy sees the Upgrade headers, it passes them on to hack.chat. hack.chat then needs to respond with the 101 response.

First of all, you want to be able to debug the actual HTTP and hack.chat transactions, so you know whether the problem is in your client app, or on the web server, or with the hack.chat communications. This will also allow you to get the exact HTTP response, which you say you currently don't know how to get at. IF the problem is server side though, you may still be looking to get better logs of what went wrong.

You can capture the http traffic with tcpdump or ngrep. I'd go for ngrep. Doing it like this will give you output in your terminal.

ngrep 'Host: chat.axellink.fr' port 80

You probably want to run that on your web server, though if you have a desktop environment that can run ngrep you might chose to run it there. If you run it on the server, you can also capture the interaction between nginx and your chat server.

ngrep '.' port 6060

The terminal output is often enough to see what you need, but you can also use the '-O' flag to ngrep to write the data to a pcap dump file as tcpdump does with '-w'. You can then bring that file back to your desktop to look at it with a graphical client like wireshark.

You might be able to spot the problem from this, but if not, capture the various interactions and add them to your question.

Is there a problem with javascript asking for the hack.chat communication on a different domain to the one the javascript is served from? This might be relevant: https://stackoverflow.com/questions/20093070/unable-to-create-cross-domain-websocket-connection-to-node-js-socket-io-server