nginx - bind() to 0.0.0.0:80 failed (98: Address already in use)

Question

I've got a reverse proxy with 5 server configs set up, however since the start it's always been complaining about not being able to listen on the ports I specified so the server has been unable to start.

I've tried a few things:

Split config into multiple files inside sites-available and creating symlinks to sites-enabled
Delete the listen property and allowing it to fall back to defaults
Changing the ports to ipv6only=on - this caused an error
Changing the ports to [::]:80
Changing the ports so that there are no conflicts, ie, 80, 81, 82, etc.

Using netstat I can see that nothing is running on the ports I need, and when I run nginx -t I get a successful output saying syntax is ok and nginx.conf test is successful.

This is my nginx output:

nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address already in use)
nginx: [emerg] bind() to 0.0.0.0:81 failed (98: Address already in use)
nginx: [emerg] bind() to 0.0.0.0:82 failed (98: Address already in use)
nginx: [emerg] bind() to 0.0.0.0:83 failed (98: Address already in use)
nginx: [emerg] bind() to 0.0.0.0:84 failed (98: Address already in use)

Does anybody know why this is caused? Thanks in advance

Edit: lsof -Pni | grep LISTEN output:

sshd      1288     root    3u  IPv4  15712      0t0  TCP *:22 (LISTEN)
sshd      1288     root    4u  IPv6  15714      0t0  TCP *:22 (LISTEN)
nginx    14025     root    6u  IPv6 852470      0t0  TCP *:80 (LISTEN)
nginx    14025     root    7u  IPv6 852471      0t0  TCP *:443 (LISTEN)
nginx    14026 www-data    6u  IPv6 852470      0t0  TCP *:80 (LISTEN)
nginx    14026 www-data    7u  IPv6 852471      0t0  TCP *:443 (LISTEN)
dnsmasq  23894  dnsmasq    5u  IPv4 650875      0t0  TCP *:53 (LISTEN)
dnsmasq  23894  dnsmasq    7u  IPv6 650877      0t0  TCP *:53 (LISTEN)

Here's my server configs:

server {
        listen 80;
        server_name hac-staging-proxy.redant.cloud;

        location / {
                proxy_pass http://195.219.8.212/;
                proxy_set_header Host www.uat2prd.halfordsautocentres.com;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_buffering off;
        }
}

server {
        listen 81;
        server_name halfords-c1-staging-proxy.redant.cloud;

        location / {
                proxy_pass http://195.219.8.206/;
                proxy_set_header Host www.c1.uat2prd.halfordsautocentres.com;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_buffering off;
        }

}

server {
        listen 82;
        server_name halfords-c2-staging-proxy.redant.cloud;

        location / {
                proxy_pass http://195.219.8.206/;
                proxy_set_header Host www.c2.uat2prd.halfordsautocentres.com;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_buffering off;
        }

}

server {
        listen 83;
        server_name halfords-staging-proxy.redant.cloud;

        proxy_redirect http://www.uat2prd.halfords.com http://halfords-staging-proxy.redant.cloud;
        proxy_redirect https://www.uat2prd.halfords.com https://halfords-staging-proxy.redant.cloud;

        location / {
                add_header Set-Cookie Experiment=FH;
                resolver 127.0.0.1;
                proxy_pass http://www.uat2prd.halfords.com;
                # sub_filter_types *;
                sub_filter 'www.uat2prd.halfords.com' 'halfords-staging-proxy.redant.cloud';
                sub_filter 'www.c1.uat2prd.halfords.com' 'halfords-c1-staging-proxy.redant.cloud';
                sub_filter 'www.c2.uat2prd.halfords.com' 'halfords-c2-staging-proxy.redant.cloud';
                sub_filter_once off;
                proxy_set_header Set-Cookie Experiment=FH;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_buffering off;
        }
}

server {
        listen 84;
        server_name halfords-staging-proxy.redant.cloud2;

        proxy_redirect https://www.uat2prd.halfords.com https://halfords-staging-proxy.redant.cloud;
        proxy_redirect http://www.uat2prd.halfords.com http://halfords-staging-proxy.redant.cloud;

        ssl_certificate           /etc/nginx/domain.cert;
        ssl_certificate_key       /etc/nginx/domain.key;
        ssl_dhparam               /etc/ssl/certs/dhparam.pem;

        ssl on;
        ssl_session_cache  builtin:1000  shared:SSL:10m;
        ssl_protocols  TLSv1 TLSv1.1 TLSv1.2;
        ssl_ciphers "INTENTIONALLY REMOVED"
        ssl_prefer_server_ciphers on;

        location / {
                add_header Set-Cookie Experiment=FH;
                resolver 127.0.0.1;
                proxy_pass https://www.uat2prd.halfords.com;
                # proxy_ssl_session_reuse off;
                # sub_filter_types *;
                sub_filter 'www.uat2prd.halfords.com' 'halfords-staging-proxy.redant.cloud';
                sub_filter 'www.c1.uat2prd.halfords.com' 'halfords-c1-staging-proxy.redant.cloud';
                sub_filter 'www.c2.uat2prd.halfords.com' 'halfords-c2-staging-proxy.redant.cloud';
                sub_filter_once off;
                proxy_set_header Set-Cookie Experiment=FH;
                proxy_set_header Host www.uat2prd.halfords.com;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_buffering off;
        }
}

Kindly include the output of 'lsof -Pni | grep LISTEN' (or 'netstat -atnp'). Address already in use typically means either a) there is something actually listening on that address, or b) there is nothing LISTENING on the address, but there is still some old connections in TIME_WAIT state and the server software hasn't used the SO_REUSEADDR socket option (very unlikely for nginx, mentioned for completeness) — Cameron Kerr, Nov 02 '17 at 10:47
@CameronKerr I've edited my answer to include the `lsof` output. It showed 4 nginx processes running, I killed one of them and that killed them all, so I rebooted the server with `service nginx start` and I got the same error unfortunately. — JmJ, Nov 02 '17 at 10:52
Your problem already has a solution: https://chrisjean.com/fix-nginx-emerg-bind-to-80-failed-98-address-already-in-use/ — Cameron Kerr, Nov 02 '17 at 10:56

score 2 · Accepted Answer · answered Nov 02 '17 at 11:05

2

Don't have 'listen *:80' at the same time as 'listen [::]:80' (and similarly for port 443).

The reason being on a dual-stack IPv4/IPv6 host, if you listen on an IPv6 TCP port X, and there is nothing listening on IPv4 TCP port X, you will get both. If you see a connection come in as ::ffff:1.2.3.4 (where 1.2.3.4 is an IPv4 address), then this is what is happening. This is called an IPv4-mapped IPv6 addresses.

This, when nginx then goes to listen on the IPv4 port 80 (0.0.0.0:80), it will run into the fact that it also has effectively done so (via [::]:80).

There are plenty of implementation-specifics around this behaviour, but unless you are versed in network programming, I won't mention them here.

Hope that helps, Cameron

answered Nov 02 '17 at 11:05

Cameron Kerr

3,919
18
24

Thanks for your help earlier Cameron, you've been very helpful thus far. I don't have more than 1 listen directive in each server block, and I didn't have *:80 at the same time as [::]:80 either. The problem still occurs, however it does seem to be proxying my requests correctly regardless, although these warnings/errors are still a bit concerning. – JmJ Nov 03 '17 at 01:31
Accepting as this helped point us in the right direction (still getting warnings but the reverse proxy appears to be working!) Thanks Cameron. – JmJ Nov 07 '17 at 14:11

nginx - bind() to 0.0.0.0:80 failed (98: Address already in use)

1 Answers1

Linked