I'm running instances of Symfony or Drupal websites on two Debian servers, with Nginx listening to 443, Varnish listening to 80 and passing to nginx on listening custom ports 80** for each vhost.
Recently I added a new website to one of the servers. Then I began to run in this quite documented error nginx: [emerg] bind() to [::]:80 failed (98: Address already in use).
Despite there is no nginx server block at all listening to :80 port, neither any server block without listen directive, Nginx began to listen on port 80 all together with the custom ports.
sudo netstat -tlpn| grep nginx
tcp 0 0 0.0.0.0:8080 0.0.0.0:* LISTEN 4191/nginx: master
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 4191/nginx: master
tcp 0 0 0.0.0.0:8081 0.0.0.0:* LISTEN 4191/nginx: master
tcp 0 0 x.x.x.x:8082 0.0.0.0:* LISTEN 4191/nginx: master
tcp 0 0 y.y.y.y:8083 0.0.0.0:* LISTEN 4191/nginx: master
tcp 0 0 z.z.z.z:8084 0.0.0.0:* LISTEN 4191/nginx: master
tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 4191/nginx: master
tcp 0 0 0.0.0.0:8000 0.0.0.0:* LISTEN 4191/nginx: master
tcp6 0 0 :::8080 :::* LISTEN 4191/nginx: master
tcp6 0 0 :::80 :::* LISTEN 4191/nginx: master
tcp6 0 0 :::8081 :::* LISTEN 4191/nginx: master
tcp6 0 0 :::443 :::* LISTEN 4191/nginx: master
tcp6 0 0 :::8000 :::* LISTEN 4191/nginx: master
I already read a bunch of questions and posts about handling dual-stack IPv4 and IPv6 correct new syntax, and tried , AFAIK, all possible syntaxes such as below, no way.
Working directive before crash : listen x.x.x.x:8082;
Tried adding listen [::]:8082 ipv6only=on; . No change.
I listed, and killed process many times with sudo fuser -k 80/tcp before restarting systemctl varnish, nginx, even daemon-reload...
Last, I checked my history but can't find what could have caused this sudden behavior. The lone point I'm not sure about is I changed a couple of sysctl.conf params, but hopefully reverted them, just in case, I'm not used to this part od administration : cat /etc/sysctl.conf | grep net.ipv4.conf
#net.ipv4.conf.default.rp_filter=1
#net.ipv4.conf.all.rp_filter=1
#net.ipv4.conf.all.accept_redirects = 0
# net.ipv4.conf.all.secure_redirects = 1
#net.ipv4.conf.all.send_redirects = 0
#net.ipv4.conf.all.accept_source_route = 0
#net.ipv4.conf.all.log_martians = 1
Here's my configuration.
cat /etc/nginx/nginx.conf (relevant 2 lines, no server block in it)
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
cat /etc/nginx/conf.d/default.conf
server {
listen 8000 default_server;
listen [::]:8000 ipv6only=on default_server;
server_name _;
listen 443 ssl default_server;
listen [::]:443 ssl ipv6only=on default_server;
}
One of the sites-available vhosts (they all follow exactly same pattern) :
server { # this block only redirects www to non www
listen x.x.x.x:443 ssl;
server_name www.example.com;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_certificate /var/www/clients/client0/web3/ssl/example.com-le.crt;
ssl_certificate_key /var/www/clients/client0/web3/ssl/example.com-le.key;
return 301 https://example.com$request_uri;
}
server {
listen x.x.x.x:443 ssl;
server_name example.com
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_certificate /var/www/clients/client0/web3/ssl/example.com-le.crt;
ssl_certificate_key /var/www/clients/client0/web3/ssl/example.com-le.key;
location / {
# Pass the request on to Varnish.
proxy_pass http://127.0.0.1;
# Pass some headers to the downstream server, so it can identify the host.
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# Tell any web apps like Drupal that the session is HTTPS.
proxy_set_header X-Forwarded-Proto https;
proxy_redirect off;
}
}
server {
listen x.x.x.x:8082;
# listen [::]:8082 ipv6only=on;
server_name example.com www.example.com;
root /var/www/example.com/web/public;
location / {
# try to serve file directly, fallback to index.php
try_files $uri /index.php$is_args$args;
}
location ~ ^/index\.php(/|$) {
fastcgi_pass 127.0.0.1:8998;
fastcgi_split_path_info ^(.+\.php)(/.*)$;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
fastcgi_param DOCUMENT_ROOT $realpath_root;
internal;
}
location ~ \.php$ {
# return 404;
}
error_log /var/log/ispconfig/httpd/example.com/error.log;
access_log /var/log/ispconfig/httpd/example.com/access.log combined;
location ~ /\. {
deny all;
}
location ^~ /.well-known/acme-challenge/ {
access_log off;
log_not_found off;
root /usr/local/ispconfig/interface/acme/;
autoindex off;
try_files $uri $uri/ =404;
}
location = /favicon.ico {
log_not_found off;
access_log off;
expires max;
add_header Cache-Control "public, must-revalidate, proxy-revalidate";
}
location = /robots.txt {
allow all;
log_not_found off;
access_log off;
}
}
cat /etc/default/varnish relevant part
DAEMON_OPTS="-a :80 \
-T localhost:6082 \
-f /etc/varnish/default.vcl \
-S /etc/varnish/secret \
-s malloc,3G"
I'm wondering what could have caused a config I'm working with since years to bug ?
I carefully studied these Q&A and a bunch of doc or posts, with no success : Nginx tries to run on port 80 but the configs have been removed ; Nginx will not start (Address already in use) ; nginx - bind() to 0.0.0.0:80 failed (98: Address already in use)
EDIT
Here's the output of nginx -T. (Since body is limited to 30000 characters I had to paste it in pastebin).
