nginx conf: http2 module not working in Chrome in ubuntu 18.04

Question

I'm trying to setup the http2 module in nginx on a fresh Ubuntu 18.04 vps. Http2 works fine in Firefox (I checked the response headers). Some third party websites indicate that my http2 configuration is ok:

• https://tools.keycdn.com/http2-test
• https://www.ssllabs.com/ssltest/analyze.html?d=example.com (I've got A+ on this site)

But when I try to open the website on Chrome, it won't even open (This site can’t be reached), although in the access log (/var/log/nginx/access.log) I get an "HTTP/2.0" 200 response from nginx, indicating that nginx behaves as if everything was ok.

From curl, I've this answer:

$ curl -I https://example.com/
HTTP/1.1 200 OK
Server: nginx/1.14.0 (Ubuntu)
Date: Sun, 19 May 2019 13:20:33 GMT
Content-Type: text/html; charset=UTF-8
Connection: keep-alive
Vary: Accept-Encoding
X-Frame-Options: SAMEORIGIN
X-Content-Type-Options:: nosniff
X-XSS-Protection: 1;mode=block
Strict-Transport-Security: max-age=15768000

So, not http2, but at least I've a fallback to http1.1 with curl

So, apparently http2 works, but not in Chrome (blank page), and I don't know what's the problem.

If forgot to say that when I browse the page with Chrome, I don't get any error in the error log (/var/log/nginx/error.log), however, from time to time, an ssl related error appears (here is a sample):

2019/05/19 14:53:33 [crit] 11931#11931: *639 SSL_do_handshake() failed (SSL: error:1417D102:SSL routines:tls_process_client_hello:unsupported protocol) while SSL handshaking, client: 64.41.200.103, server: 0.0.0.0:443
2019/05/19 14:56:34 [notice] 12616#12616: signal process started
2019/05/19 15:06:52 [notice] 12638#12638: signal process started
2019/05/19 15:08:48 [notice] 12647#12647: signal process started
2019/05/19 15:10:02 [notice] 12724#12724: signal process started
2019/05/19 15:10:07 [crit] 12725#12725: *706 SSL_do_handshake() failed (SSL: error:1417D102:SSL routines:tls_process_client_hello:unsupported protocol) while SSL handshaking, client: 91.107.64.185, server: 0.0.0.0:443
2019/05/19 15:10:07 [crit] 12725#12725: *707 SSL_do_handshake() failed (SSL: error:1417D18C:SSL routines:tls_process_client_hello:version too low) while SSL handshaking, client: 91.107.64.185, server: 0.0.0.0:443
2019/05/19 15:25:22 [crit] 12725#12725: *724 SSL_do_handshake() failed (SSL: error:1417D102:SSL routines:tls_process_client_hello:unsupported protocol) while SSL handshaking, client: 80.82.77.139, server: 0.0.0.0:443
2019/05/19 15:25:23 [crit] 12725#12725: *726 SSL_do_handshake() failed (SSL: error:1417D18C:SSL routines:tls_process_client_hello:version too low) while SSL handshaking, client: 80.82.77.139, server: 0.0.0.0:443
2019/05/19 15:25:23 [crit] 12725#12725: *727 SSL_do_handshake() failed (SSL: error:1417D102:SSL routines:tls_process_client_hello:unsupported protocol) while SSL handshaking, client: 80.82.77.139, server: 0.0.0.0:443
2019/05/19 15:25:27 [error] 12725#12725: *735 open() "/home/me/example.com/www/sitemap.xml" failed (2: No such file or directory), client: 80.82.77.139, server: example.com, request: "GET /sitemap.xml HTTP/1.1", host: "123.456.789.123"
2019/05/19 15:37:22 [notice] 12745#12745: signal process started

Again, the errors are not provoked by me refreshing the page in Chrome (they are probably bots or other people accessing the website).

Here is my configuration:

In /etc/nginx/nginx.conf:

user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
        worker_connections 1024;
        # multi_accept on;
}

http {

        ##
        # Basic Settings
        ##

        sendfile on;
        tcp_nopush on;
        tcp_nodelay on;
        keepalive_timeout 65;
        types_hash_max_size 2048;
        server_tokens off;


        ##
        # Security settings
        ##

        # Avoid iframes for clickjacking attacks
        # add_header X-Frame-Options DENY;
        add_header X-Frame-Options SAMEORIGIN;

        # Avoid mime type sniffing
        add_header X-Content-Type-Options: nosniff;

        # Avoid certain type of XSS attacks (if browser understands it)
        add_header X-XSS-Protection "1;mode=block";

        ##
        # DoS and DDoS Protection Settings
        ##

        #Define limit connection zone called conn_limit_per_ip with memory size 15m based on the unique IP
        limit_conn_zone $binary_remote_addr zone=conn_limit_per_ip:15m;

        #Define limit request to 40/sec in zone called req_limit_per_ip memory size 15m based on IP
        limit_req_zone $binary_remote_addr zone=req_limit_per_ip:15m rate=40r/s;

        #Using the zone called conn_limit_per_ip with max 40 connections per IP
        limit_conn conn_limit_per_ip 40;

        #Using the zone req_limit_per_ip with an exceed queue of size 40 without delay for the 40 additonal
        limit_req zone=req_limit_per_ip burst=40 nodelay;

        #Do not wait for the client body or headers more than 5s (avoid slowloris attack)
        client_body_timeout 5s;
        client_header_timeout 5s;
        send_timeout 5s;

        #Establishing body and headers max size to avoid overloading the server I/O
        client_body_buffer_size 256k;
        client_header_buffer_size 2k;
        client_max_body_size 3m;
        large_client_header_buffers 2 2k;


        # server_names_hash_bucket_size 64;
        # server_name_in_redirect off;

        include /etc/nginx/mime.types;
        default_type application/octet-stream;

        ##
        # SSL Settings
        ##

        ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
        ssl_prefer_server_ciphers on;

        ##
        # Logging Settings
        ##

        access_log /var/log/nginx/access.log;
        error_log /var/log/nginx/error.log;

        ##
        # Gzip Settings
        ##

        gzip on;

        gzip_vary on;
        # gzip_proxied any;
        gzip_comp_level 4;
        gzip_buffers 16 8k;
        gzip_http_version 1.1;
        gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

        ##
        # Virtual Host Configs
        ##

        include /etc/nginx/conf.d/*.conf;
        include /etc/nginx/sites-enabled/*;
}

And in (symlink) /etc/nginx/sites-enabled/example.com:

server {

        listen 80 default_server;
        listen [::]:80 default_server;

        # Prevent site from being displayed under a different domain (by creating another domain pointing to our server)
        return 301 https://example.com;
}


server {
    listen 80;
    listen [::]:80;

    server_name mysite.com;

    # Redirect all HTTP requests to HTTPS with a 301 Moved Permanently response.
    return 301 https://example.com$request_uri;
}


server {

        listen 443 ssl http2;
        listen [::]:443 ssl http2;



        # certs sent to the client in SERVER HELLO are concatenated in ssl_certificate
        ssl_certificate /etc/nginx/certs/mysite.com/fullchain.pem;
        ssl_certificate_key /etc/nginx/certs/mysite.com/key.pem;
        ssl_session_timeout 1d;
        ssl_session_cache shared:SSL:50m;
        ssl_session_tickets off;

        # modern configuration. tweak to your needs.
        ssl_protocols TLSv1.2;
        ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256';
        ssl_prefer_server_ciphers on;


        ##
        # Security settings
        ##

        # Avoid iframes for clickjacking attacks
        # add_header X-Frame-Options DENY;
        add_header X-Frame-Options SAMEORIGIN;

        # Avoid mime type sniffing
        add_header X-Content-Type-Options: nosniff;

        # Avoid certain type of XSS attacks (if browser understands it)
        add_header X-XSS-Protection "1;mode=block";

        # HSTS (ngx_http_headers_module is required) (15768000 seconds = 6 months)
        add_header Strict-Transport-Security max-age=15768000;


        # OCSP Stapling ---
        # fetch OCSP records from URL in ssl_certificate and cache them
        ssl_stapling on;
        ssl_stapling_verify on;

        ## verify chain of trust of OCSP response using Root CA and Intermediate certs
        ssl_trusted_certificate /etc/nginx/certs/mysite.com/ca.pem;



        root /home/me/example.com/www;

        # Add index.php to the list if you are using PHP
        index index.php index.html;

        server_name mysite.com;
        location / {
                # First attempt to serve request as file, then
                # as directory, then fall back to displaying a 404.
                #try_files $uri $uri/ =404;
                try_files $uri $uri/ /index.php?$query_string;
        }


        # pass PHP scripts to FastCGI server
        location ~ \.php$ {
                include snippets/fastcgi-php.conf;

                # With php-fpm (or other unix sockets):
                fastcgi_pass unix:/var/run/php/php7.2-fpm.sock;

        }

        location ~* \.(jpg|jpeg|gif|png|css|js|ico|xml)$ {
                access_log        off;
                add_header Cache-Control public;
                add_header Pragma public;
                add_header Vary Accept-Encoding;
                expires 365d;

        }


        # deny access to .htaccess files, if Apache's document root
        # concurs with nginx's one
        #
        location ~ /\.ht {
                deny all;
        }

        location ~ /\.git {
                deny all;
        }
}

Here is the output of:

root@example.com:/etc/nginx# nginx -V
nginx version: nginx/1.14.0 (Ubuntu)
built with OpenSSL 1.1.0g  2 Nov 2017
TLS SNI support enabled
configure arguments: --with-cc-opt='-g -O2 -fdebug-prefix-map=/build/nginx-FIJPpj/nginx-1.14.0=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC -Wdate-time -D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -fPIC' --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --modules-path=/usr/lib/nginx/modules --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-debug --with-pcre-jit --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_v2_module --with-http_dav_module --with-http_slice_module --with-threads --with-http_addition_module --with-http_geoip_module=dynamic --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module=dynamic --with-http_sub_module --with-http_xslt_module=dynamic --with-stream=dynamic --with-stream_ssl_module --with-mail=dynamic --with-mail_ssl_module

I've created the certificate using Certbot, and it worked fine.

What am I missing?

Answer 1

I suspect it’s this:

# Avoid mime type sniffing
add_header X-Content-Type-Options: nosniff;

HTTP/2 is a bit more strict about HTTP Headers than HTTP/1.1 was and in this header, unlike your others, you have included a colon in the header name which is a mistake. This leads to a double colon in the output:

X-Frame-Options: SAMEORIGIN
X-Content-Type-Options:: nosniff
X-XSS-Protection: 1;mode=block

Chrome rejects invalid headers like this. See this article on how to debug this to see if it was that: https://www.michalspacek.com/chrome-err_spdy_protocol_error-and-an-invalid-http-header But suspect when you correct that it will work.