Rewriting client "Host" http header before nginx validates the field

Question

I get following request from a device and should proxy it to hd10.vtech.com:

   GET http://hd10.vtech.com/test/pp_firmware/HD10-CH010_SUOTA.bin HTTP/1.1
   Host: http://hd10.vtech.com/test/pp_firmware/HD10-CH010_SUOTA.bin
   Range: bytes=0-59

Unfortunately nginx decides, that the client host header field is not valid and throws a 400 error. Is there a way for me to rewrite the client host header before nginx/openresty validates the request? I tried the more_set_input_headers routine to modify the header but this takes place after validation...

nginx.conf(TESTING):

user  nobody;
#number of cores:
#grep processor /proc/cpuinfo | wc -l
worker_processes  2;

#
pid        logs/nginx.pid;

#daemon off;
error_log /var/log/nginx.log info;

events {
#number of files that can be opened simultaniously by a process:
#ulimit -n
    worker_connections  1024;
}


http {
    include       mime.types;
    default_type  application/octet-stream;
    # note that the log_format directly below is a single line
    log_format main '[$time_local] remote_ip: $remote_addr realip: $realip_remote_addr remote_user: $remote_user  request: "$request" status: $status body_byte: $body_bytes_sent referer: "$http_referer" agent: "$http_user_agent" proxy: "$proxy_host" host: "$host"';

    access_log  /var/log/access.log  main;

    ignore_invalid_headers on;
    sendfile        on;
    #tcp_nopush     on;

    #context should be upstream - don't know if applied
    #keepalive_timeout  0;
    keepalive_timeout  15;

    #gzip  on;
    lua_package_path "/usr/local/openresty/lua-resty-http/lib/?.lua;/usr/local/openresty/src/?.lua;;";
    lua_package_cpath "/usr/local/openresty/lualib/?.so;;";
    #lua_code_cache off;
    lua_shared_dict whitelist 500k;
    lua_shared_dict useragent 100k;
    lua_shared_dict captive 100k;
    lua_shared_dict redirecttable 100k;
    lua_shared_dict clients 2m;

    init_by_lua_block {
        require("initialize").go()
        }

    server {
        lua_socket_connect_timeout 5m;
        proxy_connect_timeout   15;
        proxy_set_header HOST $host;
        proxy_buffering off;
        proxy_set_header Connection "";
        proxy_http_version 1.1;

        #proxy_ignore_client_abort on;
        listen 172.16.23.238:8080;
        ###security hardening
        #removes version of webserver in response headers
        server_tokens off;
        #overwrites Server headers
        more_set_headers 'Server: Webproxy';
        #prevents clickjacking - debatable if the proxy should enforce this
        add_header X-Frame-Options "SAMEORIGIN";
        #protects clients with Webkit browsers (IE8+) from XSS attacks
        add_header X-XSS-Protection "1; mode=block";
        #limits some types from turning into executable code (e.g style can only be text/css, if it's something else it is blocked)
        add_header X-Content-Type-Options nosniff;
        #proxy headers - passes useful info and original ip
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_bind 62.202.200.246;

        #delete if breaks stuff
        proxy_read_timeout 15;
        proxy_send_timeout 15;

        location / {

            resolver 1.1.1.1 10.212.10.10 ipv6=off;  #
            set $target '';

            access_by_lua_block {
            require("sharedmemory").go()
            require("useragent").go()
            }
            #pass remote Server in Header - currently overwritten by more_set_header
            proxy_pass_header Server;
            proxy_pass http://$target;
        }
  }

}

Thanks and regards David

Please post your nginx config and a log of the request to see what is going on. — Drifter104, Sep 14 '18 at 12:20
That's because **the Host: header _is not valid_**. Talk to the device vendor and get it fixed. Nothing at all should be expected to communicate properly with this device in its current state. If necessary, quote RFC chapter and verse to them and hold them in breach of contract. — Michael Hampton, Sep 14 '18 at 13:25
@MichaelHampton: Yes i agree, but `1. If Request-URI is an absoluteURI, the host is part of the Request-URI. Any Host header field value in the request MUST be ignored.` from [https://www.w3.org/Protocols/rfc2616/rfc2616-sec5.html] - could also mean an invalid host header should be ignored... I can test again on monday. thanks for the comments. — gspoosi, Sep 15 '18 at 09:48
Er, you should quote _current_ RFCs, not _obsolete_ ones. [RFC 7230 section 5.4](https://tools.ietf.org/html/rfc7230#section-5.4) makes it clear that only an HTTP _proxy_ needs to ignore the Host: header when an absolute URI is given in the request line. Of course, an origin server _should_ try to make sense of such a request, and you might get somewhere by filing a bug report on nginx, but the device is still broken. — Michael Hampton, Sep 15 '18 at 17:03
I agree on the broken device. BUT the request above is the request that upgrades these devices, so for the devices to get fixed, I NEED to handle the broken requests :) If I can't we have to omit the proxy which is ugly too. — gspoosi, Sep 17 '18 at 09:09

score 2 · Answer 1 · answered Sep 14 '18 at 13:14

2

Your Host header has an underscore in it. The default value for the directive underscores_in_headers is Off. So you can try setting it to On.

answered Sep 14 '18 at 13:14

Tommiie

5,547
2
11
45

This does not solve the problem. Default value in `On` btw. – gspoosi Sep 17 '18 at 07:54
Sorry. According to the documentation the default is `Off` so I thought it would help here. https://nginx.org/en/docs/http/ngx_http_core_module.html#underscores_in_headers – Tommiie Sep 17 '18 at 07:56
you are right. ignore_invalid_headers is default `On`. – gspoosi Sep 17 '18 at 09:04
That is not what I said. – Tommiie Sep 17 '18 at 09:07

Rewriting client "Host" http header before nginx validates the field

1 Answers1