Getting "real" IP in Nginx behind AWS network load balancer

Question

I have a network load balancer which is forwarding traffic to an Nginx docker container running in ECS (using awsvpc network mode). My nginx config is as follows:

user  nginx;
worker_processes  1;

error_log  /var/log/nginx/error.log info;
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    server {

        allow all;
        listen 443 default ssl;
        ssl_certificate /etc/nginx/ssl/cert.pem;
        ssl_certificate_key /etc/nginx/ssl/privkey.pem;
        ssl_trusted_certificate /etc/nginx/ssl/chain.pem;

        ssl_prefer_server_ciphers on;
        ssl_protocols TLSv1.2;

        add_header Strict-Transport-Security "max-age=31536000";
        access_log /var/log/nginx/access.log;

        location / {
            proxy_pass http://blahblahblah;
            proxy_set_header Host $host;
            proxy_set_header X-Forwarded-For $remote_addr, $proxy_add_x_forwarded_for;
        }
    }
}

However when looking at the logs the X-Forwarded-For header doesn't contain the client's "real" IP, it just contains a series of internal IPs (one of which is the network load balancer's internal IP).

My understanding of the Network Load Balancer was that the client's "real" IP was supposed to be preserved as the incoming IP, why is is not showing in this header?

Multiple IPs means multiple proxies. https://serverfault.com/questions/846489/can-x-forwarded-for-contain-multiple-ips — ceejayoz, Mar 05 '18 at 13:46
I agree that the X-Forwarded-For header will contain multiple IPs, I'm trying to work out why the client's "real" IP isn't showing in the header — thewire247, Mar 05 '18 at 13:49
It should actually be easy to tell if the IP is IPv4 or IPv6 since the addresses look nothing alike. — Ron Maupin, Mar 05 '18 at 15:53

score 1 · Answer 1 · answered Feb 22 '21 at 22:57

nginx started supporting PROXY protocol v2 since 1.13.11, which is about a month after you asked this question. I encountered this problem today, so in case this is useful for anyone else:

http {
    #...
    server {
        listen 80   proxy_protocol;
        listen 443  ssl proxy_protocol;
        #...
    }
}
   
stream {
    #...
    server {
        listen 12345 proxy_protocol;
        #...
    }
}

Add proxy_protocol to your listener. This will allow your nginx to accept Proxy Protocol V2 connections.

If you're using AWS LB, you need to enable Proxy protocol v2 on your target group:

("Preserve client IP addresses" option doesn't seem to be necessary, $proxy_protocol_addr appears correctly in logs).

score 1 · Answer 2 · answered Mar 05 '18 at 14:03

1

https://aws.amazon.com/blogs/aws/new-network-load-balancer-effortless-scaling-to-millions-of-requests-per-second/

Source Address Preservation – With Network Load Balancer, the original source IP address and source ports for the incoming connections remain unmodified, so application software need not support X-Forwarded-For, proxy protocol, or other workarounds. This also means that normal firewall rules, including VPC Security Groups, can be used on targets.

answered Mar 05 '18 at 14:03

ceejayoz

32,469
7
81
105

Thats what I expected to happen. However I've just stumbled across this forum post: https://forums.aws.amazon.com/ann.jspa?annID=5162 It suggests that as the instances are registered by IP (due to awsvpc network type in ECS), that the source IP isn't preserved. I've looked at extracting the IP using proxy protocol however it only seems to support v2, nginx however only seems to support v1 so I'm a bit stuffed there – thewire247 Mar 05 '18 at 14:13
1

You're probably in "contact support" territory, then. – ceejayoz Mar 05 '18 at 14:45

Getting "real" IP in Nginx behind AWS network load balancer

2 Answers2