0

I have a Ubuntu 16.04 environment with Docker on Digital Ocean and I am booting a service using docker-compose:

version: '2'
services:
  cis-api:
    image: docker.myserver.com/cis-api:latest
    ports:
      - 8080:5000
    environment:
      DB_CONN_STRING: 'Server=tcp:...;Initial Catalog=cis-stage;Persist Security Info=False;User ID=...;Password=...;MultipleActiveResultSets=False;Encrypt=True;TrustServerCertificate=False;Connection Timeout=30;'

Then I have the vanilla Nginx installed with one config inside /etc/nginx/conf.d:

upstream cis-api {
  server 0.0.0.0:8080;
}

server {
  listen 80;
  server_name cis-api.myserver.com;

  location / {
    proxy_pass                          http://cis-api;
    proxy_set_header  Host              $http_host;   # required for docker client's sake
    proxy_set_header  X-Real-IP         $remote_addr; # pass on real client's IP
    proxy_set_header  X-Forwarded-For   $proxy_add_x_forwarded_for;
    proxy_set_header  X-Forwarded-Proto $scheme;
    proxy_read_timeout                  900;
  }
}

When I try to request a URL bypassing nginx, everything works:

root@cis-stage:/etc/nginx# curl localhost:8080/businesses -H 'Authorization: Bearer ...'
[{"id":1,...,"email":null}]

However if I try to do the exact same request going through nginx the connection hangs, or if I set a 30s timeout it times out:

root@cis-stage:/etc/nginx# curl --max-time 30 --connect-timeout 30 http://cis-api.mysever.com/businesses -H 'Authorization: Bearer ....'

curl: (28) Operation timed out after 30000 milliseconds with 0 bytes received

The app is written in ASP.Net Core 1.1 and here's the output difference when I run it bypassing nginx:

cis-api_1  | dbug: Microsoft.AspNetCore.Server.Kestrel[1]
cis-api_1  |       Connection id "0HL3OL7D1CF19" started.
cis-api_1  | info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
cis-api_1  |       Request starting HTTP/1.1 GET http://localhost:8080/businesses
cis-api_1  | info: Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerMiddleware[2]
cis-api_1  |       Successfully validated the token.
cis-api_1  | info: Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerMiddleware[3]
cis-api_1  |       HttpContext.User merged via AutomaticAuthentication from authenticationScheme: Bearer.
cis-api_1  | info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2]
cis-api_1  |       Request finished in 414.6261ms 200 application/json; charset=utf-8
cis-api_1  | dbug: Microsoft.AspNetCore.Server.Kestrel[6]
cis-api_1  |       Connection id "0HL3OL7D1CF19" received FIN.
cis-api_1  | dbug: Microsoft.AspNetCore.Server.Kestrel[9]
cis-api_1  |       Connection id "0HL3OL7D1CF19" completed keep alive response.
cis-api_1  | dbug: Microsoft.AspNetCore.Server.Kestrel[10]
cis-api_1  |       Connection id "0HL3OL7D1CF19" disconnecting.
cis-api_1  | dbug: Microsoft.AspNetCore.Server.Kestrel[7]
cis-api_1  |       Connection id "0HL3OL7D1CF19" sending FIN.
cis-api_1  | dbug: Microsoft.AspNetCore.Server.Kestrel[8]
cis-api_1  |       Connection id "0HL3OL7D1CF19" sent FIN with status "0".
cis-api_1  | dbug: Microsoft.AspNetCore.Server.Kestrel[2]
cis-api_1  |       Connection id "0HL3OL7D1CF19" stopped.

Vs when I run using nginx as a proxy:

cis-api_1  | dbug: Microsoft.AspNetCore.Server.Kestrel[1]
cis-api_1  |       Connection id "0HL3OL7D1CF1B" started.
cis-api_1  | info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
cis-api_1  |       Request starting HTTP/1.0 GET http://cis-api.myserver.com/businesses
cis-api_1  | info: Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerMiddleware[2]
cis-api_1  |       Successfully validated the token.
cis-api_1  | info: Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerMiddleware[3]
cis-api_1  |       HttpContext.User merged via AutomaticAuthentication from authenticationScheme: Bearer.

It hangs here until the request times out, and then:

cis-api_1  | dbug: Microsoft.AspNetCore.Server.Kestrel[6]
cis-api_1  |       Connection id "0HL3OLAD6F4QN" received FIN.
cis-api_1  | info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2]
cis-api_1  |       Request finished in 4615.4522ms 200 application/json; charset=utf-8
cis-api_1  | dbug: Microsoft.AspNetCore.Server.Kestrel[10]
cis-api_1  |       Connection id "0HL3OLAD6F4QN" disconnecting.
cis-api_1  | dbug: Microsoft.AspNetCore.Server.Kestrel[7]
cis-api_1  |       Connection id "0HL3OLAD6F4QN" sending FIN.
cis-api_1  | dbug: Microsoft.AspNetCore.Server.Kestrel[8]
cis-api_1  |       Connection id "0HL3OLAD6F4QN" sent FIN with status "-107".
cis-api_1  | dbug: Microsoft.AspNetCore.Server.Kestrel[2]
cis-api_1  |       Connection id "0HL3OLAD6F4QN" stopped.

Any ideas?

kolrie
  • 225
  • 3
  • 12

1 Answers1

1

After a lot of research I found this answer here:

https://github.com/aspnet/KestrelHttpServer/issues/468#issuecomment-165951249

And the problem got solved by changing the nginx config file to add a new Proxy header:

proxy_set_header  Connection        keep-alive;

The new location block became:

  location / {
    proxy_pass                          http://cis-api;
    proxy_set_header  Host              $http_host;   # required for docker client's sake
    proxy_set_header  X-Real-IP         $remote_addr; # pass on real client's IP
    proxy_set_header  X-Forwarded-For   $proxy_add_x_forwarded_for;
    proxy_set_header  X-Forwarded-Proto $scheme;
    proxy_set_header  Connection        keep-alive;
    proxy_read_timeout                  900;
  }

Now everything works as expected

kolrie
  • 225
  • 3
  • 12