Nginx location blocks breaking Meteor with reverse proxy

Question

I have two servers:

(A) Reverse Nginx proxy, which routes requests internally or to B
(B) Meteor deployment using meteor-up version 1.2.11 (Nginx inside Docker)

I have two upstream destinations, one on each server:

upstream remote-app {
  server 123.45.67.890:8080;
}

upstream local-app {
  server localhost:7000;
}

If I use separate server blocks, everything works fine.

server {
  listen 80;
  server_name local.* local.myapp.com;
  access_log /var/log/nginx/local.myapp.access.log;
  error_log /var/log/nginx/local.myapp.error.log debug;
  error_page 404 /4xx.html;
  error_page 500 502 503 504 /5xx.html;

  location / {
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarder-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_set_header X-NginX-Proxy true;

    proxy_pass http://local-app;
    proxy_redirect off;

    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
  }
}

server {
  listen 80;
  server_name remote.* remote.myapp.com;
  access_log /var/log/nginx/remote.myapp.access.log trace;
  error_log /var/log/nginx/remote.myapp.error.log debug;
  error_page 404 /4xx.html;
  error_page 500 502 503 504 /5xx.html;

  location / {
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarder-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_set_header X-NginX-Proxy true;

    proxy_pass http://remote-app;
    proxy_redirect off;

    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";

    add_header Cache-Control no-cache;
  }
}

But I need to route conditionally based on the path, not just the subdomain. So I try adding a location block to redirect some of the local traffic to the remote server.

server {
  listen 80;
  server_name local.* local.myapp.com;
  access_log /var/log/nginx/local.myapp.access.log;
  error_log /var/log/nginx/local.myapp.error.log debug;
  error_page 404 /4xx.html;
  error_page 500 502 503 504 /5xx.html;

  location = /broken {
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarder-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_set_header X-NginX-Proxy true;

    proxy_pass http://remote-app;
    proxy_redirect off;

    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
  }

  location / {
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarder-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_set_header X-NginX-Proxy true;

    proxy_pass http://local-app;
    proxy_redirect off;

    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
  }
}

Now when I visit http://local.myapp.com/broken, I get a blank white screen with a ton of console errors:

The SyntaxError: Unexpected token < appears to be the browser trying to read javascript from the html file.

I can't figure out how to debug this. I see readv() not ready (11: Resource temporarily unavailable) in server A's Nginx error log, but I don't know if that is related. I don't know how to trace the request on server B because the deployment is in a Docker container. Any suggestions?

score 0 · Answer 1 · answered Oct 02 '17 at 19:48

After receiving some guidance from @zodern, I've realized that the initial request to /broken is handled correctly, but Meteor is not serving up a single response with all of the js and css. It serves an html file, which then causes 90 other requests back to server A. The location block handling all of those requests is the one with no modifiers and a root path /, which is sending everything to the local upstream app. So I'm not sure that it will be possible to run two different Meteor apps on different servers based on the requested path.

Nginx location blocks breaking Meteor with reverse proxy

1 Answers1