NGINX: Send a copy of requests to another upstream

Question

Our team has developed an analytic app which records and analyses all incoming (not dropped) requests. To that extent, I have to send a copy of all requests to another upstream in my reverse-proxy solution.

Is it possible with NGINX alone? Or is there any other solution with minimum impact on the current reverse-proxy structure?

It is preferable to do so within NGINX so that processes on requests by NGINX is already done and the request will be the same as it is sent to destination upstream.

PS: The analytic app does not return any response

Seems like there was a topic with an answer to that question http://serverfault.com/questions/515284/copy-and-deliver-a-request-to-another-nginx-server-for-real-traffic-testing/515531#515531 — iMil, Aug 06 '16 at 14:16
@iMil post_action is not documented and there is a reason. It clogs up the connection until the post_action is done and reduces performance. Since it is on a reverse proxy server it severely damages the delivery time of all services. — Cunning, Aug 07 '16 at 08:57
I came across this blog post from nginx.com that might be of interest in your situation https://www.nginx.com/blog/performing-a-b-testing-nginx-plus/ — iMil, Aug 07 '16 at 16:53
@iMil Thanks! This is an interesting article. It is somewhat different with A/B testing in which I need all requests mirrored into two destinations. I have managed to do a brainstorm with network staff and lower level solutions were offered like port mirroring on a switch or NetFlow/sFlow solutions, syslog parsers and such. Thanks again! I think I leave this question open in case a better solution emerges. — Cunning, Aug 09 '16 at 06:45
speaking of, you also could do port mirroring on your server, it's a bit tricky but can be achieved, here's well written blog post on the subject https://adamkuj.net/blog/2016/05/18/pro-tip-port-mirroring-in-linux/ — iMil, Aug 09 '16 at 08:06
After some searhing in nginx docs I found a directory mirror in http://nginx.org/ru/docs/http/ngx_http_mirror_module.html. Think, this is what u need. — aligin, Jul 18 '19 at 12:02
@aligin I think that's the answer. Thank you! and if you wouldn't mind, submit this as an answer so It could be marked as the solution. Thanks again! — Cunning, Jul 19 '19 at 10:28

score 3 · Accepted Answer · edited Jul 22 '19 at 09:25

After some searhing in nginx docs I found a directory mirror in nginx.org. Think, this is what u need.

We are using kubernetes with nginx as ingress, so our config is

      location /api/what-i-want-to-mirror {
          set $proxy_upstream_name "gateway-api-service-svc-80";
          proxy_pass http://upstream_balancer;
          mirror /mirror;

          proxy_set_header Host $host;
          proxy_set_header X-Real-IP $remote_addr;
          proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      }

      location = /mirror {
          set $proxy_upstream_name "integration-gateway-api-service-svc-80";
          proxy_pass http://upstream_balancer;
          internal;
          proxy_set_header Host $host;
          proxy_set_header X-Real-IP $remote_addr;
          proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

          proxy_set_header X-Original-URI $request_uri;
      }

We've done this as ASAP but it works like a charm.

NGINX: Send a copy of requests to another upstream

1 Answers1