Mirroring Traffic with Nginx

Question

I need to duplicate/mirror the traffic comming to my server/resource.

I mean, I have server A and B and N (nginx) as webserver.

All the traffic comming to N/resource --> redirect to both A and B

Is it possible?

You probably don't want to do this. What are you _really_ trying to do? — Michael Hampton, Jun 06 '13 at 20:25
I have a use case. I have a client pushing data to my servers that our production infrastructure uses. I need the exact same data pushed to my dev servers to further development. The client is only capable of sending the data to 1 endpoint. Don't ask why, I don't know. — The Dude, Jun 07 '13 at 18:01
Oh I need to duplicate traffic, I mean the traficc will process by A, but B will recevid make offline process — Agus, Jun 07 '13 at 19:29
You want the HTTP-requests to be duplicated and the response of one machine to be discarded? — cljk, Jun 07 '13 at 21:16
I found another version of this question with an answer that helped me: http://serverfault.com/a/515531/175380 Basically, you set the location to be mirrored up in a `post_action` directive. This will run after the request has been satisfied by the production machine. — Adam Lukens, Jul 29 '15 at 05:18

score 15 · Answer 1 · answered Aug 29 '17 at 20:19

15

Nginx now has an http mirror module. Documentation is at https://nginx.org/en/docs/http/ngx_http_mirror_module.html

Example configuration from the documentation:

location / {
    mirror /mirror;
    proxy_pass http://backend;
}

location /mirror {
    internal;
    proxy_pass http://test_backend$request_uri;
}

answered Aug 29 '17 at 20:19

Slack Undertow

251
2
3

2

Is there a way that I can use a mirror even if one of the host is down, I have a use case for that. – Sagar Kharab Jan 29 '19 at 12:58

score 3 · Answer 2 · answered Nov 01 '17 at 21:17

Here is new feature from nginx (1.13.4): http://nginx.org/en/docs/http/ngx_http_mirror_module.html#mirror

The ngx_http_mirror_module module (1.13.4) implements mirroring of an original request by creating background mirror subrequests. Responses to mirror subrequests are ignored.

score 2 · Answer 3 · answered Jun 07 '13 at 18:51

2

Based on their README It looks that GOR could help you solving your problem.

I never used it before, just stumbled upon it this week, so good luck!

answered Jun 07 '13 at 18:51

Marcel

1,575
8
14

Hey on using GOR it just acknowledges with a http 200. So if he has a response object or validation he wants done with what the client is sending him I dont think this will work for him. As far as I can tell, and I'm new to Gor, you can't pass on the result object from one of your output segments. – PatrickWalker Sep 07 '15 at 12:41
We are testing GOR right now and we have a couple of problems. It interprets headers like: Location: //newpath generating requests like GET //newpath. Redirects loose original headers (including host useragent etc.) We are migrating to another tool. – Aalex Gabi Aug 27 '18 at 14:43

score 1 · Answer 4 · answered Aug 27 '16 at 05:15

1

you cannot mirror requests with NGINX but the request is available as a variable. You could use Lua scripting to send that variable to a third party server as a sub request though. http://nginx.org/en/docs/http/ngx_http_core_module.html#var_request

answered Aug 27 '16 at 05:15

Mairon

149
3
12

Would you have an example of this? I'm looking for exactly the same – Lobo May 18 '20 at 18:55

score 0 · Answer 5 · edited Apr 13 '17 at 12:13

After digging and digging I found a solution. You can see it here Copy and deliver a request to another nginx server for real traffic testing by @scari

Just thought I should mention it here to ease the search for other people. On my setup I didn't use uwsgi - instead I wanted to duplicate every incoming request to an additional servers (in addition to the backbones servers that was load balanced already.

Hope it will help,
Liron

Mirroring Traffic with Nginx

5 Answers5