Nginx / uWsgi / Django site can handle more traffic with rewrite URL

Question

I'm running a Django app, using uWsgi behind Nginx. I've been doing some performance tuning and load testing using ApacheBench and have discovered something unexpected which I wonder if someone could explain for me. In my Nginx config I have a rewrite directive which catches lots of different URL permutations and then forwards them to the canonical URL I wish to use, eg, it traps www.mysite.com/whatever, www.mysite.co.uk/whatever and forwards them all to http://mysite.com/whatever.

If I load test against any of the URLs listed with a redirect (ie, NOT the canonical URL which it is eventually forwarded to), it can serve 15000 concurrent connections without breaking a sweat.

If I load test against the canonical URL, which the above test I would have expected got forwarded to anyway, it can't handle nearly as much. It will drop about 4000 of the 15000 requests, and can only handle about 9000 reliably.

This is the command line I'm using to test:

ab -c15000 -n15000 http://www.mysite.com/somepath/

and

ab -c15000 -n15000 http://mysite.com/somepath/

I've tried several different types - it makes no different which order I do them in. This doesn't make sense to me - I can understand why the requests involving a redirect may not handle quite so many concurrent connections, but it's happening the other way round. Can anyone explain? I'd really prefer it if the canonical URL was the one which could handle more traffic.

I'll post my Nginx config below.

Thanks loads for any help!

server {
    server_name www.somesite.com  somesite.net www.somesite.net somesite.co.uk www.somesite.co.uk;
    rewrite ^(.*) http://somesite.com$1 permanent;
}

server {
    root   /home/django/domains/somesite.com/live/somesite/;

    server_name  somesite.com somesite-live.myserver.somesite.com;

    access_log  /home/django/domains/somesite.com/live/log/nginx.log;

    location / {
        uwsgi_pass unix:////tmp/somesite-live.sock;
        include    uwsgi_params;
    }

    location /media {
        try_files $uri $uri/ /index.html;
    }

    location /site_media {
        try_files $uri $uri/ /index.html;
    }

    location = /favicon.ico {
        empty_gif;
    }
}

score 4 · Accepted Answer · answered Dec 16 '10 at 00:26

The answer is pretty simple, actually. In your first test (the redirect) ab is only benchmarking a "301 Moved Permanently" request. The second test is testing your actual content.

To illustrate what I mean, curl your redirect URL. You'll get something like:

<html>
<head><title>301 Moved Permanently</title></head>
<body bgcolor="white">
<center><h1>301 Moved Permanently</h1></center>
<hr><center>nginx/0.8.52</center>
</body>
</html>

It's not hard to handle 15000 concurrent requests for just that! ;)

score 0 · Answer 2 · answered Dec 08 '10 at 11:02

Have you presented this to the dev mailing list? Very interesting for sure. Have you tried enabling full debugging; you might have had to enable it during compile. Once enabled do one request to the canonical and one to another. Perhaps the debug logs will shed some light. Have you tried removing the second server_name domain(somesite-live..)? Good luck, let us know if you figure it out!

Nginx / uWsgi / Django site can handle more traffic with rewrite URL

2 Answers2