281

I'm looking for an overly simplified answer to the following question. I'm trying to build a foundational understanding of how Nginx works alongside something like Gunicorn.

Do I need both Nginx and something like Gunicorn to deploy Django apps on Nginx?

If so, what actually handles the HTTP requests?

Ps. I don't want to use Apache and mod_wsgi!

techraf
  • 4,163
  • 8
  • 27
  • 44
a.m.
  • 2,915
  • 3
  • 13
  • 6
  • 1
    Apache and mod_wsgi is the most simple way of implementing the bridge between your django application and http requests that is also very capable in a production environment. For many developers, this means 'Apache is better than nginx' if they did but know it, but as 'betamax is better than VHS', alas, Dogma rules – MagicLAMP Mar 10 '17 at 08:25

4 Answers4

401

Overly simplified: You need something that executes Python but Python isn't the best at handling all types of requests.

[disclaimer: I'm a Gunicorn developer]

Less simplified: Regardless of what app server you use (Gunicorn, mod_wsgi, mod_uwsgi, cherrypy) any sort of non-trivial deployment will have something upstream that will handle the requests that your Django app should not be handling. Trivial examples of such requests are serving static assets (images/css/js).

This results in two first tiers of the classic "three tier architecture". Ie, the webserver (Nginx in your case) will handle many requests for images and static resources. Requests that need to be dynamically generated will then be passed on to the application server (Gunicorn in your example). (As an aside, the third of the three tiers is the database)

Historically speaking, each of these tiers would be hosted on separate machines (and there would most likely be multiple machines in the first two tiers, ie: 5 web servers dispatch requests to two app servers which in turn query a single database).

In the modern era we now have applications of all shapes and sizes. Not every weekend project or small business site actually needs the horsepower of multiple machines and will run quite happily on a single box. This has spawned new entries into the array of hosting solutions. Some solutions will marry the app server to the web server (Apache httpd + mod_wsgi, Nginx + mod_uwsgi, etc). And its not at all uncommon to host the database on the same machine as one of these web/app server combinations.

Now in the case of Gunicorn, we made a specific decision (copying from Ruby's Unicorn) to keep things separate from Nginx while relying on Nginx's proxying behavior. Specifically, if we can assume that Gunicorn will never read connections directly from the internet, then we don't have to worry about clients that are slow. This means that the processing model for Gunicorn is embarrassingly simple.

The separation also allows Gunicorn to be written in pure Python which minimizes the cost of development while not significantly impacting performance. It also allows users the ability to use other proxies (assuming they buffer correctly).

As to your second question about what actually handles the HTTP request, the simple answer is Gunicorn. The complete answer is both Nginx and Gunicorn handle the request. Basically, Nginx will receive the request and if it's a dynamic request (generally based on URL patterns) then it will give that request to Gunicorn, which will process it, and then return a response to Nginx which then forwards the response back to the original client.

So in closing, yes. You need both Nginx and Gunicorn (or something similar) for a proper Django deployment. If you're specifically looking to host Django with Nginx, then I would investigate Gunicorn, mod_uwsgi, and maybe CherryPy as candidates for the Django side of things.

Paul J. Davis
  • 4,126
  • 1
  • 15
  • 2
  • 18
    Thanks for taking the time to write such a detailed answer! Any recommended reading on the this "3 tier architecture"? – a.m. Nov 16 '11 at 01:23
  • 11
    Great answer, however I don't understand the issue with slow clients. – Mads Skjern Aug 22 '14 at 07:29
  • 5
    @MadsSkjern I'm guessing here, but if you assume all clients are fast, then you can use a fixed pool of worker processes, and not have to code for the case where many or all of them get blocked waiting for a client. – Jonathan Hartley Dec 11 '14 at 12:32
  • 3
    @a.m. https://en.wikipedia.org/wiki/Multitier_architecture – Jonathan Hartley Dec 11 '14 at 12:34
  • 2
    @Paul J. Davis, so when GUNICORN is hosted on another machine which is different with the one that nginx is hosted. How do the two communicate? How does nginx forward requests to GUNICORN? also through http? then GUNICORN must also have a web server to recieve requests? am I right ? If they are hosted on the same machine, then IPC(Inter Process Communication) is good enough? – Alex Jul 23 '15 at 08:22
  • 1
    @Alex NGINX and app server talk both ways [WSGI protocol](https://en.wikipedia.org/wiki/Web_Server_Gateway_Interface) -- either locally e.g. via pipe or via network via TCP sockets. – silpol Jul 12 '16 at 13:02
  • 12
    my django app only serves json no static content can i just go with gunicorn and no nginx – Sar009 Oct 29 '16 at 00:32
  • 3
    `You need something that executes Python but Python isn't the best at handling all types of requests.` - I agree with this sentence, since I had some experiences that proved that this is correct. But, does anybody has any document of anything else that can demonstrate an analysis, showing `Why Python isn't the best at handling all types of requests?`. Also, thank you very much #Paul, for such detailed and rich answer. – ivanleoncz Jan 18 '18 at 19:22
  • 1
    Then, what roles does the `celery` or `rabbitMQ` play in the architecture? Thanks – Stallman Jun 12 '18 at 17:09
  • 2
    @Stallman They do not play any role in serving a web request. celery (which uses rabbitMQ) is specifically for things not related to web requests like resizing image files or sending emails. – Dustin Wyatt Jan 21 '19 at 16:22
  • 3
    Great answer, but very ironically (given your role as a dev for Gunicorn) omits what Gunicorn _does_ do exactly, if NGINX handles the heavy lifting of request manager. This is leading to no one really being able to address @ShadowDoom's claim below that Gunicorn isn't necessary at all. – ijoseph Jun 01 '20 at 00:18
50

I liked this explanation in its simplicity:

Nginx will face the outside world. It will serve media files (images, CSS, etc) directly from the file system. However, it can't talk directly to Django applications; it needs something that will run the application, feed it requests from the web, and return responses.

That's Gunicorn's job. Gunicorn will create a Unix socket, and serve responses to nginx via the wsgi protocol - the socket passes data in both directions:

The outside world <-> Nginx <-> The socket <-> Gunicorn

https://gist.github.com/Atem18/4696071

Juuso Ohtonen
  • 601
  • 5
  • 6
  • 6
    It doesn’t have to be sockets, just in case others are wondering. – akshay Aug 03 '18 at 09:05
  • 2
    in the other word, a wsgi server is like a python app executor, which receives a dynamic content request and calls python app to generate the output. Is this correct? – Roy Ling Feb 14 '20 at 07:00
3

I'm looking for an overly simplified answer...

Do I need both Nginx and something like Gunicorn to deploy Django apps on Nginx?

If so, what actually handles the HTTP requests?

Overly simplified answer:

YES.

Both Nginx and Gunicorn.

Since you are deploying on Nginx, of course you need Nginx.

Since you are deploying Django, which is a web framework, you need something bridging the talk between the web server (Nginx) and the web framework (Django). In Python world, such a thing is called a WSGI server (but think it like a middle ware), examples of which include Gunicorn and uWSGI. When handling a request, Nginx proxies the request to Gunicorn or uWSGI, which in turn calls Django code and returns the response.

This document and Paul's answer will help you learn it better.

Cyker
  • 205
  • 2
  • 10
-1

Gunicorn is a waste of resources. You can simply proxy pass to django listening on a port instead of running gunicorn on top django and again nginx on top of all that. In benchmarks, I have seen very noticeable speed increase. Nginx can easily handle direct requests to django. Gunicorn is nothing else than a flyover(actually a slower flyover) above the normal road. It just sits and eats your resources and tries to claim to be powering your website.

nginx basically buffers all the requests and handles the static file requests by itself(if you have configured it like that). And proxies all the dynamic content to another server.(gunicorn/django).

Gunicorn has no other use than to just pass the request to application. It's like a straw you can drink from glass directly or drink from the straw at a limited pace(here the person drinking is django). And nginx is the waiter who brought you the glass of juice.

I have benchmarked and found this - with gunicorn: 22k req / s without gunicorn: 34k req / s

Your site will need the extra req/s in heavy load.

ShadowDoom
  • 23
  • 2
  • 11
    Are you talking about running the development server in production?! – Michael Hampton Jun 01 '19 at 05:11
  • 1
    The development server can be run behind a production server(like nginx). Because it will be getting requests in correct place and security and efficiency will be handled by the production server. It's just like using just WSGI + flask. Instead you can use just nginx + django(without any middleware hogging, like gunicorn). Please test the setup on heavy load and you will understand. – ShadowDoom Jun 01 '19 at 07:11
  • 11
    https://github.com/django/channels/issues/142 (TLDR: its a bad idea) – igor Jun 20 '19 at 09:25
  • 3
    There is a few issues. 1) Django says do not use dev server on production as it's not build for that purpose (security and all that stuff). 2) pure req per s can be solved by having more than one gunicorn. 3) Gunicorn has N number of workers that help sort out i/o delays, gunicorn restarts deadlocked/idle workers. All that stuff that django runserver do not do and btw Flask is the same, you wrap your flask app around some gunicorn/uwsgi, just Flask do not have this motion of runserver that behaves like a server but is not really. – Drachenfels Sep 04 '20 at 09:54