55

Nginx worker_connections "sets the maximum number of simultaneous connections that can be opened by a worker process. This number includes all connections (e.g. connections with proxied servers, among others), not only connections with clients. Another consideration is that the actual number of simultaneous connections cannot exceed the current limit on the maximum number of open files". I have few queries around this:

  1. What should be the optimal or recommended value for this?
  2. What are the downsides of using a high number of worker connections?
cnst
  • 12,948
  • 7
  • 51
  • 75
Aarti
  • 629
  • 1
  • 7
  • 6

5 Answers5

65

Let's take the pragmatic approach.

All these limits are things that were hardcoded and designed in the past century when hardware was slow and expensive. We're in 2016 now, an average wall-mart toaster can process more requests than the default values.

The default settings are actually dangerous. Having hundreds of users on a website is nothing impressive.

worker_process

A related setting, let's explain it while we're on the topic.

nginx as load balancer:

  • 1 worker for HTTP load balancing.
  • 1 worker per core for HTTPS load balancing.

nginx as webservers:

This one is tricky.

Some applications/frameworks/middleware (e.g. php-fpm) are run outside of nginx. In that case, 1 nginx worker is enough because it's usually the external application that is doing the heavy processing and eating the resources.

Also, some applications/frameworks/middleware can only process one request at a time and it is backfiring to overload them.

Generally speaking, 1 worker is always a safe bet.

Otherwise, you may put one worker per core if you know what you're doing. I'd consider that route to be an optimization and advise proper benchmarking and testing.

worker_connections

The total amount of connections is worker_process * worker_connections. Half in load balancer mode.

Now we're reaching the toaster part. There are many seriously underrated system limits:

  • ulimits is 1k max open files per process on linux (1k soft, 4k hard on some distro)
  • systemd limits is about the same as ulimits.
  • nginx default is 512 connections per worker.
  • There might be more: SELinux, sysctl, supervisord (each distro+version is slightly different)

1k worker_connections

The safe default is to put 1k everywhere.

It's high enough to be more than most internal and unknown sites will ever encounter. It's low enough to not hit any other system limits.

10k worker_connections

It's very common to have thousands of clients, especially for a public website. I stopped counting the amount of websites I've seen went down because of the low defaults.

The minimum acceptable for production is 10k. Related system limits must be increased to allow it.

There is no such thing as a too-high limit (a limit simply has no effect if there are no users). However a too-low limit is a very real thing that results in rejected users and a dead site.

More than 10k

10k is nice and easy.

We could set an arbitrary 1000kk limits (it's only a limit after all) but that doesn't make much practical sense, we never get that traffic and couldn't take it anyway.

Let's stick to 10k as a reasonable setting. The services which are going for (and can really do) more will require special tuning and benchmarking.

Special Scenario: Advanced Usage

Sometimes, we know that the server doesn't have much resources and we expect spikes that we can't do much about. We'd rather refuse users than try. In that case, put a reasonable connection limit and configure nice error messages and handling.

Sometimes, the backend servers are working good and well but only up to some load, anything more and everything goes south quickly. We'd rather slow down than have the servers crash. In that case, configure queuing with strict limits, let nginx buffer all the heat while requests are being drained at a capped pace.

user5994461
  • 2,749
  • 1
  • 17
  • 30
  • 1
    I like the answer but to make a truly educated guess on how high one should set that, it appears we'd have to know roughly how much RAM one connection takes (e.g. to save a normal static file with `sendfile`) so that we can multiply to compute how much RAM would be needed to sustain a given number of `worker_connections`. – nh2 Jun 27 '18 at 14:13
  • 2
    You can do up to 10k without too much tuning. The connection buffers are set in the ``sysctl`` settings. – user5994461 Jul 08 '18 at 10:07
  • 1
    I had a case where a Nodejs app was calling istelf through `localhost` quite intensively by design and this used a good part of the `worker_connections`. So even though the `worker_connections` limit was 1024, the EC2 `t3.small` instance handled ca. 20 req/s, used only 40% RAM and 20% CPU, but that led to about 0.5% of 400 and 502 responses. This is to say that it is not just about "how much RAM / process", it also depends on the app design. – Manuel Apr 21 '20 at 00:22
24

ulimit -a will tell you how many open files your system allows a process to use.

Also, net.ipv4.ip_local_port_range sets the total range of sockets to enable per IP.

So, your worker_connections cannot be more than any of those, or your client connections will queue until net.core.netdev_max_backlog - the total size of the TCP queue.

Keep in mind that if you're using nginx as reverse-proxy, that uses two sockets per connection. You might want to play a little bit with net.ipv4.tcp_fin_timeout and other kernel tcp related timeouts to try to switch state of sockets quickly. Another thing to take note is that each socket allocate memory of the TCP memory stack, you can also set some limits of the TCP memory stack using sysctl, you can put more pressure in the RAM as long as you have CPU and enough file handlers.

FYI it's possible given enough computing resources, to have one server with around 32GB ram and some virtual network interfaces to handle 1MM simultaneous connections with some kernel tuning using sysctl. During my tests when dealing with more than 1MM and sending a payload of around 700Bytes the server was taking around 10 seconds to update about 1.2MM simultaneous clients. Next was to increase the network bandwidth by bonding some extra NICs and ditching virtual nics. It's possible to achieve pseudo near real-time communication with more than 1.2MM clients, given the payload, bandwidth and reasonable time to update all clients.

Happy tuning!

Marcel
  • 1,575
  • 8
  • 14
  • please fix the command to ulimit not ulimits – Ali.MD Oct 06 '16 at 14:40
  • 1
    Note `net.ipv4.ip_local_port_range` is only relevant for _outgoing_ connections, not not for incoming connections (as is typical with nginx on e.g. ports 80 and 443); see [here](https://stackoverflow.com/a/11129641/263061). – nh2 Jun 27 '18 at 14:19
  • @nh2 but if one's using nginx as reverse proxy, there are at least 2 sockets open per client connection, and then it matters how many local ports the kernel can allow sockets to bind to. – Marcel Jun 27 '18 at 14:56
  • Yes that's correct. – nh2 Jun 27 '18 at 20:12
1

The appropriate sizing can be discovered through testing, as it is variable based on the type of traffic Nginx is handling.

Theoretically, nginx can handle: max clients = worker_processes * worker_connections (* =multiply) and worker_processes = number of processors

To find out processors, use: grep processor /proc/cpuinfo | wc -l

Sachin Gargya
  • 37
  • 3
  • Actually with reverse proxy: max_clients = (worker_processes * worker_connections ) / (X * 2) where X is however many concurrent connections these clients make to you. Also, connection structures are used for listen sockets, internal control sockets between nginx processes, and for upstream connections. So this max clients = worker_processes * worker_connections won't work as we don't know many connections are being used in internal control sockets. – Aarti Jul 05 '16 at 07:34
1

Marcel's answer really needs to be upvoted! If ulimits are set to a default value of around 1k, max_connections should be set around the same value otherwise there is no benefit to setting max_connections to 10k.

You will get queued request and sockets closed on nginx if "your worker_connections cannot be more than any of those, or your client connections will queue until net.core.netdev_max_backlog - the total size of the TCP queue."

A single process can open as may connection as the ulimits allow. num_workers * max_connections is the formula but outside loadbalancer/proxy max connections and ulimits need to be taken into account for a reasonable values. Setting max_connection to a really high value may backfire as ulimits will be a limiting factor.

Cryptophobia
  • 11
  • 1
  • 2
    That's factually wrong. The soft limit on desktop linux is 1k but it doesn't prevent processes from using more than that if they request to, up to the hard limit (32k or more). nginx will increase the ulimit automatically if ``max_connections`` is higher than the default soft limit. – user5994461 Feb 19 '19 at 21:58
0

Setting lower limits may be useful when you may be resource-constrained. Some connections, for example, keep-alive connections (not only with the clients, but with the upstream servers, too), are effectively wasting your resources (even if nginx is very efficient, which it is), and aren't required for the correct operation of a general-purpose server, meaning that they can be safely dropped to make more resources available for the rest of the operation.

Having a lower resource limit would thus indicate to nginx that you may be low on physical resources, and those available should be allocated to new connections, rather than to serve the idling keep-alive connections at the expense of newer connections having trouble being established to serve the more pressing needs.

What is the recommended value? It's the default.

The defaults are all documented within the documentation:

Default: worker_connections 512;

And can be confirmed at the source-code level at event/ngx_event.c, too

13#define DEFAULT_CONNECTIONS 512

Community
  • 1
cnst
  • 12,948
  • 7
  • 51
  • 75