6

What nginx or haproxy setup is suggested for target 100K concurrent websocket connections? What I think, a single nginx will not be able to take such traffic as well as concurrent connections. How should traffic to nginx/haproxy be split (DNS lvl or any Amazon/Google option available)? How much concurrent websockets a single nginx be able to handle?

Tried gathering relevant information from google search and SO posts.

Ravi Kumar
  • 195
  • 1
  • 1
  • 4
  • Currently I am restructuring my app architecture to use Websocket totally, no ajax. We have around 12-13k peak users, but they tend to open multi-browser-tabs. Right now target is to support 50K, but just to be safe and know what kind of websocket load-balancing people use, I have elevated target to 100K. – Ravi Kumar Sep 28 '15 at 09:25
  • @AD7six that duplicate is waste of time. I asked about websocket. Please read the question. – Ravi Kumar Sep 29 '15 at 06:09
  • @DukeLion thanks, I am learning from people like you. Did you read my question, I am not asking what benchmarks are available, rather how the connection/traffic pooling should be structured for high throughput. – Ravi Kumar Sep 29 '15 at 06:12
  • Great, @RaviKumar, please consider editing your question clarifying that. – DukeLion Sep 29 '15 at 06:53
  • 1
    Read this - https://mrotaru.wordpress.com/ – Tan Hong Tat Sep 29 '15 at 08:04
  • @DukeLion , ok, I will revise it – Ravi Kumar Sep 29 '15 at 08:20
  • @DukeLion Why such the condescending attitude? – NiCk Newman Jun 11 '16 at 08:28
  • Moderators who has marked it duplicate has no idea of what I asked. Its about websocket, not DB or other things, and those have different capacity planning. – Ravi Kumar Oct 26 '16 at 10:34
  • Lol, it's clearly not a duplicate of that question, cmon. – Skeeve Aug 29 '18 at 09:44

1 Answers1

9

There are people running chat servers behind haproxy load balancers at even higher loads. The highest load that was reported to me in private e-mail (with the copy of the stats page) was at around 300k connections per process (hence 600k sockets). Note that under Linux by default a process is limited to 1M file descriptors (hence 500k end-to-end connections), but that can be tweaked in /proc.

The most important thing to consider at such loads is the amount of RAM you need. The kernel-side socket buffers will always require at least 4kB per direction per side, hence 16kB minimum per end-to-end connection. HAProxy 1.5 and lower will have two buffers per connection (eg: 4kB buffers are enough for websocket). 1.6 can run without those buffers and only keep them allocated for the rare connections with data. So at least that's 16 GB of RAM per million of connection, or around 24 GB with older versions. It may be worth spreading this over multiple processes on SMP machines to reduce the latency. Keep in mind that in order to simply establish 1M connections, it can take 10 seconds at 100k conns/s. All these connections induce some work for a few bytes each, and dealing with 1M active connections will definitely induce an important work and a high load on the system.

Willy Tarreau
  • 3,894
  • 1
  • 19
  • 12
  • wow, thanks Willy. These type of information I am seeking for, so I can get some direction for researching over more. – Ravi Kumar Sep 29 '15 at 08:15
  • I have questions, why there are two sockets per connection, is it IN + OUT buffered socket (will use file descriptor). Is 1M limit default to all Linux distro. I am guessing its same as ulimit, and that would be 1024 on my ubuntu. If thats the case, I can raise it (to what max limit).? – Ravi Kumar Sep 29 '15 at 08:18
  • 1
    you have two sockets because the first one is between the client and haproxy and the second one between haproxy and the server. The million is the kernel's default and I'm not aware of any distro changing it by default, but it's just a sysctl, it's unlimited. The 1024 you're seeing in ulimit is for non-root users, root is allowed to raise the limit up to this sysctl :-) – Willy Tarreau Oct 21 '15 at 20:53