84

If there is a limit on the number of ports one machine can have and a socket can only bind to an unused port number, how do servers experiencing extremely high amounts (more than the max port number) of requests handle this? Is it just done by making the system distributed, i.e., many servers on many machines?

alh
  • 979
  • 1
  • 7
  • 5

3 Answers3

120

You misunderstand port numbers: a server listens only on one port and can have large numbers of open sockets from clients connecting to that one port.

On the TCP level the tuple (source ip, source port, destination ip, destination port) must be unique for each simultaneous connection. That means a single client cannot open more than 65535 simultaneous connections to a single server. But a server can (theoretically) serve 65535 simultaneous connections per client.

So in practice the server is only limited by how much CPU power, memory etc. it has to serve requests, not by the number of TCP connections to the server.

Bergi
  • 105
  • 5
Dennis Kaarsemaker
  • 18,793
  • 2
  • 43
  • 69
  • I wonder how carrier grade NAT will affect this – TheLQ Aug 26 '13 at 17:34
  • @TheLQ Without CGN each client cannot open more than 65535 connections to the same server, no client should need nearly that many connections to the same server. With CGN each CGN cannot open more than 65535 connections to the same server, those would have to be shared among all of the clients using that CGN. Whether the CGN can simultaneously open 65535 connections to one server and 65535 connections to another server is an implementation detail, which can vary between different CGN implementations. – kasperd May 02 '14 at 08:02
  • The limit can be increased by adding more IP addresses to the CGN or by deploying more CGNs. But you can also just deploy dual stack. Then connections to servers with IPv6 support won't go through the CGN, so they won't be consuming precious port numbers. – kasperd May 02 '14 at 08:04
  • @Dennis Kaarsemaker, can you also please have a look at this question? https://serverfault.com/questions/1005157/maximum-number-of-simultaneous-server-sent-events-sse-connections-to-a-web-ser – thewebjackal Mar 01 '20 at 04:46
  • 1
    LImit is 65535 connections per ip address. You can easily add more virtual network interfaces/ virtual ips to the same physical network interface and have 1mln+ connections or more on same server. – Dannyboy Sep 29 '21 at 08:54
  • @DennisKaarsemaker, So the server can serve virtually endless clients using the same port. Can a client talk to virtually endless servers (different) using the same port also? – YoavKlein Oct 12 '21 at 20:03
23

You are mistaken - the socket's uniqueness is determined by five factors:

  1. the local IP address
  2. the local port number
  3. the remote IP address
  4. the remote port number
  5. the transfer protocol (TCP/UDP)

When offering network services, 1. and 2. typically are static (e.g. IP 10.0.0.1, port 80) but unless you are expecting thousands of connections from a single client (or a single NAT gateway), you are not going to push the boundaries for the possible combinations of 3. and 4. before you run out of local resources.

So although practically a client will not use a port already in use for a connection to open a connection to a different destination IP address, port number depletion is going to be the least of your problems for nearly any application - be it on the server or client side.

The problem is a very real one with NAT gateways (routers) serving clients with a high number of open outbound connections (e.g. torrents) - there you will see port number depletion after the port pool available for NAT has been emptied. In this case the NAT gateway is unable to create any additional associations, thus effectively cutting clients off the internet.

Emanuel Adrian Lucaci
  • 3
  • 2
the-wabbit
  • 40,319
  • 13
  • 105
  • 169
  • can you also please have a look at this question? https://serverfault.com/questions/1005157/maximum-number-of-simultaneous-server-sent-events-sse-connections-to-a-web-ser – thewebjackal Mar 01 '20 at 04:48
6

The question was how to handle large (>64k) connection counts. The two most common methods are:

  • Adding more servers, which increases the number of src/dst addresses and port number tuples. There are multiple ways to share load across multiple servers; DNS round robin is one; there are others

  • Deploy "carrier-grade NAT" (which a friend derisively and correctly in my view refers to as "crummier-grade NAT"). This is essentially a NAT of a NAT. This has very bad implications for applications, but it's what some large providers do when they run out of IPv4 space and/or port numbers, and/or they don't want to move to IPv6.

user8162
  • 270
  • 2
  • 9
  • 2
    If you read more than the title, you'll see that this question is about port exhaustion and the OP was mistaken about how it works. How exactly does this answer add anything new of value to that? – MDMarra Sep 26 '13 at 18:51
  • 2
    The OP asks explicitly (and not just in the headline) "how do servers experiencing extremely high amounts ... of requests". Leaving aside the confusion over how sockets work, that's a valid question. – user8162 Sep 26 '13 at 18:57
  • 1
    Excellent job of cutting off the question mid-sentence. The rest of that sentence says: `(more than the max port number)`. – MDMarra Sep 26 '13 at 20:30
  • 2
    Perhaps you missed the part where I stipulated the OP doesn't understand socket mechanics. I'm of the opinion that how to exceed 64k ports is a valid topic for discussion; I'm sorry if you disagree, but that WAS the question and that WAS what I provided a couple of answers for. You asked how this was responsive; that's how. – user8162 Sep 26 '13 at 20:39
  • 1
    I think we're going to have to agree to disagree here. Especially considering the OP gave the accepted answer to Dennis who sets him straight about how tuples work :) I'm not saying that your information is incorrect, just that it doesn't answer the intent of the OP's question. – MDMarra Sep 26 '13 at 20:48
  • 3
    That's half the question. The bigger-picture question -- as supported by the OP's final sentence -- is how to break the 64k barrier. – user8162 Sep 26 '13 at 20:56
  • 1
    @user8162 I agree with you. It's ridiculous to downvote an answer that attempts to explain the other half of the question. The fundamental misunderstanding happened on multiple levels, all of which deserve an explanation. – Anthony Nov 10 '16 at 17:33