11

Given the fact there are Quad ports PCI/PCI express cards out there, I wonder how many of them (x4) can I plug into a single server?

Perhaps the capacity bottleneck is the PCI bus, but I am not familiar with how to calculate that.

Skyhawk
  • 14,149
  • 3
  • 52
  • 95
Tzury Bar Yochay
  • 717
  • 11
  • 24
  • Is this asking if there's a theoretical limit, or do you really need a large number yourself? How many? If you're asking about the bus do you mean communicating at with the CPU /OS at full capacity, etc.? – Rup Jul 12 '10 at 10:17
  • I do need to run a router software and have at least 12 ports. I wonder if I add three of these Quad NICs would be working or am I to meet the wall – Tzury Bar Yochay Jul 12 '10 at 10:19

4 Answers4

16

From a purely physical standpoint, there's no limit to the number of interfaces you can have. You can get chassis with 12+ PCI slots, and extenders can bump it up to over 100.

From a bandwidth standpoint, a 64-bit PCI bus has a bandwidth of 528 MBps, which translates into 42 100 Mbps links or four 1 Gbps links blasting non-stop. Most will not be doing that however, so many more cards can be used.

From an OS standpoint, an OS that uses an unsigned byte to identify NICs will be limited to 255 or 256 cards. Linux uses a full name instead, which means that there is no theoretical limit.

In short, 12 ports is nothing.

Ignacio Vazquez-Abrams
  • 45,019
  • 5
  • 78
  • 84
  • 2
    There is a maximum number of PCI slots a buss can accommodate (I don't recall the number), so systems with large numbers of slots must have multiple buses. That would mean the total potential bandwidth would be even higher. – John Gardeniers Jul 12 '10 at 12:03
  • @John: Or it could use a bridge to connect the multiple buses together, which would not increase the total bandwidth. – Ignacio Vazquez-Abrams Jul 12 '10 at 12:07
  • I expect that you'll run out of processing power just handling the network protocol long before you run out of bus bandwidth. – symcbean Jul 12 '10 at 12:15
  • 2
    Just go buy a switch! – symcbean Jul 12 '10 at 12:15
  • good point, although I would expect any system with that many sockets would be designed for better performance that a bridge would provide. – John Gardeniers Jul 12 '10 at 13:07
2

A friend of mine said me the max he could plug was 4 card of 4 interfaces on PCI (don't remember if it's pci-e) slots.

aif
  • 371
  • 1
  • 8
2

Actual ports and the ability to serve those ports are two very different things.

If you simply wanted to have a silly number of ports then you could use multiple multi-port USB cards and hubs and have literally hundreds or thousands of USB-based Ethernet adapters - each with their own 10/100 port and a MAC but their performance would be appalling (although I doubt their drivers support such numbers).

In terms of 'proper' NICs there's no reason why you couldn't have ten or more multi-port GigE cards, or even use HP's Virtual Connect Flex-10 cards which will happily provide up to 24 x 1Gbps links over a few 10Gbps trunks.

What matters though is matching the functional need to the available bandwidth. Personally I prefer using fewer 10Gbps NICs and then use VLAN-tagged virtual NICs from within the OS, but that's not for everyone.

Chopper3
  • 100,240
  • 9
  • 106
  • 238
  • Getting 2 USB Ethernet adapters to work is a pain. They've gotta be on seperate hardware buses for the drivers to work effectively. Getting more than 2 to work is probably a fool's errand. – Tom O'Connor Apr 02 '12 at 10:47
1

When using multi-port NICs or multiple NICs, the actual performance scaling is far from being lineal. First of all the actual performance and the performance scaling will highly depend on the NIC type and vendor. Secondly, the CPU and memory bus overhead is increasing not linearly when using multiple NICs, especially if you will try to reach the maximum possible performance.

If you are looking for performance, check 10GB Ethernet solutions, which have better CPU offloading capabilities at high speeds.

ocsid80
  • 159
  • 1
  • 1