1

Is there a way to get multiple TCP connections from the same IP/MAC to use all of the physical links in a LACP configuration?

Here's my setup:

The central switch (Meraki MS42P) has a 10GbE link to one server and a 10GbE link to the access switch (Cisco 2960-CX) on the other side of the building. The second server is using 4 x 1GbE in LACP configuration on the access switch.

If I open one connection each from multiple 1GbE clients anywhere in the network, I can saturate the 4 x 1GbE aggregated link (i.e. up to 4 clients get full 1Gb at the same time). However, if I open 4 simultaneous TCP connections from the first server (with a single 10GbE link to the switch) to the second server, they always share a single 1GbE link.

Stefan Wild
  • 13
  • 2

2 Answers2

2

This is dependend on the load-balancing algorithm which decides which connections should always be on the same wire.

I'm not familiar with Meraki in this regard but for Cisco the default would be source-dest-mac which is connections from the same source MAC address and destination MAC address would go over the same wire.

You'd like something like source-dest-port where every connection from a different port or to a different port would each go over another wire of the LAGG group.

pacey
  • 3,833
  • 1
  • 15
  • 31
  • Great pointer. The configuration on the Cisco (which has the LACP link to the second server) defaulted to `src-mac`. Unfortunately, the only available configurations are `dst`, `src-dst` or `src` on either `ip` or `mac` according to the [documentation](http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst2960x/software/15-0_2_EX/layer2/configuration_guide/b_lay2_152ex_2960-x_cg/b_lay2_152ex_2960-x_cg_chapter_010.html#task_1276204) – Stefan Wild Sep 28 '16 at 00:35
2

Replace LACP to Roundrobin bonding mode (mode=0). Change Cisco port-channel mode to on. Before all remove all interfaces from this port-channel group.

Why i advice you to use RR (round robin)? LACP is very often applied between switches when distance between them is long or connections may be damaged. Linux can use level3+level4 load balancing algorithm for LACP, but there aren't switches or routers witch can do it. Level4 means that source and destination IP ports will be used to calculate transmit hash. If you will use LACP between two servers directly then you will give 1Gbit/s per one IP connection. With RR you will get full speed connection 4Gbit/s. But if one of four links will damage you will get 25℅ packet loss. How often it may be when server and switch are in one room? It wasn't in my 15 years practice.

Mikhail Khirgiy
  • 2,003
  • 9
  • 7