1

I am learning k8s and I have 3 nodes k8s cluster. I have just recently deployed k8s with kubeadmin and so far it is working great. everything working perfectly but the only problem that I am facing is network throughput. my three nodes and replica sets are like this

Pentagon

2 Pods on each node can communicate to each other just fine with 9GB throughput in iperf test. However, when pods on different nodes communicate with each other they only send/receive 3MB or less traffic. I am not able to troubleshoot the issue.

As described in the diagram Pod1a can communicate to Pod1b just fine and with a throughput of around 10GB but when Pod1a communicate with Pod2a or Pod3a only 3MB is the available bandwidth.

I am using Calico network layer in k8s. any guide to help me troubleshoot the issue will be highly appreciated.

Thanks,

Thank you,

sysadmin1138
  • 131,083
  • 18
  • 173
  • 296
user3852599
  • 33
  • 1
  • 5

1 Answers1

0

tl;dr

The issue occured on 10Gbps cards, by using 1Gbps cards the issue disappeared

Details

I am encountering the same issues inside a 2 VM setup serving as Kubernetes nodes (based on microk8s, using a default calico setup) with a 10Gbps card.

While the communication of the VM host to host and inside each node is close to maximum the card supports, I too was getting ~3MBit speeds for node <-> node communication.

After replacing the 10Gbps card with a 1Gbps card, the node <-> node speed was close to maximum supported.

If you managed to solve this issue utilizing a 10Gbps card, I'd be keen to learn how you did it.

Jackob T. Daw
  • 1