0

We are building a Kubernetes Cluster on bare metal. As the cluster is getting physically installed, I was asked to weigh in on the following question:

Each node has two network connections of 25Gbps. We will be using Ceph as a Filesystem for the Kubernetes cluster. Each node has 3 SSDs, of which 2 will be used for Ceph. Should we use a dedicated network connection for Ceph (so 25Gbps dedicated to Ceph and 25Gbps for "everything else") or should we use an aggregated connection of 50Gbps?

bountrisv
  • 1
  • You still have the ceph public traffic within the k8s network, so there's no real need to separate them except if it's a requirement from the network department etc. I haven't seen 25Gbps saturated with openstack and k8s, but it might depend on the size of your cluster. But reading about only 2 SSDs per ceph node doesn't sound like a lot, so you probably should be fine either way. I would probably configure LACP with all 25G NICs. – eblock Sep 23 '22 at 10:22

0 Answers0