I'm setting up a Kubernetes cluster, where the Nodes are distributed across different networks. Additionally, all Nodes are part of a WireGuard network. Each Node therefore has 2 IP addresses, one bound to its primary NIC (private or public IPv4) and a WireGuard one.
Usually, I would have used Weave Net as CNI, but as there is already an (encrypted) WireGuard connection between the nodes, the actual question was if using a simple CNI like Flannel VXLAN or WireGuard might be a better choice, as it would reduce the overall complexity.
Do you have any experience on relying on an existing VPN as transport for a CNI, or do you see good reasons to not follow that approach and use a dedicated CNI connection like Weave Net instead, even though it would induce additional load on all Nodes?