I need help with some deeper networking stuff. Hopefully somebody can help me or at least push me to the right direction.
At the moment I have a two node bare metal (VPS) kubernetes cluster setup with microk8s. Testingwith website deployments work great, routing based on ingress :-)
But now I want do deploy other services that are accessable by ports. I don't want to use NodePort, as the used ports will always be above 30000. Instead I want to use native ports, maybe 21 for a FTP server or something like that.
Deeper reading brought me to the metalLB extension (LoadBalancer for bare metal K8S). But unfortunatly, metalLb wants an IP range so I cannot use the different pubic IPs. My both machines have very different public IPs on eth0, accessable from the internet.
As a solution, I installed a second NIC on both machines (eth1) with a local network (192.168.1.10-192.168.1.11). Now I have a range of IPs I can provide to metalLB configuration. On both hosts, I can ping the other machine on it's local address, so LAN is working.
But the Deployment (FTP for example) opened the port on the LAN address, maybe 192.168.1.11:21 That seems corrects as I provides these LAN-IPs to metalLB.
Now I want to access this service through the public IP. On each host, the public IPs (let's call them 5.5.5.5 and 6.6.6.6 to make clear they are non-local) are the default gateways, so there is NO default gateway for the LAN IPs (192.168.1.x)
I guess I need some DNAT / SNAT to route traffic, arriving at the public IP of a host the the second NIC (eth1) where a service is listening.
Finally I played a little bit with IPTABLES and tried to nat the traffic between the interfaces, but without luck.
I use latest ubuntu on both hosts, Port 21 is opened on the firewall on both hosts... But finally I want to use multiple services with different ports. TCP and UDP aswell.
Is this solution possible at all? Thank you for any ideas!
