33

I use Google Kubernetes Engine to run on demand pods. Each pod is publicly exposed to the internet using a nodeport service.

I am looking for a way in GKE to get a single IP or an IP range for outbound connections, to give them to third party API's to whitelist them.

The GKE node IPs are not manageable when nodes autoscale or when I upgrade them. I need a way to maintain a consistent outbound IP.

I have tried using a simple NAT Gateway to the Kubernetes nodes (Using the example from here), and while this routes the outbound connections to the NAT Gateway, it breaks inbound traffic to the pods(Nodeport service) since they get dropped off at the NAT gateway.

  • Is there a default IP range for a google cloud region which I can provide to the third party to whitelist (or)

  • If GKE provides a way to select node external IPs from a pre reserved list of static IP's (or)

  • Is there any other way to accomplish having a single static IP or an IP range that represents the outbound traffic from the pods

I have found similar questions like this, but they don't address my problem as the pods should be externally connectable, which breaks when using a NAT.

Parag
  • 650
  • 1
  • 7
  • 12

2 Answers2

17

Google Cloud now provides a managed NAT Gateway service - Cloud NAT.

This gateway can be used with a GKE cluster, which provides a stable public egress IP to all the pods inside it, which enables them to be whitelisted by third party service providers.

The example implementation for using Cloud NAT with GKE is provided here - https://cloud.google.com/nat/docs/gke-example

Parag
  • 650
  • 1
  • 7
  • 12
  • 1
    Does the cluster have to be private? – Gajus Oct 22 '19 at 04:13
  • 3
    Yes, the cluster has to be private, but the master can be public. If any of your services also need to be public, they can be exposed by a loadbalancer. – Parag Oct 22 '19 at 13:21
  • @Parag Is it mandatory that the cluster has to be private? Is there a way, I can use if the the Node IPs are public. – Suhas Chikkanna Jan 02 '20 at 17:31
  • 1
    @SuhasChikkanna With CloudNAT no. If you want the nodes to have external IPs but also want a single outbound IP, you can create your own NAT gateway instance using Squid, and have conditional routing, which will route your internal servers to the instances directly, but any traffic from outside comes in through the NAT. This will give your users a single outbound IP, while making your nodes accessible to selected IP sources. – Parag Apr 09 '20 at 07:25
  • @Parag Sounds good! Would definitely try that. Thanks a lot :) – Suhas Chikkanna Apr 09 '20 at 10:47
  • Is there a way to add this to an existing cluster, or do I need to spin up a new one and migrate everything over? – robbrit Jun 03 '20 at 23:08
3

You can try this solution :

https://cloud.google.com/solutions/using-a-nat-gateway-with-kubernetes-engine

Under normal circumstances, Google Kubernetes Engine nodes route all egress traffic through the internet gateway associated with their node cluster. The internet gateway connection, in turn, is defined by the Compute Engine network associated with the node cluster. Each node in the cluster has an ephemeral external IP address. When nodes are created and destroyed during autoscaling, new node IP addresses are allocated automatically.

The default gateway behavior works well under normal circumstances. However, you might want to modify how ephemeral external IP addresses are allocated in order to:

  • Provide a third-party service with a consistent external IP address.
  • Monitor and filter egress traffic out of the Google Kubernetes Engine cluster.
Community
  • 1
alphayax
  • 131
  • 4