1

I have set Kubernetes up with kubeadm, bare-metal on Ubuntu Linux instances. These are instances on GCE, but I am not using the specific GKE native cluster service. Each instance (as a node) has a static, external IPv4 address. I am trying to make a service available to the outside using an ingress. I've tried MetalLB or port-forwarding, but can't get it to work. The former seems to require addresses to be commissioned on demand, which I don't have.

Is there a recommended way to get LoadBalancing to work with static, external IPs?

chris_l
  • 11
  • 1
  • Could you provide your configuration manifests and more details about env? What OS did you use, Ubuntu/Debian? How did you configure Ingress, services. How did you test it. All steps to replicate your environment would be good. – PjoterS Nov 23 '20 at 11:30
  • Hi, thanks for the offer to help. As I have moved on to a different approach it probably makes little sense to discuss all the details. I am now using Cloudflare's Argo tunnel to connect web content directly with the CDN. – chris_l Nov 24 '20 at 12:54
  • Could you provide your solution as an answer? It might help others users who encounter similar issue. – PjoterS Nov 25 '20 at 11:12
  • Well, I don't have a solution to the original problem. I sidestepped the issue by tunneling between server pods and CDN. See here: https://developers.cloudflare.com/argo-tunnel/create-tunnel/kubernetes-sidecar-model – chris_l Nov 25 '20 at 15:46

1 Answers1

0

I think root cause of this issue is the Firewall configuration. You can connect to cluster in a few ways. Below I will provide you examples of NodePort methods.

Assuming that there is a Kubernetes cluster is configured correctly (with CNI, Docker, Kubeadm bootstrap, taints, etc).

You can use below example to spawn hello-app pods that will respond to your traffic:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment-1
spec:
  replicas: 1
  selector:
    matchLabels:
      key: application-1
  template:
    metadata:
      labels:
        key: application-1
    spec:
      containers:
      - name: hello1
        image: gcr.io/google-samples/hello-app:1.0
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 8080

Using NodePort service type.

You are using GCE instance so your VM already have static ExternalIP.

Hello-app was deployed in previous step, now you will need to expose it with service of type NodePort

apiVersion: v1
kind: Service
metadata:
  name: service-one
spec:
  type: NodePort
  selector:
    key: application-1
  ports:
    - port: 80
      targetPort: 8080

By default, firewall will block the incoming requests on ports that are not whitelisted. You will need to create a rule that will allow traffic to enter on specific ports (ie. 31085). For details, please check Configuring Firewall Rules.

You can do it by following below path: Navigation Menu > VPC Network > Firewall > Create Firewall Rule.

You can create a rule that allows:

  • single port for NodePort for example 31085
  • whole range 30000-32767
$ kubectl get svc
NAME          TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
kubernetes    ClusterIP   10.96.0.1        <none>        443/TCP        50m
service-one   NodePort    10.110.184.238   <none>        80:31085/TCP   5m53s

To reach your cluster you can use

$ curl ExternalNodeIP:NodePort

$ curl 35.246.148.237:31085
Hello, world!
Version: 1.0.0
Hostname: deployment-1-77ddb77d56-v5826

If you wouldn't create FirewallRule for this NodePort, curl output will be:

curl: (7) Failed to connect to 35.246.148.237 port 31085: Connection refused

Using Ngin Ingress Controller as baremetal considerations

When using Nginx Ingress you can use:

  • Over a NodePort Service
  • host network

You can depoy Nginx BareMetal using:

$ kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.41.2/deploy/static/provider/baremetal/deploy.yaml

It will create Nginx service as NodePort type. It's very similar to previous method, but difference is that your services can be ClusterIP and NodePort type.

You can apply Ingress to expose more than 1 service using this method.

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: test-ingress
  annotations:
    kubernetes.io/ingress.class: "nginx"      
spec:
  rules:
  - http:
      paths:
      - path: /one
        backend:
          serviceName: service-one
          servicePort: 80
      - path: /two
        backend:
          serviceName: service-two
          servicePort: 80
$ kubectl get svc -A
NAMESPACE       NAME                                 TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
default         service-one                          ClusterIP   10.106.21.3      <none>        80/TCP                       2m4s
default         service-two                          ClusterIP   10.105.161.86    <none>        80/TCP                       2m2s
ingress-nginx   ingress-nginx-controller             NodePort    10.110.149.194   <none>        80:31337/TCP,443:30417/TCP   6m31s
ingress-nginx   ingress-nginx-controller-admission   ClusterIP   10.98.15.73      <none>        443/TCP                      6m31s

### Output:
$ curl 35.246.148.237:31337/one
Hello, world!
Version: 1.0.0
Hostname: deployment-1-77ddb77d56-v5826
curl 35.246.148.237:31337/two
Hello, world!
Version: 2.0.0
Hostname: deployment-2-fb984955c-8pfls 
PjoterS
  • 615
  • 3
  • 11