3

I have a Kubernetes cluster on AWS that installed using kops (public topology). One of my deployments is a web server that I'd like to expose via CloudFront, and the other is an API server that I'd like to expose via ALB.

In both cases, that would mean certificates are taken care of for me. For the API server in particular, it's important for me to have client IP addresses, so I need HTTP load balancing, not just TCP. I also use WebSockets, so I need ALB, not ELB.

Creating an "external service" gives me an ELB TCP load balancer. For the web server, I could just point CloudFront to that (is the elastic IP address is gives it stable?) for TLS and caching. For the API server, I could point ALB to it, but that seems a bit redundant.

According to the k8s documentation, I should use an nginx ingress controller behind ELB, but then I have to provide certificates to nginx (is the solution integrated with Let's Encrypt at all?), and I'd lose the client IP address. Unfortunately, the examples are yet to be written.

What's currently the right solution for what I want to do? I presume it's quite a common case.

Isvara
  • 215
  • 1
  • 12
  • I've found [kube-lego](https://github.com/jetstack/kube-lego), at least, if I need to go the nginx way. – Isvara Mar 10 '17 at 23:12

1 Answers1

2

You don't need ELBs and ALBs.

Dedicate some of your cluster nodes to be loadbalancer nodes. Put them in a different node group, give them some label: mynodelabel/ingress: nginx, and than you host an nginx ingress daemonset on that node group.

Most important options are:

spec:
  restartPolicy: Always
  dnsPolicy: ClusterFirst
  hostNetwork: true
  nodeSelector:
    mynodelabel/ingress: nginx

and

  ports:
    - containerPort: 80
      hostPort: 80
    - containerPort: 443
      hostPort: 443

This way you get raw user traffic!

Nginx ingress controller works perfectly with kube-lego Let's Encrypt!

Assign elasticIPs to your k8s loadbalancer nodes and just point route53 DNS entry at them.

Optionally you can taint your loadbalancer nodes so that regular pods don't work on them and slow down the nginx.

cohadar
  • 764
  • 7
  • 6