0

I have been trying to setup kubernetes for a week now as part of an evaluation, but I can only find

  • extremely complex manual solutions that are already out of date or outright broken
  • insanely simple solutions that break a mind numbing amount of best practices
  • "get to know kubernetes cluster all on one box"

My base requirements going in are:

  1. I have AWS
  2. I have VPCs, Subnets + peering already created
  3. I need to be able to build one cluster per environment in the appropriate VPC
  4. Ubuntu 16.04
  5. I should have control over the SGs and instances (terraform + ansible)
  6. Absolutely no single points of failure.
  7. I don't want a solution that creates unnecessary layers by adding tools or controllers above the masters. (WHY?!!)
  8. No blatant security holes

I've tried a couple solutions, but they have failed spectacularly, and require #7

kubeadm (breaks #6, #7)

  • I gave this a try because it was supposed to be a reasonable solution for using existing infrastructure.
  • I was able to get a cluster operational with 1 master and 2 nodes, but it just didn't seem to be working fully (kubernetes-dashboard was inaccessible)
  • In the end, even the tool itself didn't recommend production due to the fact that the docs state the master node cannot be made HA.

juju / conjure-up with "canonical kubernetes" (breaks #2, #3, #5, #6, #7, #8)

  • did not allow specification of VPC. Nevermind managing SGs, instances, working via bastion host.
  • It actually created instances in ec2-classic...?
  • instance naming in AWS is awful (can't tell what's what - masters vs etcd vs ??)
  • 15 security groups for a 3 worker cluster, 9 of which are completely empty?
  • conjure-down fails with 'NoneType' object has no attribute 'controller'
  • It created a single point of failure in the "controller machine"
  • INGRESS TCP 22 0.0.0.0/0 <- I am going to cry (this is why ec2-classic went away years ago)
Ryan
  • 65
  • 2
  • 7

2 Answers2

1

Try out kops - it has support for Terraform output. It should be possible to meet your other requirements with it (although some, like no single point of failure, may require changing the default config).

ConnorJC
  • 921
  • 1
  • 7
  • 19
0

You can check typhoon. Personally I'm using kube-aws which is more flexible, but unfortunately no terraform.

I also created a quick start script based on kube-aws.

Camil
  • 101
  • 2