1

I would like to extend my in-house on-premises stacked HA Kubernetes with 3 control planes, 3 worker nodes cluster across 3 office location. So each control plane and 1 worker would be in each of the offices. Each control plane would have its own static IP.

I am having trouble understanding how the load balancer (i am using metallb) could do its job if let's say office 1 had a power outage. How is the traffic re-directed to office 2 & 3?

e.g. External, public, users accessing nextcloud running on office-1-node may be using an fqdn cloud.company.com that is associated with office-1-node Static IP. (although, nextcloud maybe running on a worker or master in office-2-node, or office-3-node...but as far as users are concerned, it is not a concern.)

If office-1-node has a power outage, what would direct cloud.company.com to the static IP of office-2-node or office-3-node?

I am still learning, so please bare that in mind when answering.

imharindersingh
  • 15
  • 6
Fawzi Masri
  • 21
  • 1
  • 3

1 Answers1

1

I could see that Metal LB is deployed(https://metallb.universe.tf/installation/) as a controller(which handles IP assignment) and daemonset (component - speaker -> This is I think handling external announcements as per https://metallb.universe.tf/concepts/). So this LB is basically deployed as a Kubernetes resource.

Coming to your question now if office-1-node has an outage, the pods in daemonset from other nodes will function as required(do external announcement and make the services reachable) and controller from your other master nodes will continue to handle IP assignments.

Resources:

  1. Kubernetes Controller https://kubernetes.io/docs/concepts/architecture/controller/
  2. Kubernetes Daemonset https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/
  3. Kubernetes Metallb manifest - https://github.com/metallb/metallb/blob/main/manifests/metallb.yaml

Edit: Answering question in second comment. First you'll need to buy a Domain Name(from godaddy, bigrock, AWS Route53, etc). Second create a CNAME to point that domain name to your Load balancer's IP(with port optionally)(in your case, IPs of office-1,2,3 can be used in round-robin manner to ensure uptime). This round robin manner is called DNS load balancing. Or you can implement DNS failover like in AWS Route53 by implementing health checks in your load balancer. Once done, the external user will continue to have access to services in kubernetes(from office-2 and 3)

imharindersingh
  • 15
  • 6
  • yes metallb is a resource and will continue to do its job on the 2 avilable nodes after a power outagae on the first node. – Fawzi Masri Aug 26 '20 at 15:07
  • yes metallb is a resource and will continue to do its job on the 2 available nodes after a power outage on the first node... but how does an "external" user benefit from that. Somehow the DSN record for cloud.company.com should be changed to point to office-2-node. no? – Fawzi Masri Aug 26 '20 at 15:13
  • of all the links provided the second is most relevant. metallb uses these to handle announcing IP: ARP, NDP, or BGP. the first 2 are more of LANonly. the BGP sounds like it will be the one protocol to use. however this statement is puzzling "establish BGP peering sessions with nearby routers that you control". what does "you control" mean? – Fawzi Masri Aug 27 '20 at 05:25
  • @FawziMasri I've updated my answer to your question in second comment. – imharindersingh Aug 28 '20 at 17:48