Posting this answer as community wiki as the question portrayed could be wide and not get the definitive answer.
Feel free to expand it.
As for:
Does something like this work ?
Yes. In fact there are multiple ways to configure your cluster. I encourage you to visit sites below for an overview of Kubernetes components and architecture:
You can use below link (with a guide) to setup a cluster that will use the minimum of 3 masters and 3 workers nodes for highly available setup:
Some tips!
- For production environments, you would need a dedicated HAProxy load balancer node (physical, virtual)
- While master components can run on any machine, best practice dictates using a separate server for the master and not running any user containers on this machine.
- It is best practice to run your clusters in a multi-master fashion in Production – to ensure high availability and resiliency of the master components themselves. This means you’ll need at least 3 Master nodes (an odd number, to ensure quorum
As for quorum in etcd!
Please remember that etcd requires a quorum to be operational. You can read more about it by following below link:
Do all VM instances have to be in the same network ?
It will depend on the networking architecture of your setup. The easiest solution would be to create a private network that would span across multiple physical machines that you intend to deploy Kubernetes (to have nodes in the same network).
Paraphrasing what was just said:
VM #1 on Physical server #1 could easily communicate with VM #2 on Physical server #2
Would that work with VPN ?
Please specify what exactly you mean by a VPN connection:
- Is it the connection between nodes in your network?
- Is it the connection that will connect the nodes between themselves from an offsite location?
- Is it a connection for developers/sysadmins to get access to your Kubernetes cluster?
As for a side note you can look on Wireguard (network plugin with built-in encryption):
Does this need another master node for each physical server ?
In short no but. There is always a but:
- You can create a cluster where there will be single master node and multiple workers on multiple physical machines. This setup will work but it won't be highly available solution.
- You can create a cluster where there will be multiple master nodes span across multiple physical servers. This heavily depends on what exact needs you have.
I thought about this question and it could be a long shot but if you think that master and worker on each physical machine are coupled in any term, they aren't. Each worker should be able to communicate with each master node and vice versa.
Do they have to be in the same private network as well ? Or do you just create a master node on each server and join only the master nodes together ?
You need to run $ kubeadm init to initialize the cluster on a single master node. Then you will need to run $ kubeadm join on each of the nodes (master/worker). The difference between them (master/worker) is the parameter:
--control-plane # <-- this parameter will indicate a master node (use it only on a master node).
I also encourage you to check additional resources:
