Kubernetes - vSphere Cloud Provider

Question

I'm following this doc https://cloud-provider-vsphere.sigs.k8s.io/tutorials/kubernetes-on-vsphere-with-kubeadm.html

I am using a load balancer as my ControlPlaneEndpoint, now I would like to join a new master to the cluster passing the cloud-provider flag as well, through the below method it was possible join the workers however I can't do the same with a new Master.

kubectl -n kube-public get configmap cluster-info -o jsonpath='{.data.kubeconfig}' > discovery.yaml

# tee /etc/kubernetes/kubeadminitworker.yaml >/dev/null <<EOF
apiVersion: kubeadm.k8s.io/v1beta1
caCertPath: /etc/kubernetes/pki/ca.crt
discovery:
  file:
    kubeConfigPath: /etc/kubernetes/discovery.yaml
  timeout: 5m0s
  tlsBootstrapToken: y7yaev.9dvwxx6ny4ef8vlq
kind: JoinConfiguration
nodeRegistration:
  criSocket: /var/run/dockershim.sock
  kubeletExtraArgs:
    cloud-provider: external
EOF

The first Control Plane was created the following way:

kubeadm init --config kubeadminit.yaml

apiVersion: kubeadm.k8s.io/v1beta2
bootstrapTokens:
- groups:
  - system:bootstrappers:kubeadm:default-node-token
  token: y7yaev.9dvwxx6ny4ef8vlq
  ttl: 0s
  usages:
  - signing
  - authentication
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 10.20.121.22
  bindPort: 6443
nodeRegistration:
  criSocket: /run/containerd/containerd.sock
  kubeletExtraArgs:
    cloud-provider: external
  name: cjblvk8smst1
  taints:
  - effect: NoSchedule
    key: node-role.kubernetes.io/master
---
apiServer:
  timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controlPlaneEndpoint: kubeproxy:6443
controllerManager: {}
dns:
  type: CoreDNS
etcd:
  local:
    dataDir: /var/lib/etcd
imageRepository: k8s.gcr.io
kind: ClusterConfiguration
kubernetesVersion: v1.20.5
networking:
  dnsDomain: cluster.local
  podSubnet: 10.244.0.0/16
  serviceSubnet: 10.96.0.0/12
scheduler: {}

I tried to join the second master the following way however it is jointed as a worker node:

kubeadm join --config kubeadminitSecondmaster.yaml

apiVersion: kubeadm.k8s.io/v1beta2
caCertPath: /etc/kubernetes/pki/ca.crt
discovery:
  file:
    kubeConfigPath: /etc/kubernetes/discovery.yaml
  timeout: 5m0s
  tlsBootstrapToken: y7yaev.9dvwxx6ny4ef8vlq
kind: JoinConfiguration
nodeRegistration:
  criSocket: /run/containerd/containerd.sock
  kubeletExtraArgs:
    cloud-provider: external
  name: kubemst2
  taints:
 - effect: NoSchedule
    key: node-role.kubernetes.io/master
---
apiServer:
  timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controlPlaneEndpoint: kubesproxy:6443
controllerManager: {}
dns:
  type: CoreDNS
etcd:
  local:
    dataDir: /var/lib/etcd
imageRepository: k8s.gcr.io
kind: ClusterConfiguration
kubernetesVersion: v1.20.5
networking:
  dnsDomain: cluster.local
  podSubnet: 10.244.0.0/16
  serviceSubnet: 10.96.0.0/12
scheduler: {}

Cluster information:

Kubernetes version: 1.20.5
Cloud being used: bare-metal - vSphere
Installation method: Kubeadm
Host OS: Centos 7.9
CNI and version: Weave 0.3.0
CRI and version: Containerd 1.4.4

Thanks

What exactly have you tried? What commands? What error did you encounter? — acid_fuji, Apr 13 '21 at 07:36
I edited the post with your answer, I got no error, but the node is jointed as a worker. — Alexandre Cardoso, Apr 13 '21 at 11:36

score 2 · Answer 1 · answered Apr 13 '21 at 11:55

2

Your kubeadm join command is missing the --control-plane parameter.

kubeadm join --control-plane --config kubeadminitSecondmaster.yaml

Without the parameter the node becomes a worker.

answered Apr 13 '21 at 11:55

Gerald Schneider

19,757
8
52
79

I got this: can not mix '--config' with arguments [control-plane] To see the stack trace of this error execute with --v=5 or higher, actually a think I had tried that too, thanks. – Alexandre Cardoso Apr 13 '21 at 13:43
Then use the parameters with the token your init command showed you. Your instructions say to use the config file to set up worker nodes, not control nodes. – Gerald Schneider Apr 13 '21 at 14:55
Can I pass the cloud-provider flag that way, how? Because the instruction says to pass it to the workers, so I imagined even more for a Master. – Alexandre Cardoso Apr 13 '21 at 15:24

score 0 · Accepted Answer · answered Jul 07 '21 at 13:34

One of my issues was resolved updating the cluster this way:

kubeadm upgrade apply --config kubeadm-config.yaml --ignore-preflight-errors all --upload-certs --force --v=5

File:

apiServer:
  extraArgs:
    cloud-config: /etc/kubernetes/vsphere.conf
    cloud-provider: vsphere
    authorization-mode: Node,RBAC
  extraVolumes:
  - hostPath: /etc/kubernetes/vsphere.conf
    mountPath: /etc/kubernetes/vsphere.conf
    name: cloud
  timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controlPlaneEndpoint: k8s-proxy:6443
controllerManager:
  extraArgs:
    cloud-config: /etc/kubernetes/vsphere.conf
    cloud-provider: vsphere
  extraVolumes:
  - hostPath: /etc/kubernetes/vsphere.conf
    mountPath: /etc/kubernetes/vsphere.conf
    name: cloud
dns:
  type: CoreDNS
etcd:
  local:
    dataDir: /var/lib/etcd
imageRepository: k8s.gcr.io
kind: ClusterConfiguration
kubernetesVersion: v1.20.6
networking:
  dnsDomain: cluster.local
  serviceSubnet: 10.96.0.0/12
scheduler: {}

Kubernetes - vSphere Cloud Provider

2 Answers2