0

I have a Kubernetes cluster setup using KubeAdm (Setup in EC2 instances in AWS Cloud). The CNI used is Flannel and the cluster is running quite well. My requirement is that I am trying to setup KubeVirt (https://kubevirt.io/) in my Cluster to provision VMs along with pods in my cluster. KubeVirt is an open-source sandbox project from CNCF. I have setup KubeVirt and it's running fine. I followed the steps in this link to setup KubeVirt: https://kubevirt.io/quickstart_cloud/

After that, I wanted to spin up a VM in my Kubernetes cluster. So, I followed the official documentation from: https://kubevirt.io/labs/kubernetes/lab1.html

The VM got created, but was in a 'scheduling' phase. I observed that along with the VM, an associated pod also got created in the cluster.The issue is that this particular pod is not getting scheduled and is in pending state.

enter image description here

After describing the pod, I got the following error messages which says something about a device plugin:

enter image description here

Warning FailedScheduling 2m5s (x4 over 22m) default-scheduler 0/2 nodes are available: 2 Insufficient devices.kubevirt.io/kvm. preemption: 0/2 nodes are available: 2 No preemption victims found for incoming pod.

There are no taints in my worker node, however this pod won't get scheduled. Then I found a link which was mentioning about the Device plugin of KubeVirt (https://kubevirt.io/2018/KVM-Using-Device-Plugins.html).

Looking at the Worker node description, under 'allocatable->devices.kubevirt.io/kvm' it's mentioned as "110" and when I edited my worker node, it was "0"

enter image description here

I tried to edit the worker node and change the value to "110". I saved the change and it displayed node/kube-c2-workernode-02-poc edited.

However, after I edit it one more time, the previously added change was totally gone as if I didn't modify it at all. I tried to increase the worker node specs by increasing the CPU cores, memory and IOPS in the EBS volume. But effortlessly, the result is still same.

So, after going through the merry-go-round and coming down to the issue, how do I make changes to resource allocation of a worker node in Kubernetes and properly save it?

arjunbnair
  • 25
  • 1
  • 2
  • 8
  • Please don't post textual content as screenshots; it impairs searching, is hard to read, and is explicitly mentioned in the [how to ask](https://serverfault.com/help/how-to-ask) page – mdaniel Aug 05 '22 at 20:01

1 Answers1

1

Kubevirt requires virtualization to be enabled.

Those "allocatable" / "capacity" is not something that you can/should not change yourself. They just reflect the state of your node: you can not host virtual machines here, thus your virt-launcher Pod is stuck Pending.

Currently, the only way to have this in EC2 is to use metal instances.

On bare-metal, in general: we should make sure virtualization instructions are enabled at the BIOS level (VT-x / AMD-v). At which point, your node would register that virtual machines can be scheduled.

SYN
  • 1,751
  • 8
  • 14
  • So in short, this cant be performed in ordinary EC2 instances or EKS/AKS? Is that what we have arrived at? – arjunbnair Aug 05 '22 at 19:34
  • 1
    Hey it worked! I changed the instance type of both master and worker node to 'm5.metal' and it automatically worked. "Kubevirt requires nested virtualization to be enabled." - Think this is what was missing in the setup. Thanks a lot for the tip. – arjunbnair Aug 05 '22 at 19:49
  • 1
    and actually: "nested virtualization instructions" is what you would need to enable on the hypervisor, running a Kubernetes node that needs to run its own virtual machines. From the Kubernetes node perspective: you just need virtualization instructions, ... my mistake / edited my answer – SYN Aug 06 '22 at 16:57