3

I am migrating from Xen to Kvm.

In Xen I was able to easily pin host cpus to guest vms, and also to pin host cpus to the "dom0" .

In Kvm I can also easily pin host cpus to guests vms, but, as far as I can I see, nothing prevents application running on the host OS to use these cpus. I want to prevent the case where a program running on the host starves / increases the latency of guests.

I could manually do an elaborate cgroup policy, but maybe am I just missing a setting in libvirt / centos7 ?

Also there is also an "emulatorpin" setting for guests. Should I pin the "emulator" to dedicated host cpus, or should I just restrict it to the guest cpus ? The goal is to limit as much as possible the latency on the guest.

Olivier S
  • 2,719
  • 1
  • 12
  • 14

3 Answers3

5

If I understand your question correctly what you want to achieve is to restrict the hypervisor so it can only use a single CPU/core (or a limited number) for it's own processes, interrupt handling and everything. And that all other cores can be assigned by libvirt to guest systems.

Relatively simple is the isolcpus boot parameter which allows you to isolate one or more CPUs from the scheduler. This prevents the scheduler from scheduling any user-space threads on this CPU.

i.e. on your hypervisor in /etc/default/grub set:

GRUB_CMDLINE_LINUX="... quiet isolcpus=0,1"

that ought to prevent any userspace programs on the hypervisor from using cores > 1. Libvirt can then pin virtual servers to the remaining free cores.

I'm not sure if the isolcpus boot parameter also ensures that all interrupts will be restriced to those cores. Otherwise interrupts also have their own affinity property, smp_affinity, that defines the processors that will handle the interrupt request. The interrupt affinity value for a particular interrupt request is stored in the associated /proc/irq/irq_number/smp_affinity file and the default is set with /proc/irq/default_smp_affinity. smp_affinity is stored as a hexadecimal bit mask representing all processors in the system. The default value is f, meaning that an interrupt request can be handled on any processor in the system. Setting this value to 1 means that only processor 0 can handle the interrupt.

The tool to control processor and scheduling affinity for RHEL and CentOS 7 is called tuna

HBruijn
  • 72,524
  • 21
  • 127
  • 192
  • Thank you for these informations. Sorry the bounty expired. – Olivier S Oct 03 '17 at 16:41
  • For interruptions: if you are using the ( RedHat ) real time kernel you can activate FOLLOW_ISOLCPUS=yes in /etc/sysconfig/irqbalance, else you can setup IRQBALANCE_BANNED_CPUS on non RT kernels. Also nohz_full and rcu_nocbs will help remove unwanted interruptions from the selected cpus – Olivier S Oct 06 '17 at 10:32
1

In Linux, if you want a process use only an specific cpu in your host, taskset command can help

running a new programm on two cpus:

taskset -c 0,2 /home/app/myprogramm

To change the cpu affinity of already running proccess:

taskset -p -c 0,2 <pid_of_your_proccess>

there is no dom0 in kvm, you have kvm kernel module, so everything is integrated in the kernel, is not like in xen that you have dom0 as privileged domain, so you can pin the process that kernel run.

enter image description here

Cristian Ciupitu
  • 6,226
  • 2
  • 41
  • 55
c4f4t0r
  • 5,149
  • 3
  • 28
  • 41
  • you do not answer the question. Libvirt already does a taskset via cpu pinning configuration. – Olivier S Sep 12 '17 at 13:58
  • How can I ( easily) DEDICATE a cpu to a guest? What about the emulatorpin ? taskset does not dedicate a cpu. It restricts cpu usage to a process. – Olivier S Sep 12 '17 at 14:32
  • 1: kvm guest has pid like a normal process, so you can do cpu affinity with taskset too on a guest, 2: you wrote "host cpus to the "dom0" and I wrote "there is no dom0 in kvm", so if you have an application on host use taskset, anyway, I don't use any application on host, because host is for kvm guest vm – c4f4t0r Sep 12 '17 at 14:34
  • You clearly do not understand the question. – Olivier S Sep 12 '17 at 14:56
  • you don't understand the answer, you don't have dom0 in kvm, but I think the clear question that you need to ask is, how can exclude a cpu from the linux scheduler, in this way, the cpu is not used for process scheduling " kernel parameter isolcpus" – c4f4t0r Sep 12 '17 at 15:30
1

isolcpus is now deprecated:

          [Deprecated - use cpusets instead]
          Format: [flag-list,]<cpu-list>```

use cpuset with libvirt

Cristian Ciupitu
  • 6,226
  • 2
  • 41
  • 55
Stuart Cardall
  • 531
  • 4
  • 7