16

I am trying to put a hard limit in CPU usage for a dd command . I have created the following unit file

[Unit]
Description=Virtual Distributed Ethernet

[Service]
ExecStart=/usr/bin/ddcommand
CPUQuota=10%

[Install]
WantedBy=multi-user.target

which call the following simple script

#!/bin/sh
dd if=/dev/zero of=/dev/null bs=1024k

As I have seen in this guide, the CPU usage for my dd service should not exceed the 10%. But when I run the system-cgtop command the usage is about 70-75% .

Any ideas of what am I doing wrong and how can I fix it?

When I execute systemctl show dd I get the following results regarding CPU

CPUShares=18446744073709551615
StartupCPUShares=18446744073709551615
CPUQuotaPerSecUSec=100ms
LimitCPU=18446744073709551615
dawud
  • 14,918
  • 3
  • 41
  • 61
SteveGr2015
  • 263
  • 1
  • 2
  • 6
  • If you want to create BACKUP with `dd` please forget it,`dd` is low level disk dump, and if you want to recover your backup you will have some problem , please wriet the script for creating tarball and cp via ssh... – PersianGulf Apr 19 '15 at 21:12
  • You forgot to mention your Linux distribution and systemd version. – Michael Hampton Apr 19 '15 at 21:18
  • @MichaelHampton Hello I am using Fedora 20 64-bit – SteveGr2015 Apr 20 '15 at 06:58
  • 2
    @MohsenPahlevanzadeh I just want to test how I can limit the cpu-usage. So I use the dd command because it uses up to 100% of the CPU and I wan to limit it at 10% – SteveGr2015 Apr 20 '15 at 06:58
  • I am encountering the same problem on Fedora 23 and systemd version 222. The CPUQuota does not seem to have an effect. The service I want to throttle is run as a regular user. Maybe that's relevant... – gncs Jan 09 '16 at 15:36
  • @SteveGr2015, what version of systemd is this with? – Tollef Fog Heen May 25 '17 at 06:03
  • @TollefFogHeen it's been a long time since I faced this problem, I overcome it finally by using CPUshares option. Unfortunately I do not remember the systemd version which I used. – SteveGr2015 May 26 '17 at 07:27

2 Answers2

21

The Good

Your solution is the correct and should actually be quite future-proof; by using systemd to control the services cgroup settings, eg. CPUQota.

[Unit]
Description=Virtual Distributed Ethernet

[Service]
ExecStart=/usr/bin/ddcommand
CPUQuota=10%

[Install]
WantedBy=multi-user.target

See the man systemd.resource-control for more useful cgroup settings in systemd.

The Bad

There are two caveats to this though, which I (and possibly a few others) stumpled upon. Those caveats are really difficult to track down as there does not seem to be much easily findable information about this, which is the main reason for this answer.

Caveat 1:

The CPUQuota setting is only available since systemd 213, see https://github.com/systemd/systemd/blob/master/NEWS

    * The CFS CPU quota cgroup attribute is now exposed for
      services. The new CPUQuota= switch has been added for this
      which takes a percentage value. Setting this will have the
      result that a service may never get more CPU time than the
      specified percentage, even if the machine is otherwise idle.

This is for example an issue with Debian Jessie which only comes with systemd 208. As an alternative one could configure cpu.cfs_period_us and cpu.cfs_quota_us manually using cgcreate and cgset from the cgroup-bin package, eg.

sudo cgcreate -g cpu:/cpulimited
sudo cgset -r cpu.cfs_period_us=50000 cpulimited
sudo cgset -r cpu.cfs_quota_us=10000 cpulimited
sudo cgexec -g cpu:cpulimited /usr/bin/ddcommand

Caveat 2

For the settings cpu.cfs_period_us and cpu.cfs_quota_us to be available the Kernel needs to be compiled with config-flag CONFIG_CFS_BANDWIDTH. Sadly the 3.16.x Kernel for Debian Jessie is not compiled with this flag by default, see this feature request.

This will be available in Debian Stretch though. One could also use the kernel from jessie-backports, which should have the flag enabled.


I hope this answer helps a few people with the same issue as me...

PS: An easy way to test wether CPUquota is working in your environment is:

$ apt-get install stress
$ systemd-run -p CPUQuota=25% --slice=stress -- stress -c <your cpu count>

and watch with top or htop, the load should be spread (evenly) accross all cpus/cores, summing up to 25%.

Alternative

As an alternative tool one could use cpu-limit which should be available in most distros, eg.

$ apt-get install cpulimit
$ cpulimit -l 10 -P /usr/bin/ddcommand

It works by sending SIGSTOP and SIGCONT to the attached command to pause and resume its operation.

AFAIK it was difficult to control multiple separate/stand-alone processes simultaneously with this, in like group them together, but there might also be solution for this...

TomKeegasi
  • 326
  • 3
  • 5
  • Thanks a lot Tom, this was an old problem and I dealt with it by finally using CPUShares option, but your answer enlights many aspects of the problem and for that reason I will accept it. – SteveGr2015 Mar 12 '18 at 08:05
  • Thank you for the CPUQuota test with systemd-run. That was just what i was looking for! However I think it should read `systemd-run -p CPUQuota=25% --slice stress 'stress --cpu '` – tobalr Jun 05 '18 at 19:13
2

I had a similar task to limit CPU usage, but the use case was completely different. It was to stress a regression test suite on local machine to simulate what it undergoes on a loaded CI server (what causes non-deterministic or flaky behaviour of tests). This worked like a charm, in contrary to cpulimit and stress.

systemd-run -p CPUQuota="25%" --scope --uid=$USER --gid=$USER -- pytest ...

--scope made it inherit current shell environment and the command output to stdout as usual. The only difference in output was Running scope as unit run-uN.scope as the first line.

It requires root privileges to run and I have the GUI prompt from pkttyagent. It may be cumbersome to enter the password each time, if you need to run the command several times in a row, for instance to tweak CPUQuota, so prefixing the above command with sudo -E env PATH=$PATH would simplify the task.

Just in case, because there are reports it didn't work/wasn't supported on older versions.

$ uname -r
4.15.0-51-generic
$ systemd --version | head -n 1
systemd 229
saaj
  • 330
  • 3
  • 11