3

I have a multi cores Kubernetes cluster, inside it I have multiple pods configured with a CPU limit of 500 millicores:

        resources:
          limits:
            cpu: "500m"

It is possible to have, in a single pod, more than one threads running in parallel (simultaneously, at the same exact moment)?

From my understanding when the limit is less than 1000 millicores the pod can never have multiple threads running in parallel. Is it correct?

Davide Icardi
  • 157
  • 2
  • 10

2 Answers2

4

You certainly can run 2 threads in a container at once using 25% of the cputime each. However, as for whether these will be run exactly together, I'm not 100% sure.

Running the following tests do seem to indicate it can:

docker run -it  --cpus=".5" --cpuset-cpus="0,1" polinux/stress stress --cpu 2

%Cpu0  : 29.6 us,  0.0 sy,  0.0 ni, 70.4 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu1  : 29.6 us,  0.0 sy,  0.0 ni, 70.4 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
Marco
  • 287
  • 1
  • 2
  • 12
3

Yes, CPU limits in Kubernetes are implemented using the Linux CPU quota subsystem (at least on Linux, not sure on Windows). That system works by giving the cgroup a total count of timeslices it is allowed to run, and refilling that every few milliseconds. If a task (thread, process) is marked as runnable and the group it is in has available quota tokens then it will run just as it always would (and the bucket is decremented). If there are no tokens then it won't run and a timeslice_exceeded event is emitted.

Setting a limit of 500m means the token refill rate will average out to 0.5 seconds of runtime allowed for every 1 second of wall clock time. But if you have a million cores then you could use all those tokens in one jiffy if your tasks were all runnable.

coderanger
  • 836
  • 4
  • 13
  • Also I should add: you probably shouldn't use CPU limits without a very good reason, they are usually bad. – coderanger Jun 05 '21 at 20:04
  • I disagree that cpulimits are bad. There are a lot of use cases where a cpu limits fit in. For example: you don’t want a particular service overloading the node when longer execution time wouldn’t hurt. This may trigger autoscaling which will soon after the trigger value is reached balance the load of the service over multiple nodes. However, cpulimits below one core are inefficiënt. An interesting, but not completely related read can be found [here](https://engineering.indeedblog.com/blog/2019/12/unthrottled-fixing-cpu-limits-in-the-cloud/) – Marco Jun 05 '21 at 22:45
  • The lingering bugs from slow rollout of new kernels is definitely a factor, but fortunately less and less as time goes on. The bigger issue is that most people don't know that CPU requests (which you should always set along with memory requests) kiiiind of do the same thing but better. CPU reqs end up in the CPU shares value which is doing similar time slice analysis but without the explicit bucketing. That means that in a CPU exhaustion situation, you will still get roughly the requested amount of CPU even with no limits. – coderanger Jun 06 '21 at 00:33
  • And not using limits means you _can_ burst when there are timeslices that aren't used. There are use cases for CPU limits, like services you know will busyloop or otherwise burn cycles for no benefit, but those are pretty rare. – coderanger Jun 06 '21 at 00:33