0

In order to balance users CPU consumption on a shared computing server, I would think it very useful if one could:

  1. set the nice level for all processes of a user
  2. furthermore, adapt this level in realtime
  3. furthermore, adapt this level not only to future, but also to running jobs
  4. furthermore, derive this level from the user's current overall CPU usage

The idea is that a user can call as many jobs as he likes. But if there are other users, his priority will drop depending on how much he is asking for. So A can use all 32 cores at a time. Then another user B comes and starts only 1 job. Now A should get a lower priority than B. Then C comes and starts 8 jobs. He should now have a priority between A and B. However the priority should actually not be based on the number of processes but on their overall CPU demand - if that could be determined at all.

I guess this might be the same as attributing the same share of CPU to all active users as long as they make use of it and distribute the rest to all that want more.

Do you think this would be possible in any way?

Do you think it makes sense at all?

Radio Controlled
  • 113
  • 4

1 Answers1

1

Every task for themselves

An overly simplistic approach would be to not quota them at all. On this 32 CPU box, A + B + C is 41 jobs. Plus some hopefully small system overhead for the kernel. However, not everything is CPU bound, and there are bottlenecks in I/O, memory, and others. So the load average isn't actually 41, response time and throughput might be tolerable. Of course, there are no controls to prevent further load causing performance problems.

Fair Share

Perhaps resources should be partitioned proportionally to a ratio. This is what for example Solaris's fair share scheduler does. Say each of A B and C are projects in some zones out of a 32 core pool, and each has 1000 shares. (See Greg's notes for practical zone examples.) B and C would run at 100% because they used less than their third. But A runs at 23 CPUs, because 1 + 8 are necessary to give B and C their share. Unequal shares are possible, if you the administrator come up with some scheme to assign them.

Linux cgroups came along later with something similar. CPU cgroups can be used to implement proportional shares, or specific quotas. This is how systemd slices and Kubernetes CPU resources are distributed.

What to run next

With many tasks waiting to run, ideally the scheduler doesn't leave anything waiting too long to go on CPU. At the micro level, how run queues or equivalent work also are relevant for how it a system performs.

Linux CFS scheduler divides time slices up for every waiting task, which is fair in that respect. Further, those tasks who used less time are up first. Which has the useful effect of prioritizing interactive things, idling waiting for user input.

Priority, including nice value, is an input for the scheduler decision. And it does not rely on quotas artificially limiting use, which is good. However, adjusting priority, even when a scheme exists for doing so, has limits. One system wide priority number doesn't help when each user has tasks of various priority, but still needs to keep a fair share of resources.

John Mahowald
  • 30,009
  • 1
  • 17
  • 32