7

Multiple users are running java applications on a 60-core compute server (Linux/Ubuntu-based). There are different applications and most of them are not developed in-house.

While the sysadmin thinks it is okay for a given user's Java process to use 10 cores at any given moment, she would like them not to use more than 10.

Is there any Java or OS configuration which can be used to prevent the process from just grabbing compute resource in an unlimited way?

oggotron
  • 155
  • 2
  • 2
  • 3
  • You may change the CPU affinity. Check this post which details how to change a process affinity on Linux, Windows and Mac : https://stackoverflow.com/a/8882945/363573 – Stephan Nov 30 '18 at 10:34

2 Answers2

5

On the OS front:

  • I would say the the classical method is to set CPU affinity with taskset.

  • A better alternative is to use cgroups.

  • And the buzz-word solution is to run your applications in Docker containers. However, prior to Java 8, the JVM is not able to understand the limitations imposed by Docker (only available after 8u131+ backported, not sure if it is available as experimental in Java9 for the CPU part but it is for memory, and fully available in Java10)

aluncob
  • 3
  • 3
HBruijn
  • 72,524
  • 21
  • 127
  • 192
2

I would say, it is worth viewing this video. https://vimeo.com/181900266

It gives a very good overview about Java vs. cgroups vs. containers, and how the JVM finds out e.g. the number of CPUs.

Based on this I would say, that up to Java 8 the JVM determines the number of available CPUs on a way, that it always consider that all "online" CPUs are available for it, because it uses sysconf(_SC_NPROCESSORS_ONLN) for this purpose. Even if it is cgroup'd or put into a container.

From Java 9 JVM will use sched_getaffinity() and it will help if cpuset is used to control the number of CPUs for the JVM. I.e. the JVM will see only the configured CPUs. For example, if one uses Docker's cpuset-cpus option, then it will really create a limit for the JVM. Note! If cpushare is configured for the JVM, that still does not provide a solution for the problem. The JVM will still see all the CPUs in the system (no matter if it is in a container) if only cpushare is configured.

janosi
  • 21
  • 3