0

In the monitoring tab for Composer (Airflow) on Google Cloud there is a graph showing "CPU usage per node". How should the values in this graph be interpreted? What value would indicate that the load is reaching the limit and that there might be insufficient CPU resources available?

The Composer Kubernetes cluster is running on 3 nodes of machine type n1-standard-1.

There is a short description available in the graph but I'm not sure how to interpret it:

"The usage of CPU cores aggregated from all running pods on the Kubernetes node. Measured in core time usage ratio. Doesn't include the usage by system components running on the node."

monitoring-graph-screenshot

mwfearnley
  • 757
  • 9
  • 21
Niemi
  • 101

1 Answers1

0

From GCP documentation

CPU usage per node A chart showing the usage of CPU cores aggregated over all running Pods in the node, measured as a core time usage ratio. This does not include CPU usage of the App Engine instance used for the Airflow UI or Cloud SQL instance. High CPU usage is often the root cause of Worker Pod evictions. If you see very high usage, consider scaling out your Composer environment or changing the schedule of your DAG runs.

Vit
  • 445
  • 2
  • 10
  • The documentation mentions "very high usage" but I would need help understanding what value is a high "core time usage ratio"? Is 2.3 as shown in the attached graph in the question considered high or low? – Niemi Oct 06 '20 at 16:01
  • i'm trying to find more info for you – Vit Oct 06 '20 at 16:07
  • unfortunately wasn't able to find more on this – Vit Oct 14 '20 at 23:12