Performance tuning is as much a black art as it is a science. Complete books have been written about it, include quite a bit as part of the vendor documentation
You start with good monitoring, add load, get a baseline and only then you start adjusting and observing if your adjustment did or did not have the intended result.
Rinse and repeat until you get the optimal result.
You don't mention your major RHEL or CentOS release but they all include automatic tuning as well, with for instance the tuned
daemon. Predefined performance profiles can be selected from tuned-adm list
and then activated with tune-adm profile <profile-name>
.
For your use-case the RHEL 7 profile latency-performance
sounds like the one you might benefit from:
latency-performance
A server profile focused on lowering latency. This profile is recommended for latency-sensitive workloads that benefit from c-state tuning and the increased TLB efficiency of transparent huge pages.
This profile favors performance over power savings by setting intel_pstate
and max_perf_pct=100
. It enables transparent huge pages, uses cpupower
to set the performance cpufreq governor, and requests a cpu_dma_latency
value of 1
.
For more general Linux tuning Brandan Gregg has made a number of visual diagrams of which tool to use on what part of your system that make really nice cheat sheets:
Tools:
![Performance Observability](../../images/3831827612.webp)
Which options to use in sar
to monitor:
![<code>sar</code> options](../../images/3836349587.webp)
And closely related:
![Linux Benchmarking Tools](../../images/3819310220.webp)
Translate observations into tuning with:
![tuning tools](../../images/3817213029.webp)