0

I'm running a 40 thread / 125GB RAM server.

The server is based on CentOS 7.

I have noticed that the server response time is higher when the server is idle:

enter image description here

You can see on the screenshot here that the server response between 4pm and 10PM was lower than during other times.

I looked at the logs and GoogleBot was hitting us at 4 requests / second at that moment which loaded up the server. Most of the requests from GoogleBot were 302 redirects (large ecommerce catalog with daily changes in live products).

enter image description here

Here you can see the throughput of the server - during the times when the server was busy, that's when the response times were low.

How can I debug this?

What's causing this?

Could 302 redirects be cheaper than the 200 responses which skewed the data?

Could cache (Redis / Opcache / APCu) be evacuating too early which causes recreation of cache during idle time?

We're currently running: Apache 2.4 Nginx Proxy MySQL Redis Opcache APCu Elasticsearch

UPDATE:

Looking at the separated processes PHP is taking up the most time:

enter image description here

MySQL somewhat is correlating to PHP, but not fully:

enter image description here

Kalvin Klien
  • 145
  • 7
  • Does this answer your question? [Can you help me with my capacity planning?](https://serverfault.com/questions/384686/can-you-help-me-with-my-capacity-planning) – djdomi Apr 06 '22 at 17:46
  • @djdomi this is not really a question about capacity - its a question about why when the server is idle, the response time is higher than when the server is loaded. Could there be any specific settings I should use to reduce the response time when the server is idle? I talked to my server guy and he said it could be because 302 requests are cheaper than 200, so it can be misinterpreting data because of the cheap 302 requests. – Kalvin Klien Apr 06 '22 at 18:13
  • 1
    The cause is probably something like this: In a busy state, most important parts are already cached in RAM from previous request (directories, static files, php skripts, probably even PHP bytecode, and even SQL queries), whereas in idle times every request may have to load almost everything from disk. In case your whole *server* is a VM, it may even be the case that this takes a little bit of time to obtain resources as well – Hagen von Eitzen Apr 06 '22 at 18:59
  • @HagenvonEitzen hmmmm, but you can see from the graphs, its not like there's a huge amount of time that passes between the weird states of low vs high response time. I'd think the graphs would be a lot smoother in between the transitions. The Server is outfitted with NVME drives and is not really inhibited by those(3500MB/s). Most of the data is in RAM already and we're only using about 70GB out of 125GB. – Kalvin Klien Apr 06 '22 at 19:59
  • FYI this is a baremetal system with NVME drives, 125 GB RAM, 2xCPUs / 40 threads. – Kalvin Klien Apr 06 '22 at 20:07

0 Answers0