1
Operating in a very separated IT-as-infrastructure/Engineering-as-Software environment.
I have a server that, after random periods of uptime, suffers from severe CPU speed degradation.
The server is a largely unmodified Apache webserver running on RedHat, using AMD Opteron(tm) Processor 6176 SE.
cat /proc/cpuinfo
over various periods will report roughly the spec'ed hardware speeds (2300MHz) when the server first restarts. Over time, this reported speed will degrade to ~600MHz. This is reflected in multiple processes slowing down (rendering, interpreter times, content delivery, etc). We cannot even stress these CPUs back to their limits in targeted fashion.
We've requested that our IT organization look closely at the hardware to make sure there is no failure going on. Last time they looked they determined a backup PSU was bad, but that doesn't explain why the server would degrade over time, does it? Since the PSU was replaced, we're still seeing the behavior on this server. Now that we're requesting further inquiries, IT has resorted to canned messages of "We've investigated the server and can find no issues," even while the CPUs are operating <30% of their advertised speeds.
Is there something obvious I'm missing? At this point it feels like I need to escalate this failure to management but I'm searching for all other options. I'm definitely not a SysAdmin, and don't understand what could be happening that's crippling my resources so terribly.
Are you able to take this server off-line for a bit? I think you're on the right track (with it sounding like a hardware issue); maybe if you can run a live USB of another OS, just to see what happens to the CPU speeds. If I had to guess, it's a motherboard issue. – ajgringo619 – 2019-10-21T21:55:39.710
I'm in the middle of setting up some application containerization for redeployment. My IT organization has neglected this server for ~10 years. I was hoping for something definitive I could take, but with everything that's stacking up, a failure in management seems like the only answer. I am hardware-ignorant, and so I was hoping for some insight on the kind of timeline to expect in case the hardware is failing.
It sounds like "ASAP" is the only answer, so far. – David Crane – 2019-10-21T22:10:20.020
IT orgs exist for one reason: to keep systems running so that users can access their resources. If they are not willing to expend any more energy to solve your problems, then you need to escalate this as far as you need to. – ajgringo619 – 2019-10-21T22:20:49.057
What information is reported by
cpupower frequency-info
over time? (If the server's too old to have that, thencpufreq-info
instead.) – user1686 – 2019-10-21T22:49:17.380@ajgringo619 If only I could flag comments as answers. I'm a developer, not a sys-admin. I'm not equipped for this type of work and it's taking away from what I'm best at. – David Crane – 2019-10-22T00:04:26.900
@DavidCrane - I feel for you, my friend. Incidences like this give IT a bad name; wouldn't have lasted 24 hours in my last job if I treated customers this way. Oh, and by the way - you are the customer here. – ajgringo619 – 2019-10-22T00:07:12.780
@grawity
no or unknown cpufreq driver is active on this CPU
Severe negligence, it seems. – David Crane – 2019-10-22T17:51:24.573