6

I am asked to investigate a high CPU usage alert which occurred this morning. I used sar -p and saw the high CPU usage showing up at that time

Next I used ps -eo pcpu,pid,user,args | sort -r -k1 | less to list the top 10 memory hogs at this hour

Now how do I find out what processes caused the bottleneck at that specific time in the morning. I am a java developer and not a Linux expert.

Is it even possible?

juniorbansal
  • 163
  • 1
  • 1
  • 4

1 Answers1

9

There are several options:

  1. use a script which writes needed data on a regular basis to a logfile. You could use cron to write the output of ps (and other commands) every x minutes into a logfile.
  2. Better it would be to use a specialized program, which does this for you. atop is very good at this, at it takes care of logfile retention.

atop is available via the EPEL repo for CentOS/RHEL/Fedora and via the default repos of Debian/Ubuntu.

You can use atop like a normal real-time top utility, with slightly different behaviour (check out the manpage for keystrokes).

The more interesting part is: Once installed a daemon starts logging data into /var/log/atop and you can read these files with atop again:

atop -r /var/log/atop/atop_20160128

You have then access to all 'top' like functions (sorting/looking at memory/CPU/IO usage, etc.) and you can jump 10 minutes forward in time via 't' and 10 minutes back with 'T' or jump at a specific time via 'b'.

Check out the atop manpage and google has lots of howtos about it.

There might be other solutions, but atop is easy to understand and use and a good start before doing some more bespoke setups.

Thomas
  • 381
  • 1
  • 3
  • So - is there anything that linux offers out of the box? I am not the linux admin and do not have control over installing atop. assuming i can't install anything new is it possible to get the list of processes from the past? – juniorbansal Jan 28 '16 at 20:26
  • 1
    option 1: set up a cronjob to log every x minutes the data you need. PS: then it should be the job of the admin to provide you the data... – Thomas Jan 28 '16 at 21:11
  • @Shiv: Unless you had something running to record them in the past, there is no way to get that information now. So the only way to get what you want in the future is to install something now, like atop, or run a custom shell script from cron. Or I suppose you could download atop, configure it to run in your user directory and run it from there, if you really cannot install system packages. – Zan Lynx Jan 28 '16 at 21:24