Why is my router CPU at 40% when no processes use more than 2%?

Question

I have a problem, I have a Cisco 1841 running Cisco IOS 15, and I get strange behavior. The CPU usage is shown as 40%, but there is no processes that is using this much CPU power.

Here is an example:

lev1841#show processes cpu sorted 
CPU utilization for five seconds: 41%/39%; one minute: 42%; five minutes: 32%
 PID Runtime(ms)     Invoked      uSecs   5Sec   1Min   5Min TTY Process 
  96          88      147299          0  1.11%  1.04%  0.92%   0 Ethernet Msec Ti 
 117          40       36582          1  0.15%  0.19%  0.17%   0 IPAM Manager     
 240          28       36535          0  0.15%  0.14%  0.12%   0 MMON MENG        
   2          92         236        389  0.07%  0.04%  0.03%   0 Load Meter       
 183          24        1775         13  0.07%  0.02%  0.00%   0 CEF: IPv4 proces 
 140           8        4661          1  0.07%  0.02%  0.00%   0 SSS Feature Time 
 121        2236        1958       1141  0.07%  0.19%  0.17%   0 IP Input         
 176           4        1172          3  0.07%  0.00%  0.00%   0 RUDPV1 Main Proc 
 212           4        2287          1  0.07%  0.00%  0.00%   0 CCE DP URLF cach 
  95           8        6733          1  0.07%  0.03%  0.02%   0 Ethernet Timer C 

lev1841#show processes cpu history 

lev1841   04:11:07 PM Saturday Jan 14 2012 UTC




      444444444444444444444444444444444444444444444444444444444444
      333111111111111111111111111122222111111111111111111111111133
  100                                                             
   90                                                             
   80                                                             
   70                                                             
   60                                                             
   50                                                             
   40 ************************************************************
   30 ************************************************************
   20 ************************************************************
   10 ************************************************************
     0....5....1....1....2....2....3....3....4....4....5....5....6
               0    5    0    5    0    5    0    5    0    5    0
               CPU% per second (last 60 seconds)

lev1841# show processes cpu extended 
################################################################################
Global Statistics
-----------------
5 sec CPU util 41%/39% Timestamp 00:22:52
Queue Statistics
----------------
          Exec Count  Total CPU    Response Time           Queue Length
                                    (avg/max)                (avg/max)
Critical           1          0          0/0                   1/1         
High             683          0          0/0                   1/2         
Normal           462          8          0/4                   1/7         
Low               14          0          0/0                   1/2         
Common Process Information
-------------------------------
 PID Name            Prio Style
-------------------------------
  95 Ethernet Timer C H  New
  96 Ethernet Msec Ti H  New

CPU Intensive processes
-------------------------------------------------------------------------------
 PID Total       Exec    Quant         Burst  Burst size  Schedcall  Schedcall 
     CPUms      Count   avg/max        Count avg/max(ms)      Count Per avg/max
-------------------------------------------------------------------------------

Priority Suspends
------------------------------------
 PID Exec Count Prio-Susps
------------------------------------
  95         38         19
  96        644         19

Latencies
-------------------------
 PID Exec Count   Latency
                  avg/max
-------------------------
################################################################################

I can't find what is causing all this CPU load.

Can you help me ?

Here is some information on the router :

Cisco 1841 (revision 6.0) with 358400K/34816K bytes of memory.
System image file is "flash:c1841-adventerprisek9-mz.151-4.M1.bin"

The system has just been updated to IOS 15.4M1 from IOS 13.

Thank you.

Have you had a look at the Cisco troubleshooting guide for high cpu utilisation here : http://www.cisco.com/en/US/products/hw/routers/ps133/products_tech_note09186a00800a70f2.shtml — Lucas Kauffman, Jan 14 '12 at 16:11
I'm looking at it, looks like the high cpu usage of because of interruptions... — Kedare, Jan 14 '12 at 16:18
How much traffic are you moving? What are you doing to that traffic? — David Schwartz, Jan 15 '12 at 04:18
Maximum 1.5MBps of traffic, doing NAT and inspection, disabling inspection don't change anything, and I did not have all this load before the firmware upgrade — Kedare, Jan 15 '12 at 16:43

Lucas Kauffman · Answer 1 · 2012-01-14T21:44:45.300

9

Have you looked at the Cisco troubleshooting guide for high cpu ? It has an extensive guide on what to do when encountering high CPU loads.

Included are :

Determining Causes and Solving the Problem :

High CPU Utilization due to Interrupts
High CPU when Enabling Netflow NDE on Cisco 7600 Series Router
High CPU Utilization due to Processes
PCI and FAST Memory Pools Show Very High Utilization
%SNMP-4-HIGHCPU: Process exceeds [dec]ms threshold ([dec]ms IOS quantum) for [chars] of [chars]--result [chars]
High CPU due to Software Encryption

edited Jan 14 '12 at 21:44

answered Jan 14 '12 at 16:21

Lucas Kauffman

16,818
9
57
92

I found it's because of interrupts, but I can't find why – Kedare Jan 14 '12 at 16:47
1

tried this ? http://www.cisco.com/en/US/products/hw/routers/ps359/products_tech_note09186a00801c2af0.shtml – Lucas Kauffman Jan 14 '12 at 16:49
1

Paraphrasing the important parts of the linked text would be nice. If the link moves or breaks, this answer won't help anyone in the future. – Tim Post Jan 14 '12 at 21:21
The important parts of a 5 page troubleshooting guide ? :p – Lucas Kauffman Jan 14 '12 at 21:32
Summarizing the points at the very least. For the purposes mentioned by @TimPost. The guide could move then the answer is not helpful. – Belmin Fernandez Jan 14 '12 at 21:34

score 9 · Answer 2 · answered Jan 14 '12 at 17:23

9

One thing to try:

router(config)#no logging console

no logging console will disable the output of the debug message on the serial console. As the console is interrupt-driven, each character is an interrupt.

answered Jan 14 '12 at 17:23

petrus

5,287
25
42

score 2 · Accepted Answer · edited May 17 '12 at 06:17

The figures shown by a "show processes cpu" are a little misleading, in your output below:

lev1841#show processes cpu sorted 
CPU utilization for five seconds: 41%/39%; one minute: 42%; five minutes: 32%

The five seconds figure of 41%/39% should be read as "Total CPU usage"/"CPU Usage Caused by traffic".

i.e. Your processes are using 2%, the rest is being caused by the normal flow of traffic through the router. If you were to shut down the traffic carrying interfaces, this figure would drop away to around 2%.

It might be worth checking that you've enable CEF on the router?

Why is my router CPU at 40% when no processes use more than 2%?

3 Answers3