top - 16:33:08 up 155 days, 6:05, 5 users, load average: 27.93, 35.31, 31.34
At this moment it only shows 27.0 but the behavior is like so:
While mysqld uses 0% cpu the load raises to high values (also the server is on 99.20 Idle)
Whenever a query arrives the server, mysqld process starts using some CPU while the load value decreases, what could cause such a wierd behavior?
During all this time, it seems like the server is on idle and nothing is written/read from the disks.
[root@mysqlsvr ~]# date ; sar
ד' אוג 14 16:29:17 IDT 2013
Linux 2.6.32-71.29.1.el6.x86_64 (mysqlsvr) 14/08/13 _x86_64_ (80 CPU)
15:30:01 CPU %user %nice %system %iowait %steal %idle
15:40:01 all 0.72 0.00 0.90 4.80 0.00 93.57
15:50:01 all 1.07 0.01 1.83 3.13 0.00 93.96
16:00:01 all 0.55 0.00 0.80 4.69 0.00 93.96
16:10:01 all 0.24 0.00 0.19 3.23 0.00 96.34
16:20:01 all 0.70 0.01 1.09 3.59 0.00 94.61
Average: all 0.66 0.00 0.96 3.89 0.00 94.49
[root@mysqlsvr ~]# date ;iostat
ד' אוג 14 16:29:32 IDT 2013
Linux 2.6.32-71.29.1.el6.x86_64 (mysqlsvr) 14/08/13 _x86_64_ (80 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
0.17 0.00 0.21 0.01 0.00 99.62
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 0.89 1.12 7.26 15004502 97409836
sdb 16.10 0.96 1315.43 12815431 17644874178
dm-0 0.95 0.93 7.23 12433962 96936968
dm-1 0.00 0.00 0.01 15360 84016
dm-2 164.38 0.96 1315.43 12813185 17644874178
dm-3 0.01 0.02 0.03 333354 385384
fioa 88.60 29.26 1492.84 392514339 20024549216
drbd0 33.50 26.50 257.39 355470226 3452628864
drbd1 25.57 0.46 204.44 6197282 2742359000
[root@mysqlsvr ~]#