I have a problem about balancing irq. I took irq numbers from /etc/interrupts and assigned each irq to each of 24 CPUs as this tutorial says
for i in {143..166};do cat /proc/irq/${i}/smp_affinity; done
000001
000002
000004
000008
000010
000020
000040
000080
000100
000200
000400
000800
001000
002000
004000
008000
010000
020000
040000
080000
100000
200000
400000
800000
After that we tests server by sending about 10 million pps and observe output of top command.
top - 15:19:06 up 5:02, 8 users, load average: 12.51, 12.86, 12.38
Tasks: 278 total, 17 running, 261 sleeping, 0 stopped, 0 zombie
%Cpu0 : 0.0 us, 0.0 sy, 0.0 ni, 0.7 id, 0.0 wa, 99.3 hi, 0.0 si, 0.0 st
%Cpu1 : 0.0 us, 0.0 sy, 0.0 ni, 1.0 id, 0.0 wa, 99.0 hi, 0.0 si, 0.0 st
%Cpu2 : 0.0 us, 0.0 sy, 0.0 ni, 0.7 id, 0.0 wa, 99.3 hi, 0.0 si, 0.0 st
%Cpu3 : 0.0 us, 0.0 sy, 0.0 ni, 1.0 id, 0.0 wa, 99.0 hi, 0.0 si, 0.0 st
%Cpu4 : 0.0 us, 0.0 sy, 0.0 ni, 4.0 id, 0.0 wa, 96.0 hi, 0.0 si, 0.0 st
%Cpu5 : 0.0 us, 0.0 sy, 0.0 ni, 4.7 id, 0.0 wa, 95.3 hi, 0.0 si, 0.0 st
%Cpu6 : 0.0 us, 0.0 sy, 0.0 ni, 5.3 id, 0.0 wa, 94.7 hi, 0.0 si, 0.0 st
%Cpu7 : 0.0 us, 0.0 sy, 0.0 ni, 6.0 id, 0.0 wa, 94.0 hi, 0.0 si, 0.0 st
%Cpu8 : 0.0 us, 0.3 sy, 0.0 ni, 5.0 id, 0.0 wa, 94.7 hi, 0.0 si, 0.0 st
%Cpu9 : 0.0 us, 0.3 sy, 0.0 ni, 5.6 id, 0.0 wa, 94.1 hi, 0.0 si, 0.0 st
%Cpu10 : 0.0 us, 0.3 sy, 0.0 ni, 5.3 id, 0.0 wa, 94.4 hi, 0.0 si, 0.0 st
%Cpu11 : 0.0 us, 0.0 sy, 0.0 ni, 6.0 id, 0.0 wa, 94.0 hi, 0.0 si, 0.0 st
%Cpu12 : 0.0 us, 0.0 sy, 0.0 ni, 1.0 id, 0.0 wa, 99.0 hi, 0.0 si, 0.0 st
%Cpu13 : 0.0 us, 0.0 sy, 0.0 ni, 1.0 id, 0.0 wa, 99.0 hi, 0.0 si, 0.0 st
%Cpu14 : 0.0 us, 0.3 sy, 0.0 ni, 1.0 id, 0.0 wa, 98.7 hi, 0.0 si, 0.0 st
%Cpu15 : 0.0 us, 0.0 sy, 0.0 ni, 0.7 id, 0.0 wa, 99.3 hi, 0.0 si, 0.0 st
%Cpu16 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu17 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu18 : 0.3 us, 0.7 sy, 0.0 ni, 99.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu19 : 1.3 us, 0.3 sy, 0.0 ni, 98.3 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu20 : 0.3 us, 0.3 sy, 0.0 ni, 99.3 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu21 : 0.0 us, 0.3 sy, 0.0 ni, 99.7 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu22 : 0.7 us, 0.3 sy, 0.0 ni, 99.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu23 : 0.3 us, 0.3 sy, 0.0 ni, 99.0 id, 0.0 wa, 0.3 hi, 0.0 si, 0.0 st
KiB Mem: 65947892 total, 1055364 used, 64892528 free, 24544 buffers
KiB Swap: 50321404 total, 0 used, 50321404 free, 348920 cached
My question is why only 16 of 24 CPU loaded? Is this normal? How can I balance irq to all CPUs?
