Influence of kernel code execution to Load average in top?

Question

I'm trying to tests the amount of CPU that iptables use for NAT. Perhaps it will be more accurate to say "the amount of CPU that the netfilter kernel module" uses.

The approach i took was to try and strip the system of all userspace processes, run the streaming and check the Load average in top. In that case, the load averages are pretty low, as if the processor isn't being affected by packet routing.

However, after reading a bit about what load average means (here) it seems it doesn't represent the CPU utilization but the length of the process queue.

From my current understanding, execution of kernel modules aren't treated as processes which is why it will never be listed in the process queue.

Does that mean that the only way to detect CPU overload by kernel code is to run it with a userspace code and see the userspace code fail due to not getting processor time?

A simple example to formulate the question. If there was a piece of kernel code having a 'busy wait' bug, how would you detect that it's the kernel that's overusing the cpu?

I've found out that monitoring what percentage of CPU is used for software interrupt handling can give a clue about netfilter's CPU usage. — TheMeaningfulEngineer, Dec 22 '14 at 13:54

score 1 · Answer 1 · answered Dec 22 '14 at 11:22

You could simple add your own tracing rules (by using unused tcp port, for sample):

for table in $(</proc/net/ip_tables_names);do while
    read dsc nam foo ;do
        if [ "$dsc" == "Chain" ];then
            iptables -t $table -I $nam -p tcp --dport 8 -j LOG --log-prefix "start-${table}-${nam} "
            iptables -t $table -A $nam -p tcp --dport 8 -j LOG --log-prefix "end-${table}-${nam} "
          fi
      done < <(iptables -n -t $table -L)
  done

Once rules ready, you could make a connection from elsewhere:

nc 192.168.1.31 8
(UNKNOWN) [192.168.1.31] 8 (?) : Connection refused

Than take a look into your kern.log:

Dec 22 12:15:33 localhost kernel: [4237698.275654] start-nat-PREROUTING IN=br0 OUT= PHYSIN=eth0 MAC=01...
Dec 22 12:15:33 localhost kernel: [4237698.275662] end-nat-PREROUTING IN=br0 OUT= PHYSIN=eth0 MAC=01...
Dec 22 12:15:33 localhost kernel: [4237698.275674] start-filter-INPUT IN=br0 OUT= PHYSIN=eth0 MAC=01...
Dec 22 12:15:33 localhost kernel: [4237698.275681] end-filter-INPUT IN=br0 OUT= PHYSIN=eth0 MAC=01...
Dec 22 12:15:33 localhost kernel: [4237698.275688] start-nat-INPUT IN=br0 OUT= PHYSIN=eth0 MAC=01...
Dec 22 12:15:33 localhost kernel: [4237698.275695] end-nat-INPUT IN=br0 OUT= PHYSIN=eth0 MAC=01...

This test rules could be removed simply:

iptables-save |
    sed -ne '/^\*/{s/^.//;h;};
             /dport 8 -j LOG/{G;s/^-A\(.*\)\n\(.*\)$/iptables -t \2 -D\1/;p}
    ' | sh

Of course, depending on details from customs rules, this could have to be adapted: mostly using tcp port could be not adequate for testing udp rules, or specific tcp traffic... — F. Hauri - Give Up GitHub, Dec 22 '14 at 12:07

Influence of kernel code execution to Load average in top?

1 Answers1