How can I measure the network traffic used by one command?

Question

I often write :

time npm install

which will give me as a result :

real    0m22.933s
user    0m24.058s
sys     0m3.346s

I would like to have the same type of information, but for network consumption.

For example,

netmeasure npm install

would return :

Upload : 499kb
Download : 44MB

That would be useful to be able to do "regression tests" on some programs.

Is is possible to get this kind of information with simple unix tools ?

I have already tried nethogs, but it is something very interactive, which might be useful if you want to know what is happening on one full unix system, but I'm interested in measuring the network consumption of one command. I'm not looking for an interactive program, but one that will spawn the process for me and wait until it exists to show those statistics.

I have found https://github.com/jonasdn/nsntrace that looks like a good tool, but inside nsntrace, I have no connectivity (see issue https://github.com/jonasdn/nsntrace/issues/22)

Answer 1

I would also have suggested running the program you want inside a network namespace, which is exactly what the nstrace tool does, though I don't understand why they don't simply script the setting up of the namespace.

The nsntrace tool requires forwarding and uses iptables to masquerade the veth-pair connection into the main network namespace to the outgoing interface. Very likely that went wrong because what it does is not compatible with your internet setup. Especially name server issues in network namespaces are a bit hairy.

So I'd suggest you read up on network namespace (ip netns), masquerading, iptables and the DNS resolver (/etc/resolv.conf, /etc/netns/NAME/resolv.conf) to be able to find out what exactly went wrong.

Here's a script I use to setup a network namespace and start an xterm "inside" this namespace, if that helps:

#!/bin/bash

# Setup network namespace with veth pair, start xterm in it

# nsterm ns0 veth0 10.0.0 yellow 24

if [[ $EUID -ne 0 ]]; then
   echo "This script must be run as root" 1>&2
   exit 1
fi

NS=${1:-ns0}
DEV=${2:-veth0}
DEV_A=${DEV}a
DEV_B=${DEV}b
ADDR=${3-:10.0.0}
ADDR_A=${ADDR}.254
ADDR_B=${ADDR}.1
MASK=${5:-24}
COL=${4:-yellow}

# echo ns=$NS dev=$DEV col=$COL mask=$MASK

ip netns add $NS
ip link add $DEV_A type veth peer name $DEV_B netns $NS
ip addr add $ADDR_A/$MASK dev $DEV_A
ip link set ${DEV}a up
ip netns exec $NS ip addr add $ADDR_B/$MASK dev $DEV_B
ip netns exec $NS ip link set ${DEV}b up
ip netns exec $NS ip route add default via $ADDR_A dev $DEV_B
ip netns exec $NS su -c "xterm -bg $COL &" YOUR_USERNAME

Adapt as required (after you read up on it to understand what it does). You still need to enable forwarding

echo "1" | sudo tee /proc/sys/net/ipv4/ip_forward

and setup masquerading in whatever form needed.

I'm afraid I can't think of a simpler solution to measure exact network usage.

Answer 2

You can get an approximation with a few basic commands. Probably nstrace is better, but I couldn't find it in my distro.

All you need to do is examine system calls used by application. An application which wants to read from network needs to call something from recv family or read. If you want to send something, you use write or send family of syscalls. Our heuristics will skip read and write, since it's too hard for one-liner to analyze.

So, here we go:

1. Save system calls used by application to a file: strace myapp 2>syscalls
2. grep the required syscall (in this case send), take their return value (last block of digits after the = sign), and count its sum. grep "send" syscalls | grep -E --only-matching "= [0-9]+$" | awk '{ print $2 }' | paste -sd+ | bc

I reiterate: this one-liner will not always give you the correct result, but should work most of the times.