4

I would like to collect stats on TCP/UDP packets sent between two hosts (A and B). I am looking for a tool to install on A and B to get the following information:

  • kb/seconds
  • packets/seconds
  • (cumulative) bytes sent so far
  • (cumulative) packets sent so far
  • avg in sliding window

There is iptraf, but I have hard times to configure it from command line. tcptrack does not aggregate data... I am a little bit lost.. :D I could script tcpdump :/

Probably, I miss something quite obvious...

UPDATE: Both servers run ubuntu 14.04.

UPDATE 2: I experiment now with tcpflow (logs grows pretty fast) and I will check collectId

UPDATE 3: The easiest way to track traffic to a given IP or network is to use IPTABLES and IPTABLES collectD plugin.

Skarab
  • 327
  • 3
  • 11
  • Which operating systems are installed? do you need longtime statistics, or will this only be a test for a short period of time? And is it wright, that you only want to monitor the traffic between these two systems? – frupfrup Apr 27 '15 at 14:45
  • I need long time statistics, I make a test for a distributed system and I need to check whether the load distribution follows the expected patterns. PCP.io already provides me a lot of information within the nodes. – Skarab Apr 27 '15 at 14:48
  • @Skarab don't forget to up vote useful help and accept an answer if it sutes you – Martin Magakian Apr 27 '15 at 18:47

4 Answers4

4

One way to do it is using tshark.

Do the following :

  1. Capture traffic with relevant filters on both sides using tshark/tcpdump/whatever produces a pcap formatted file.
  2. Once finished, run tshark option -z over capture files.

Then :

  • to get packet and byte rates per second :

    tshark -q -r myfile.cap -z io,stat,1

This will produce something like this :

=============================
| IO Statistics             |
|                           |
| Interval size: 1 secs     |
| Col 1: Frames and bytes   |
|---------------------------|
|          |1               |
| Interval | Frames | Bytes |
|---------------------------|
| 0 <> 1   |     29 |  2026 |
| 1 <> 2   |     35 |  2440 |
| 2 <> 3   |     35 |  2440 |
| 3 <> 4   |     43 |  2920 |
| 4 <> 5   |     93 |  5776 |
| 5 <> 5   |     77 | 25758 |
=============================

  • to get packet and byte aggregation as well as average TCP window :

    tshark -q -r myfile.cap -z io,stat,0,"AVG(tcp.window_size)tcp.window_size"

The result will be something of this kind :

==============================================
| IO Statistics                              |
|                                            |
| Interval size: 5.109 secs (dur)            |
| Col 1: Frames and bytes                    |
|     2: AVG(tcp.window_size)tcp.window_size |
|--------------------------------------------|
|                |1               |2      |  |
| Interval       | Frames | Bytes |  AVG  |  |
|-----------------------------------------|  |
| 0.000 <> 5.109 |    312 | 41360 | 41363 |  |
==============================================


Now, if you get TCP's window average size out of the equation (you won't get this metric without a tool inspecting traffic), you can use standard monitoring tools to graph whatever you want about packet rates/aggregation on any protocol using for instance cacti.

Edit : Not sure what you meant by sliding window given the (upvoted) other answers. If you meant average of metrics for a given period of time starting at some date (and not TCP window size), then you can do the whole thing by throwing in a date filter with tshark commands.

But it seems you are simply seeking a global monitoring solution. Cacti, collectd, zabbix, shinken, nagios (with icinga or something like that) should be okay for your needs and customizable enough to filter on specific traffic if needed.

Xavier Lucas
  • 12,815
  • 2
  • 44
  • 50
  • The idea of sliding window: – Martin Magakian Apr 28 '15 at 01:30
  • The idea of sliding window (by 3): let say your record 10,6,10,4,6,6. You take the first 3 element starting at index 0. (10+6+10)/3=8.6. Then the last 3 element starting at index 1. (6+10+4)/3= 6.6 and so on... It will simply smooth your metrics... By the way I think your answer is very smart so I up vote you – Martin Magakian Apr 28 '15 at 01:36
2

Create a port mirror for both server and run wireshark from two computers to collect the data. Thus that will not impact your server in production.

yagmoth555
  • 16,300
  • 4
  • 26
  • 48
2

This is a very interesting question.
The answer depends on the setup of your network but, I will try to cover some case.
I quite believe CollectD is part of the answer.

First, as you want to collect your metrics "every second" your need to configure CollectD Interval value

Solution N°1 - Is ServerA EXCLUSIVELY speaking to ServerB on specific interface? (and vice-versa)
Install CollectD with the Interface plugin. It will "collect" part of what you need:

  • kb/seconds
  • packets/seconds
  • (cumulative) bytes sent so far
  • (cumulative) packets sent so far

Forward CollectD metrics to a time series database such as Graphite.
You will be able to visualise those metrics and apply a moving average function. It will match your need for:

  • avg in sliding window


Solution N°2 - Is ServerA only speaking to ServerB on TCP and specific port? (and vice-versa)
Very similar. Use CollectD and monitor the traffic only on the specific TCP port using CollectD and it TCPConns plugin. Also, use Graphie for the "avg in sliding window"

Martin Magakian
  • 148
  • 7
1

My solution for running 1-2 hours tests:

  • tcpflow to collect TCP flow, I use tcpflow -i eth0 -FT to have timestamp in the result file names
  • parse report.xml (DFXML) to create report on closed connections
  • parse the file names with captured traffic + record their sizes to report on the ongoing connection during the measurement
  • use matplotlib to plot graphs

Advantage:

  • detailed view on what is happening

Disadvantage:

  • files with captures traffic grow very fast
Skarab
  • 327
  • 3
  • 11