-3

We run a couple of onapp nodes that connect through iSCSI to a SAN. This is now using up all bandwidth and we want to know where it is coming from, we have been search for days now and would like to ask the community.

iftop:

 x.x.x.x  => y.y.y.y  279Mb   239Mb   239Mb
          <=          7.09Mb  5.74Mb  5.74Mb

If you need any information, please leave a comment.

Marco
  • 287
  • 1
  • 2
  • 12
  • 2
    What? Either it's coming from the IP `x.x.x.x` or `y.y.y.y`. Those should be in your network and under your control. – Sven Jul 22 '13 at 12:01
  • x.x.x.x is sending data to y.y.y.y. They are under our control, however we don't know how to find the VM that is creating the high outgoing traffic to the SAN – Marco Jul 22 '13 at 12:07
  • 2
    If you can't find out which of your VMs create such amounts of traffic, you have major problems in the way you maintain your systems. But you don't even tell us what kind of virtualization you use. – Sven Jul 22 '13 at 12:12
  • qemu-kvm VM's on a ONAPP node – Marco Jul 22 '13 at 12:14

2 Answers2

0

I have no idea about ONAPP, but if you have virsh, you can get traffic statistics like so: 

virsh domifstat <domname> <ifname>

and if you don't even know the interface names, you get them with 

virsh domiflist <domname>

Example: 

#> virsh domifstat myvm vnet2
vnet2 rx_bytes 3684910384
vnet2 rx_packets 29540870
vnet2 rx_errs 0
vnet2 rx_drop 0
vnet2 tx_bytes 1444997265
vnet2 tx_packets 12615196
vnet2 tx_errs 0
vnet2 tx_drop 0
Sven
  • 97,248
  • 13
  • 177
  • 225
  • Do you know how this will work for disk activity? because its not "internet" activity but iSCSI – Marco Jul 22 '13 at 12:40
  • Incidentally, the *i* in [iSCSI](http://en.wikipedia.org/wiki/Iscsi) stands for "Internet". It is SCSI encapsulated in TCP and is usually just "normal" network traffic. But a short `virsh help` would reveal the subcommands `domblkstat` and `domblklist` which do the same thing as the *if* commandos, but for block devices. – Sven Jul 22 '13 at 13:00
0

The problem was a full disk in one of the storage servers, it was trying over and over, so check your disks!

Marco
  • 287
  • 1
  • 2
  • 12