17

On a network interface, speeds are given in term of data over time, in particular, they are bits per second. However, in the uber-fast world of computing -- a second is kind of a really long time.

So for example, given a linear falloff. A 1 GBit per second interface would do 500MBit per half second, 250Mbit per quarter second etc.

I imagine at certain units of time, this is no longer linear. Perhaps this is set by ethernet frequencies, system clock speeds, interrupt timers etc. I am sure this varies depending on the system -- but does anyone have more information or whitepapers on this?

One of the main reasons I am curious is to understand output drops on interfaces. Even if the speed per second is much lower than the interface can handle -- perhaps there are spikes that cause drops for only small numbers of milliseconds. Perhaps various coalescing would hide this effect -- or perhaps increase it on the receiving interface? Do queues make a difference here?

Example:
So given if this is linear down to the MS we would have 1Mbit/MS, and if Wireshark isn't distorting what I see, should I see drops when I have a spike beyond 1Mbit?enter image description here

Kyle Brandt
  • 82,107
  • 71
  • 302
  • 444
  • The rated speeds are based on different variables typically. When speaking of BPS the largest packet is used. When speaking of PPS the smallest packet is used. – dbasnett Jun 25 '11 at 21:00
  • The phrase you're looking for is "microburst", could you post the make/model of the switch? – James Cape Jun 27 '11 at 00:49
  • I`ve read up some things about microbursts. During the planned downtime of server fault at 21:00 MEST I read the blog about the problem here. Kyle: Is it possible to artificially set the incoming lines down to 10 MBit/s? This might be another approach to counter microbursts... – Nils Aug 07 '11 at 20:21
  • Does the time Wireshark reports for the frame reflect when it actually went out on the wire, or just when it went into the outbound buffer in the OS and/or driver? – rakslice Oct 01 '11 at 04:45

2 Answers2

7

Speed is not that essential as many think. I had a server that had many drops (ingoing and outgoing) - but every tool only showed low bandwidth utilization (on a gigabit link, less than 2% average).

Nevertheless the statistic counters showed dropped tx and rx on the server. An analysis of the ip-packets showed that the server is being flooded by thousands of tiny ip sessions containing just a few bytes.

So the real problem - which you might see in your case, too - migth be that your hardware and/or IP-packet-stack is not able to serve enough packets per second.

What operating system are you talking about? If it is Linux I can give you further details.

Added: 2011-06-26 (We are talking about a Cisco 2960-S Switch/Router)

Strange that Cisco does not know their own data sheets. :-/

Look at Section "Forwarding Rate: 64-Byte Packet Cisco Catalyst 2960-S" here. From this I would say the 2960-S is the bigger brother of the 2960 and should do a lot of PPS. Where do you experience that problem? Is this a trunk/uplink?

Since you talk about layer 2 forget IP - we are at ethernet packet level. But the problem could be the same.

Where/how did you run that wireshark?

Nils
  • 7,657
  • 3
  • 31
  • 71
  • Where we mainly seem them is on our Cisco 2960S L2 Switches. – Kyle Brandt Jun 25 '11 at 20:38
  • PPS is interesting, not even sure what PPS I should I be able to push per second on a single port , let alone packets per millisecond. I Have asked Cisco about the the PPS on a port but they couldn't tell me. – Kyle Brandt Jun 25 '11 at 20:56
  • If you're talking about the input and output drop counters on a specific interface, Cisco has a nice page with troubleshooting suggestions at http://www.cisco.com/en/US/products/hw/routers/ps133/products_tech_note09186a0080094791.shtml. – James Sneeringer Jun 25 '11 at 21:29
  • @kyle - maximum wire speed pps on gigabit is 1,953,125 (assuming 64-byte packets). – Greg Askew Jun 27 '11 at 01:45
  • PPS speeds are just never calculated with 64 byte packets. That may be your problem. You may be switching way too many small packets and you hit the maximum PPS speed but not the maximum bit rate speed. – Antoine Benkemoun Jun 27 '11 at 06:28
  • This switch type is pretty fast. I rather think that the congestion occurs due to a congestion on the device connected to the port... – Nils Jun 28 '11 at 19:49
2

Since the raw line speed is 1 Mbps/mS as you said, the graph showing spikes slightly exceeding that must be slightly distorted. Packets that are lost are not there, and so are not part of the graph.

psusi
  • 3,247
  • 1
  • 16
  • 9