7

I'm trying to transfer large files (3+ gigs each) across the lan. My connection is 100 megabit to the box, with a gigabit backbone. However, when transfering files between servers and from a server to my local computer, transfer speed hovers around 11 to 13 megabits per second.

Large downloads from the Internet, however, go significantly faster.

I have brought up this problem with my network infrastructure folks, but they insist that everything is fine with the network configuration.

Are there any tools out there that I can install on my computer or the servers to try to isolate the source of the slowdown?

Ryan Michela
  • 1,137
  • 3
  • 16
  • 23

8 Answers8

8

Firstly check the interface counters on each server, there should be 0 or close to 0 errors reported.

Secondly, check the duplex of both servers. If you are mixing 100mbit and GbE then you may have a duplex mismatch. Ensure that both sides are set to auto/auto, or manually set the speed and the duplex on all interfaces.

Thirdly, what kind of contention is there on the GbE backbone, can you confirm that sufficient headroom exists for your transfer ?

Finally, is your sending server capable of transmitting fast enough. As the comments below suggest, are you limited by the IO bandwidth of the senders drives, or by CPU (if you are using scp or similar) ?

Btw, 11 to 13 mega bytes per second is the theoretical max for 100mbit, are you sure the tool you are using to measure is reporting the correct units?

Dave Cheney
  • 18,307
  • 7
  • 48
  • 56
  • Altough I cannot tell you a tool which simply shows you the different throughputs, I have to add something to what Dave Cheney already said. Is your "sending" hard drive fast enough ? Is maybe ssh-compression/encryption tearing down the CPU so much there is no headroom left for NIC/RAID-driver ? I've had it once that scp was incredibly slow, so I set up a lighttpd simultanously, which ended up to be the FAR faster way. – lImbus May 20 '09 at 14:42
  • Good point, I didn't think of that. You say when sending from server A to server B you get 11-13 mbit, but when receiving from a remote server, you can achieve higher bitrates. I've updated my answer with a simple test of io bandwidth – Dave Cheney May 20 '09 at 14:53
5

Check out the following procedure that I use, which might help narrow down where the problem is.

When I test network throughput, it's best to remove as many variables as possible such as disk i/o or CPU limitations. Here's the procedure I use, which can illustrate the typical max you will ever see in your network.

Tools:

Both these utilities should be available on most modern *nix systems, either preinstalled, via packages, or able to be compiled from source.

Procedure:

  1. Run the following command on one computer to set up the UDP listener. (Using UDP will give higher performance numbers because of the lack of TCP overhead to transfer.)

    nc -ulp 5000 > /dev/null

  2. On the other machine, run the following command, substituting the listener's ip address, to send a stream of zeros across the network to the nc listener. The pv utility will show the current performance as tracked by the receiver.

    pv < /dev/zero | nc -u listener-ip-address 5000

Notes:

  • The theoretical max on a Gig Ethernet connection is 119MB/s (notice, MB, not Mb)
  • TCP can also be tested by removing the -u in both nc commands
  • In real world applications, you'll never reach the performance numbers illustrated in the pv output.

This method gives predictable results in a well designed network with properly functioning equipment. For that reason, it is a good tool to help discover if the network itself or the networking equipment is contributing to the perceived "slowness".

Dave K
  • 2,751
  • 2
  • 21
  • 17
1

Are you on Windows Vista? Note that Vista throttles LAN transfers any time you have a multimedia app open (including a web browser with flash).

I had this issue with transfers being capped at 10MB/s when they should've been xfering at 30-40MB/s.

You can disable the throttling in a registry key detailed here:

http://www.anandtech.com/systems/showdoc.aspx?i=3233&p=2

Matias Nino
  • 1,372
  • 7
  • 25
  • 40
0

Also consider differences in protocol between the two scenarios. "Large downloads from the Internet" sounds like HTTP or FTP over TCP/IP. On the LAN, are you using something different? Perhaps a mismatched set of SMB implementations, or Apple filesharing? I only mention it because I've seen SMB traffic between Windows and UNIX/Linux systems behave in some really braindead ways.

Try experimenting with different transfer mechanisms and see if that yields different results.

Adam D'Amico
  • 964
  • 8
  • 9
0

When you're transferring files 'between servers', are the servers talking directly?

Or, are you in Windows and do you have drives mapped to each server and you're dragging files across those?

In that second scenario, everything is going via your computer, so the fastest transfer you will get will be around 11MBps.

If you're on a *nix system, use ttcp to determine whether your TCP connection is the bottleneck.

MikeyB
  • 38,725
  • 10
  • 102
  • 186
0

If you are using windows I highly recommend using the free utility RichCopy. It is multithreaded and can pause and resume file copy operations. I have had very good luck using it to transfer files between severs.

notandy
  • 564
  • 1
  • 5
  • 19
0

It is a really old tool, but I've had good success with Netperf in the past. It should give you a starting idea of the bandwidth between two devices.

Aaron
  • 1,002
  • 1
  • 12
  • 18
0

Stupid suggestion, but is it possible that the transfer is not happening over the LAN but is actually going out over the internet? How are you connecting between the machines? I ask because your symptoms sound similar to those discussed in a recent episode of DotNetRocks. See the transcript of the solution start reading at:

Carl Franklin: Yeah. The Brain Buster, we wanted people to send in their suggestions or the answers they think, to what happened.

Rob Rolnick
  • 121
  • 1
  • 3