3
1
How can I transfer a large file quickly, when I don't really require a secure connection?
Often I would like to copy a large file from local machine to remote machine. The file does not already exist on remote machine. I usually use scp or rsync (which i think might use ssh underneath anyway). It can be kind of slow and I suspect it's not due to network limitations, but because it's trying to copy securely. Using blowfish cipher doesn't improve things much, and I couldn't get null cipher to work at all.
edit: for the record, copying a ~75MB file took about 17 minutes, and rsync reported 78032.72 bytes/sec. If my calculations are correct and the network is 10/100 (100,000,000 bits/sec) then isn't that only about 0.6% of the possible network speed?!
wim
Posted 2011-08-30T02:26:33.857
Reputation: 2 523
1
See also https://unix.stackexchange.com/questions/48399/fast-way-to-copy-a-large-file-on-a-lan
– rogerdpack – 2018-09-24T19:00:31.523This ServerFault Question may be of interest to you. Try FTP maybe?
– digitxp – 2011-08-30T03:53:50.1132I think your issue is not with the encryption - I regularly get much better speeds than that using scp even over encrypted wi-fi (802.11b or g). (for what it's worth, using scp, I generally see something close to 15-16 Mbit/s under optimal conditions, several times that on a wired connection). – user55325 – 2011-08-30T05:49:31.857
1Is the whole path from the local machine to the remote machine 100Mb/s? A network path cannot transfer data between two points faster than it can across the slowest link in the path. If you actually have a 100Mb/s path between them then are they close enough that you could just put the file on a USB stick and walk it to its destination? – mas – 2011-08-30T06:29:11.410
I'm sure the path isn't 100Mb/s but i doubt it's as bad as 0.6% of that either. I could use a USB stick, but that would mean.. getting the keys to the server room from the authorized person, taking down a keyboard, taking the lift to L27.. hmm and my seat is more comfortable ! :) – wim – 2011-08-30T06:32:29.557
2@digitxp Using the file transfer protocol to transfer a file? Who would want to do that? You must be joking! – Daniel Beck – 2011-08-30T19:50:01.250
Make sure it's not an IO issue on the remote machine. I once spent a lot of time tracking down what I thought was a network issue only to find it was a hard disk issue. – pmac72 – 2011-08-30T23:18:02.100
Also check latency (ping) and check for one-way bandwidth saturation. If a link is saturated in only one direction (say by other traffic or an asymmetric link speed) it will look on a bandwidth graph as though there is plenty to spare, but TCP connections will falter as ACK messages are delayed or dropped. Very high latency can have similar effects. – Slartibartfast – 2011-08-31T06:04:50.930