Downloading very large files via SFTP

Question

I need to download some very large files (circa 75 GB) from a remote server via SFTP. I've been using SFTP via the command line on my Linux netbook. Around halfway through, the transfer stops and says "stalled." Can anybody recommend a reliable way to download these files?

Answer 1

I ran into this a while back and this article helped me out alot: http://ubuntuforums.org/showthread.php?t=383505

A user suggests to use SCP

Answer 2

Sadly the link provided by MaxMackie doesn't work anymore.

Since you have access via sftp you might also have access via rsync which by default operates through an ssh tunnel (which sftp also does). rsync is generally a good choice to downloading (syncing) a lot of files or big files over slow networks. One of the many features is, that it can resume synchronisation (download).

In your case a command like

$ rsync -P machine.example.com:/path/to/bigfile .

would do what you want to accomplish. In case the connection breaks or you need to resume later on for any other reason you can just execute this command again. From the man page:

          The -P option is equivalent to --partial --progress.   Its  pur‐
          pose  is to make it much easier to specify these two options for
          a long transfer that may be interrupted.

--partial
          By default, rsync will delete any partially transferred file  if
          the  transfer  is  interrupted. In some circumstances it is more
          desirable to keep partially transferred files. Using the  --par‐
          tial  option  tells  rsync to keep the partial file which should
          make a subsequent transfer of the rest of the file much faster.

--progress
          This  option  tells  rsync  to  print  information  showing  the
          progress of the transfer. This gives a bored user  something  to
          watch.