5
This link describes how to copy tarred files to minimise the amount of date sent over the network. I am trying to do something slightly different.
I have a number of remote files on different subdirectory levels:
remote:/directory/subdir1/file1.ext
remote:/directory/subdir1/subsubdir11/file11.ext
remote:/directory/subdir2/subsubdir21/file21.ext
And I have a file that lists all of them:
remote:/directory/allfiles.txt
To copy them most efficiently, on the remote site I could just do
tar zcvf allfiles.tgz `cat /directory/allfiles.txt`
but there is not enough space to do that.
I do have enough storage space on my local disc. Is there a way to tar
an incoming stream from a remote server (using scp
or ssh
for the transfer)?
Something like /localdir$ tar zc - | ssh remote
cat /directory/allfiles.txt` I would guess - but that would only list the remote files on the local host.
Excellent, that's what I was hoping to get! I had thought about this but expected the redirection to be interpreted on the remote machine not the local. I guess the hyphen takes care of that? – alle_meije – 2015-07-06T07:41:05.910
I don't think that the
-v
is a good idea. You're going to use extra bandwidth for that feature. You could instead verify the results with atar tvf remote_files.tar.gz
once the transfer is done. – Alexis Wilke – 2019-12-27T19:32:30.660If you really have to have to most minimal bits transfered over the network, sure. But really how much more bandwidth is '-v' going to take for showing the file progress? Data send over with ssh is probably also compressed too. – dirk – 2019-12-28T21:38:34.990