How to tar/untar the output on the fly

28

12

What is the idiomatic way to do the following

  • tar to stdout
  • read this tar output from stdout and extract to some other folder.

My solution is tar --to-stdout .. | tar -C somefolder -xvf -

But may be there is more idiomatic way to do it.

shabunc

Posted 2011-10-11T16:26:21.657

Reputation: 566

Answers

36

The same -f - option works for tarring as well.

tar -cf - something | tar -C somefolder -xvf -

GNU tar uses stdio by default:

tar -c something | tar -C somefolder -xv

rsync is also popular.

rsync -av something/ somefolder/

user1686

Posted 2011-10-11T16:26:21.657

Reputation: 283 655

9

Just adding another use-case here. I had a large directory structure on a system nearly out of disk space and wanted to end up with a tar.gz file of the directory structure on another machine with lots of space.

tar -czf - big-dir | ssh user@host 'cat > /path/to/big-dir.tar.gz'

This saves on network overhead and means you don't have to tar on the other side in case you'd wanted to use rsync for the transfer instead.

quickshiftin

Posted 2011-10-11T16:26:21.657

Reputation: 219

Netcat is perfect for this. (Cat from one host to another host). – Hennes – 2015-06-25T16:40:37.587

3@Hennes: With its lack of authentication, integrity checking, data encryption, as well as having to manually start it on both sides for each individual transfer (i.e. 2× the work), it sounds a bit less than perfect – user1686 – 2015-07-23T07:00:13.533

Most of the time I gzip it before dumping it over the network. Any integretiy failures are likely to show up as decompression errors (though I never got any when I used it). As to starting two programs: Aye, true. – Hennes – 2015-07-23T07:35:01.173

3It may be more work, but for sending a large compressed archive over a link during a time-sensitive operation between machines in a secured local network or over a VPN, piping through nc will be significantly faster than SSH (over a 1Gb network, often by a factor of 2). Send over an md5 sum of the archive for integrity checking. – Spooler – 2019-01-05T19:14:53.403