3
I have to pack circa 7,000,000 small files. I'm looking for an efficient way to do this.
Is tar the best way? What about benchmark tests or any other suggestion?
tar: million of files
Answers
5
Some people like 7-Zip. For example that's how Wikipedia packages up complete images.
bmargulies
Posted 2009-11-26T13:56:11.477
Reputation: 1 121
Indeed. I'd use 7zip for EVERYTHING, except system backup on Linux. Since the files lose their attributes. That stays in .tar.gz , "but for everything else.." :~) – Apache – 2010-06-30T09:01:13.070
I'd go with 7zip. – Grumpy ol' Bear – 2009-11-26T20:50:09.093
2
TAR is actually doesn't have it's own compression. TAR is just merging directories/files into one big file. The speed of creation of tar archive depends more on filesystem and I/O subsystem than on tar itself. The best way to get the highest compression ratio is to use 7zip (7zr command on *nix/Linux). The bigger dictionary size you choose - the smaller archive you receive.
For Linux you can try:
7zr a -t7z -m0=lzma -mx=9 -mfb=64 -md=128m -ms=on /path/to/put/archive.7z folder_to_compress
melco-man
Posted 2009-11-26T13:56:11.477
Reputation: 261
0
jar is also very nice, as it's a zip (compression) of the files. If not for the manifest inclusion, it would be close to perfect (as jar seems to exist just about everywhere now)
KevinDTimm
Posted 2009-11-26T13:56:11.477
Reputation: 220
and that's different from my answer how? – KevinDTimm – 2010-02-11T16:17:50.517
1why would you use jar over zip? – lfaraone – 2010-08-20T13:33:43.597
2fyi, a jar file is a zip file w/ an automatically added manifest. No difference. – basszero – 2009-11-26T19:59:35.153
1In one directory? – Mark Byers – 2009-11-26T14:00:06.603
2Efficient in what way? Compresionspeed? Filesize? etc. – Grumpy ol' Bear – 2009-11-26T20:23:25.027