Zip many files into several archives

Question

Running Linux. I have a directory of around 150 large CSV files; simply doing a zip -9 on them results in a monolithic file that is still too large. I would like it to simply zip them in maybe four or five zip files of 30-40 CSVs each; this way sequencing or spanned zip order won't be a problem, as each zip is independent. There must be a simple way to do this. Any suggestions?

(and yes, zip is the preferred format, if possible)

Answer 1

Isn't -s switch enough? You may use zip -s to split the file into files of maximum size, e.g.:

"zip -s 300m <2 gb file>" produces:

file.zip (300 mb, master file)
file.001.zip (300 mb)
file.002.zip (300 mb)
file.003.zip (300 mb)
file.004.zip (300 mb)
file.005.zip (300 mb)
file.006.zip (200 mb)

Then "unzip file.zip" will unzip everything together.

Answer 2

Use split on the list of input files :-).

(Not tested, I've included rm commands for cleanup, take care).

ls *.csv > csvfiles
split -d -l30 - csvfiles < csvfiles
for i in csvfiles[0-9][0-9]; do
  zip "$i.zip" -@ < "$i"
done

rm csvfiles
rm csvfiles[0-9][0-9]