zcat/gzcat works in linux, not on osx. general linux/osx compatibility

Question

First the specific problem: In linux, I use zcat to list a .zip file. In osx, zcat seems to automatically append .Z to the file name. Various people suggest replacing zcat with gzcat; however, gzcat complains that the file is not in gzip format!

'file ' shows this: ...Zip archive data, at least v2.0 to extract

So neither zcat or gzcat will work in osx, what can I do?

I have a medium sized script in in bash which uses, zcat/gzcat, sed awk and other basic utilities to process a number of files. I'd like to duplicate that environment on my osx laptop so I can work offline. Any general suggestions how I can avoid such pain? I expect this is a fairly routine workflow so must have been sorted out by others.

@MichaelHampton I can zcat on linux fine. Neither zcat nor gzcat works on osx. So how can I resolve this? Is there another set of utilities I can install which will work both on linux as well as osx? These scripts are my own so I can change them (although I'd like to keep them simple). I control both linux/osx environments as well so I can do simple installs on both as well. — user23398, Jan 27 '14 at 03:25
I wouldn't say it's routine. I generally use `zip` to deal with zip files, not `gzip`. — Michael Hampton, Jan 27 '14 at 03:39

score 140 · Answer 1 · answered Jan 27 '14 at 00:48

140

You are right. It's annoying behavior.

$ zcat foo.txt.gz 
zcat: can't stat: foo.txt.gz (foo.txt.gz.Z): No such file or directory

Try this:

$ zcat < foo.txt.gz 
asdfadsf

answered Jan 27 '14 at 00:48

vy32

2,018
1
15
20

3

This works ok on OS X. – Valerio Schiavoni Jan 25 '15 at 14:18
2

Apparently my answer is wrong. Linux `zcat` will read `.zip` and `.Z` files. The issue here isn't one of stdin vs. named file, it's that the Linux maintainers of the programs modified `gzcat` to read a large number of unrelated file types. – vy32 Apr 04 '15 at 12:54
1

How would we pipe this to another process? – Petah Oct 09 '17 at 08:29
Use `|` to pipe to another process: `cat < foo.txt.gz | wc` – vy32 Oct 11 '17 at 04:24
This will fix your issue, at least on OSX – jimh Nov 14 '17 at 01:35
`cat < foo.txt.gz | wc` counts the number of lines in the compressed file, not in the uncompressed file. – Maximilian Mordig Aug 22 '22 at 12:40
1

If you only ever call `zcat` with the file argument (no other argument), you can fix it by adding the following to `~/.bashrc`: `function zcat { command zcat < "$1" }` – Maximilian Mordig Aug 22 '22 at 12:47

score 34 · Answer 2 · answered Jul 09 '15 at 06:13

34

I know this is an old question. However, I found a solution from an even older github thread.

You can simply use gunzip -c that works similar to zcat without the errors on Mac OS X.

$ gunzip -c 20150707_backup.sql.gz | mysql -u mysql_user -p

answered Jul 09 '15 at 06:13

dakdad

441
4
4

`gzip` might work instead of `gunzip` too. – Kenny Evitt Oct 29 '21 at 02:05

Peterino · Answer 3 · 2014-05-22T11:58:59.083

You can install GNU utils for Mac OS X, e.g., using homebrew (run brew install coreutils gnu-sed, which installs gzcat et al. and gsed). This will provide you with the GNU implementation of each command.

Then, to make your life easier, make a variable for each command (e.g. sed=gsed, and use $sed subsequently), or simply alias them (e.g. alias sed=gsed) in any script you write. I've written a Gist on this for your convenience: gnu-tools-for-mac.sh. Put (or include) this code on top of your scripts.

Use the GNU implementation for both compressing and extracting the archives. I'd say you can't mix the uses of the OS X and GNU implementation.

You can alternatively add `/usr/local/opt/coreutils/libexec/gnubin` to your `PATH`, which will make them available without the `g` prefix. — bfontaine, Jul 07 '14 at 17:56

score 12 · Answer 4 · answered Feb 06 '19 at 15:30

I just gave this a try and found something very interesting! For gz files, we are expected to run gzcat on Mac OS.

Solution 1

Use gzcat /path/to/file.gz and it should work.
Using zcat /path/to/file.gz will give you an error.

Solution 2

Pipe in the file to zcat and it will work.
For example, cat /path/to/file.gz | zcat will work.

I hope this helps!

score 1 · Answer 5 · answered Dec 28 '20 at 18:15

1

Instead of

zcat filename.gz | grep "hello"

Use

gunzip -c filename.gz | grep "hello"

answered Dec 28 '20 at 18:15

Geek

111
1

score 0 · Answer 6 · answered Jan 27 '14 at 03:37

0

The Mac OS X version of gzip(1) doesn't support reading zip files. From the man page:

This version of gzip is also capable of decompressing files compressed using compress(1) or bzip2(1).

But the Linux (GNU) version does.

gunzip can currently decompress files created by gzip, zip, compress, compress -H or pack.

Third parties have packaged GNU gzip for Mac OS X; you can find these from a web search. That is probably the path of least resistance.

answered Jan 27 '14 at 03:37

Michael Hampton

237,123
42
477
940

Searching the web and installing new software is *never* the path of least resistance. – vy32 Oct 11 '17 at 04:25

score 0 · Answer 7 · answered Jul 18 '18 at 21:04

Funny, I had the same issue when executing a bash-script, bash somescript.sh, which executed a for-loop with gzcat inside it. But trying these things inside the somescript.sh-script ...:

zcat < file.txt.gz
I had installed brew install coreutils already, but nonetheless, re-installed: brew reinstall coreutils
gunzip -c file.txt.gz

... amounted to nothing...

But I also used a reference to a directory:

SOMEDIR="~/DIR1/DIR2/DATA"

Which is interpreted as ~/DIR1/DIR2/DATA and apparently doesn't work within the bash-script within macOS. This worked:

SOMEDIR="${HOME}/DIR1/DIR2/DATA"

Which is interpreted as /Users/someuser/DIR1/DIR2/DATA. I don't know why - I'm not an expert macOS-BASH-techie - but it worked in conjunction with my original code ...

gzcat file.txt.gz | awk '{ print $1, $2, $3, $4, $5, $6 }' > ${SOMEDIR}/new.file.txt.

... within my somescript.sh-script.

Maybe it's useful to someone.

Best,

Sander

zcat/gzcat works in linux, not on osx. general linux/osx compatibility

7 Answers7

Solution 1

Solution 2