8

I have a set of 100 log files, compressed using gzip. I need to find all lines matching a given expression. I'd use grep, but of course, that's a bit of a nightmare because I'll have to unzip all files, one by one, grep them and delete the unzipped version, because they wouldn't all fit on my sevrer if they were all unzipped.

Anyone has a little trick on how to get that done quickly?

Julien Genestoux
  • 609
  • 8
  • 19
  • You mentioned the requirement of needing to find all lines matching a given expression. Does finding them mean that you need to know which file they're in? If not, more options are open to you, particularly those that pipe their output to the stdin of grep. – LarsH Oct 21 '13 at 20:49

5 Answers5

25

You might have a look at zgrep.

>$ zgrep -h
grep through gzip files
usage: zgrep [grep_options] pattern [files]
mveroone
  • 447
  • 7
  • 22
Francois Wolmarans
  • 1,570
  • 10
  • 14
10

The zgrep program exists for this specific purpose.

http://linux.about.com/library/cmd/blcmdl1_zgrep.htm

Emyl
  • 380
  • 2
  • 11
4

Or, if your OS doesn't come with zgrep, something like this.

gunzip -c somefile1.gz [...] | grep 'string'
USD Matt
  • 5,321
  • 14
  • 23
  • Which is how `zgrep` is actually implemented, as it is just a shell script (with some additional stuff to handle arguments and report errors properly). – liori Oct 21 '13 at 20:57
3

There's a utility called zcat, which is a version of cat that works on gzipped files. In your situation, you could do something like the following:

zcat yourfile.gz | grep 'serverfault is awesome'
Boscoe
  • 564
  • 3
  • 6
1

While zgrep appears to be the next best thing since sliced bread, I thought I'd post an alternative...

You can use piping to unzip the log files before grepping them, without taking up disk space. For example:

cat /var/log/program.log.*.gz | gunzip | grep 'stuff and things'
Soviero
  • 4,306
  • 7
  • 34
  • 59