Do curl and wget open gzipped files automatically?

2

I made two http calls to the same gzipped resource from two different machines - RHEL and OSX.

curl http://example.com/file.gz

The first call returned an expected binary output:

/�IQ���}

The second call returned the content:

Hello, world!

The same phenomenon happened with wget - binary response from the RHEL machine, plaintext from the OSX one.

Could OSX be auto-opening my resource implicitly? Can I override these settings?

Adam Matan

Posted 2013-06-13T13:13:35.730

Reputation: 5 930

Answers

3

According to the OSX curl man page, some versions have automatic decompression of lzip encoded archives:

-V, --version
          Displays information about curl and the libcurl version it uses.
[...]

      The  third  line  (starts  with "Features:") shows specific features 
      libcurl reports to offer. Available features include:

[...]
      libz   Automatic decompression of compressed files over HTTP is supported.

So, I guess that if your run curl -V | grep libz on your OSX machine, you will see that it has this feature while the RHEL one does not.

terdon

Posted 2013-06-13T13:13:35.730

Reputation: 45 216

Thanks, that was it. Any idea how to disable? – Adam Matan – 2013-06-13T13:25:22.837

@AdamMatan not really, don't have a mac and my wget does not do this. You could try piping it through gzip: curl foo.gz | gzip > foo.gz. – terdon – 2013-06-13T13:30:39.020

1@AdamMatan: I cannot test either... although my curl -V indicates that zlib is active, I doesn't gunzip automatically. But perhaps the --raw switch will help you: When used, it disables all internal HTTP decoding of content or transfer encodings and instead makes them passed on unaltered, raw. (Added in 7.16.2) – mpy – 2013-06-13T13:48:09.930

Asked: 2013-06-13T13:13:35.730

Viewed: 1 527 times

Active: 2013-06-13T13:21:11.897