file downloaded via firefox and curl have different size

When I download a file from this link by Firefox its size is 74580 B, But when I download it by curl with exactly all of header was sent by Firefox its size is 79891 B (I copied all header from Firefox and paste it in curl command). what is the problem?

If you need any additional data ask me in comment.

My curl command:

curl --header 'Host: members.tsetmc.com' --header 'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:29.0) Gecko/20100101 Firefox/29.0' --header 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' --header 'Accept-Language: en-US,en;q=0.5' --header 'Referer: http://www.tsetmc.com/Loader.aspx?ParTree=15131F' --header 'Cookie: ASP.NET_SessionId=pwzbckbdpjlzqj45vcdbd455' --header 'Connection: keep-alive' 'http://members.tsetmc.com/tsev2/excel/MarketWatchPlus.aspx?d=0' -o 'MarketWatchPlus-1393_3_14.xlsx' -L

Arash Mousavi

Posted 2014-06-04T19:19:35.120

Reputation: 111

Answers

Webserver that is serve that file has enabled gzip compression, Firefox uncompress the gzip, but for curl I should --compressed to uncompress it.

So my curl command is:

curl --header 'Host: members.tsetmc.com' --header 'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:29.0) Gecko/20100101 Firefox/29.0' --header 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' --header 'Accept-Language: en-US,en;q=0.5' --header 'Referer: http://www.tsetmc.com/Loader.aspx?ParTree=15131F' --header 'Cookie: ASP.NET_SessionId=pwzbckbdpjlzqj45vcdbd455' --header 'Connection: keep-alive' 'http://members.tsetmc.com/tsev2/excel/MarketWatchPlus.aspx?d=0' -o 'MarketWatchPlus-1393_3_14.xlsx' -L --compressed

Arash Mousavi

Posted 2014-06-04T19:19:35.120

Reputation: 111

Glad you sorted it. I find meld on Ubuntu incredibly useful for comparing text files: it installs compare options into the Nautilus file manager context menu:-

if two or more files are selected, the context option is Compare, and meld is scheduled to compare all of them (I don't recommend more than four);
if a single file is selected, the context option is Compare Later, and the next file selected has the context menu Compare to previous file, as well as Compare Later to make this the new reference file for the next selection (for use if the files are on different discs or in different directories).

AFH

Posted 2014-06-04T19:19:35.120

Reputation: 15 470

I have meld, But I think meld doesn't work for binary files. am I true? – Arash Mousavi – 2014-06-04T20:04:26.120

1You are correct that meld doesn't work for binary files (it is a pity that it doesn't at least flag if two binary files are identical or not), but web sites are HTML text files, and meld works well on them.

In your case it would show the uncompressed file as text and the other as binary, so you would know immediately that there was a significant difference. You can look at binary files with bless. – AFH – 2014-06-04T21:31:45.447

1If both files are binary, the shell command cmp will tell you if they are identical or not, while the file command gives a reasonably good indication of the file type.

There are some binary comparison tools which will try to rematch after differences, but I have not tried them, and they would be of little use for compressed files, as small differences in uncompressed files will give huge differences when compressed.

(Sorry, I did not realise that I had been logged out, and I originally added a new answer, because it would not let me add another comment.) – AFH – 2014-06-04T21:46:12.637