How to download a file from URL in Linux

Question

Usually one would download a file with a URL ending in the file extension.

To download Ubuntu ISO, one would simple

wget http://releases.ubuntu.com/14.04.3/ubuntu-14.04.3-desktop-amd64.iso

However, I came accross a site that I suspect uses ASP.Net / IIS.

A link to a ISO is in this form (I removed link contents incase of ... policies):

http://some.ip.in.here/website.com/IMAGENAME.ISO?md5=hUhpyFjk7zEskw06ZhrWAQ&expires=1454811899

I am not sure how to download this since it has the MD5 and expiry time as parameters, and so wget only downloads a web page, not this ISO.

Any suggestions?

Answer 1

Use

wget "http://some.ip.in.here/website.com/IMAGENAME.ISO?md5=hUhpyFjk7zEskw06ZhrWAQ&expires=1454811899"

Explanation: There is "&" character in the url. On linux and alike systems, this makes it a background process. Solution it to enclose url in double quoutes (") so that its treated as one argument.

Answer 2

If you are just trying to get a reasonable filename the complex URL, you can use the output-document option.

-O file
--output-document=file

Either of these forms will work.

As noted previously, be sure none of the special characters in the URL are getting interpreted by the command parser.

Answer 3

There are two ways you can do this using Curl.

In this first method you would the -O flag to write out the file based on the remote name from the URL; in this case it would most likely write the file out to the system as IMAGENAME.ISO?md5=hUhpyFjk7zEskw06ZhrWAQ&expires=1454811899; note how the URL value is contained by double quotes since & and ? might be misinterpreted as Bash commands:

curl -O -L "http://some.ip.in.here/website.com/IMAGENAME.ISO?md5=hUhpyFjk7zEskw06ZhrWAQ&expires=1454811899";

While that method technically “works” but the filename is confusing at best. So in this other method you would use output redirection—with > followed by a filename after the URL—to output the file contents to a file named IMAGENAME.ISO:

curl -L "http://some.ip.in.here/website.com/IMAGENAME.ISO?md5=hUhpyFjk7zEskw06ZhrWAQ&expires=1454811899" > "IMAGENAME.ISO";

So if you ask me, the second method works best for most average use. Also notice the -L flag being used in both commands; that commands tells Curl to follow any redirection links that a file download URL might have since a lot of times files on download services redirect a few times before landing at the destination payload file.