Mass downloading a set of URLs with login

1

Possible Duplicate:
How can I download an entire website

I'm looking a mass-downloader to help me download the entire database from a site I have an account on. The requirements are simple:

  1. Login using my user and password (form-based login)
  2. Download all URLs of the form "http://site.com/ID", where ID is less than 1000.

What should I use?

ripper234

Posted 2009-10-24T17:39:41.123

Reputation: 9 293

Question was closed 2009-11-15T16:22:11.150

Wow, so many answers, so little upvotes on any of them (0 upvoted answers so far). – ripper234 – 2009-10-25T09:51:09.293

How is this the same as downloading an entire site? I don't want to follow links, I want to download a set of links of a specific form. – ripper234 – 2009-11-16T12:46:42.423

Comparing these answers to the listed duplicate, they are all listing exactly the same answers provided to this question. Therefore it is a duplicate. Most of the software listed on the duplicate allows this functionality. – BinaryMisfit – 2009-11-16T14:08:14.427

Answers

1

GetGo Download Manager seems to do that:

You can easily set up a Batch Download that not only downloads a large list of files using an URL pattern template, but also automatically renames downloaded files using a filename pattern template.

harrymc

Posted 2009-10-24T17:39:41.123

Reputation: 306 093

1

I strictly recommend NeoDownloader. I've been using it for years.

It downloads from password protected sites and supports downloading URL ranges.

Mehper C. Palavuzlar

Posted 2009-10-24T17:39:41.123

Reputation: 51 093

0

Try Orbit Downloader. MANY options available with that.

squillman

Posted 2009-10-24T17:39:41.123

Reputation: 5 676

0

I have had good luck with Offline Explorer. Allows you to access password protected pages and has lots of advanced options.

Alok

Posted 2009-10-24T17:39:41.123

Reputation: 186

0

You can do this with wget, and possibly a very simple script (depending on how the site is set up).

CarlF

Posted 2009-10-24T17:39:41.123

Reputation: 8 576

0

HTTRACK works like a champ for copying the contents of an entire site. This tool can even grab the pieces needed to make a website with active code content work offline.

alt text

joe

Posted 2009-10-24T17:39:41.123

Reputation: 11 615