What program can I use to bulk download this list of links?

I need a program that I can give a lit of URLs to (either paste or in a file) like below and then it must be able to crawl those links and save files of a certain type, like images for example. I have tried a few spiders but not had luck.

Currently, the only way to download everything, is to open each link, then I use the "DownThemAll!" Firefox plugin which selects all the images (or any file type) on the page and downloads them. This works page by page, but I need something similar that works a whole list of URLs.

Does anyone have any suggestions?? Thanks a lot.

PS. Could I also add that it be something fairly easy to use that has a half decent user interface and doesn't run from the command line. Thanks

   http://www.someUrl.com/fhg/xbox/1/index.php?ccbill=123456    
   http://www.someUrl.com/fhg/allison_gf/1/index.php?ccbill=123456    
   http://www.someUrl.com/fhg/cookin/1/index.php?ccbill=123456    
   http://www.someUrl.com/fhg/blackjacket/1/index.php?ccbill=123456

Cheeky

Posted 2011-04-22T18:48:26.873

Reputation: 249

@DiegoDD Somehow this question received attention today and that is hilarious. You can also see the remaining parts of the links... – MC10 – 2015-08-18T18:41:37.823

if you are on linux/unix systems its a one liner, see here

– mohamnag – 2016-02-10T14:58:03.973

Annoying that this is closed as off topic, but this works amazing to save bulk html to pdf if you can write a simple c# program: https://code.msdn.microsoft.com/Convert-from-HTML-to-PDF-09ce2a1d#content Install-Package Select.HtmlToPdf

– Micro – 2016-05-19T16:36:23.390

Why would this be off topic? It's asking a question about how to accomplish a task that only a super user would typically be doing... – Jake Wilson – 2020-01-20T08:19:28.867

7i see you replaced allison19 with someUrl . kinky! =P – DiegoDD – 2013-12-21T03:41:50.760

Answers

There's not been any way of doing this from a browser or without downloading dodgy one-hit wonder freeware so I've written a Chrome browser extension that fits the bill.

It's called TabSave, available in the webstore here.

You can paste in a list of URLs and it'll download them, no fuss :-)

It also has the ability to download every tab open in the active window, hence the name. This is the default, just click the edit button to insert a list yourself.

It's all open source, the GitHub repo is linked in the webstore description if you want to send a pull request (or suggest a feature).

Louis Maddox

Posted 2011-04-22T18:48:26.873

Reputation: 576

Awesome Idea! But it crashes chrome after adding 400 urls :( – Hercules S. – 2016-11-30T02:58:44.757

your machine probably can't handle 400 URLs... feel free to fork and modify for your needs, thanks – Louis Maddox – 2016-11-30T10:36:09.930

Can an extension sync saved pages to mobile Chrome and put those web pages in "Read later"? I am looking for a way to download a lot of web pages (links would be a certain folder from my bookmarks) – Deepak Kamat – 2017-04-10T05:51:07.027

Currently this doesn't work for me and I see the same issue reported in the Github repo of the extension. When I paste a list of URL's it only downloads 1 of them. – Nikolay Tsenkov – 2017-12-08T10:38:24.723

thanks, works great for me (Vivaldi 1.10 Stable). Especially useful when your employer uses a proxy so you do not have to manually set another program and just use the internet browser. – Juhele – 2018-02-20T11:27:59.767

I tried it and it works, but unfortunately, the 'save as' dialog apperas for every item! So i guess it doesn't work... – user1584421 – 2018-07-18T21:30:59.083

Actually, i figured you can disable the 'save as' dialog in browser preferences.Your software works great! I wish if yiu could por this to firefox/safari – user1584421 – 2018-07-23T11:06:00.710

For your needs, Chrono Download Manager or TabSave can download a list of links quickly. Both are Chrome extensions, so no need to download desktop software.

And maybe this could be useful for you:

In my own experience, I prefer Chrono Download Manager because I needed to change automatically the name of the downloaded file in a BATCH-way (a list of VIDEOS from a hmm hmm... online courses) and crawling in the html code all the different videos have the same filename. So downloading it with TabSave just gives you the same name videos and you have to guess wich is the content (somewhat like "543543.mp4", "543543(1).mp4", "543543(2).mp4" and so and so).Imagine how much extra work you need to do to achieve this kind of task.

If you need quick list download of files as-is, go TabSave. If you need start to need change the name files on the run, go Chrono.

Knomo Seikei

Posted 2011-04-22T18:48:26.873

Reputation: 211

Please exercise caution when recommending software. As written, your answer may be seen as spam. More information: How do I recommend software in my answers?

– bwDraco – 2015-07-01T01:14:42.043

You were right @DragonLord , I fixed my answer so didn't look so biased. – Knomo Seikei – 2015-07-02T14:22:22.430

1Chrono saved me 234 hrs of work!!!!! THANK YOU! – Hercules S. – 2016-11-30T03:02:46.293

@HerculesS. You are welcome. :) – Knomo Seikei – 2016-11-30T06:29:26.717

Chrono Download Manager is now dead or removed from the Chrome store? – Kenny LJ – 2019-10-02T03:21:11.637

I know I'm gravedigging here, but I was searching for a similar program and found BitComet which works really well, you can import url's from textfiles etc.

ertan

Posted 2011-04-22T18:48:26.873

Reputation: 31

Almost great. The images of the batch pages aren't downloaded and link to the online source unfortunately. – user5389726598465 – 2017-07-19T03:10:06.560

Ok, I have found an application that does it beautifully. It's called Picture Ripper

Thanks for the help anyway Dudko!

Cheeky

Posted 2011-04-22T18:48:26.873

Reputation: 249