Way to Initiate Download remotely to another computer

Question

I'm looking for a solution between a proxy and wget, since neither fully addresses the issue I'm trying to solve.

I want to download files to a specific directory on my home machine (a Linux server - command-line only) from remote places (via smartphone, web browser, etc.). This sounds like a solution for wget, except many of the files I would download come from sources that create special download "sessions" for you via AJAX/Javascript (e.g. Uploaded.net, Mega.nz).

I imagine I would have to use a proxy at some point in this process, but I don't want the download data to come to my remote device at all, since that would waste bandwidth. But, to use a proxy to access the sites and create the sessions would work for me.

So, solutions that don't work (out-of-the-box, at least):

• Remote Desktop: Windows only
• VNC: I only have command-line access; also, not so friendly with smartphones and the like
• wget: download links generally only exist for a "session"; very difficult to emulate AJAX environment for sites
• proxy: sends all data to remote client; mounting the remote directory would cause the data to take three hops (site to proxy, server/proxy to client, client to remote directory). I'm looking for a direct (site to remote directory) route for the download.

In my perfect world, I'm envisioning browsing to a site via a proxy, but then when the browser classifies something as a Download, it just downloads to the remote directory (or even opens a file Save dialog and lets me choose where to save it on the remote server). The solution doesn't have to have this level of simplicity, but hopefully this helps clarify the thrust of my question.

Answer 1

For my needs, I have settled on JDownloader, an open-source Java application to handle downloads from various sites, including direct HTTP downloads. It handles page wait timers and presents the captchas to me when I need to solve one.

It has a slick GUI, but I have it running headless (no GUI) on my server machine and interface with it via JDownloader's web interface or the Android app, meaning I can initiate downloads from any browser or even my phone.

It still doesn't work perfectly with some sites, but it has certainly worked well for me overall. In my opinion the Android app has an even better interface than the browser site.

Answer 2

wget has options to save and load cookies. Sometimes depending on the site if you use wget to load the webpage that the link is on and tell it to save cookies, then make another request for the actual file and have it load the cookies it will work.

It's a two step process like this:

wget --cookies=on --keep-session-cookies --save-cookies=cookie.txt http://domain.com/page_with_link.html
wget --referer=http://domain.com/page_with_link.html --cookies=on --load-cookies=cookie.txt --keep-session-cookies --save-cookies=cookie.txt http://domain.com/some_file_to_download.zip

Answer 3

Another possibility (apart from maybe custom modifying ziproxy to add a "save these types locally" option) would be to use a X server.

There are X servers for smartphones, and for Windows too. This answer hints at HTTP client X servers.

So on your typical Android smartphone you would install Android X server, and on the Linux box a simple Firefox browser theme for small displays. You now have a local screen on the remote process, and any download will happen remotely. Very similar to VNC, but friendlier to smartphones.

On large screen devices you'd still fire up X server, but this time launching a normal Firefox instance.

Answer 4

You have the following options:

• run a virtual machine on the remote server, install your favourite OS to the VM, and run your favourite browser through the graphical console of the VM, accessed via the remote desktop protocol supported by your VM software (for example, VirtualBox can be accessed with rdesktop), optionally tunneled through SSH,
• run your favourite browser natively on the server, via remote X protocol, optionally tunneled through SSH (it's wise to use the compression option of SSH as well),
• run a text browser (for example: lynx) through SSH (optionally in screen),
• run a dumb HTTP client (for example: wget, curl) through SSH (optionally in screen),
• run an HTTP proxy (for example: squid) with non-standard settings: you'd probably have to increase the maximum allowed file size to be stored on disk, and it's practical to decrease the abort limit (the point at which the proxy will finish downloading even if the client has aborted, so that you can fire up the downloading, and then stop the client early, and let the proxy server finish the job),
• write your own HTTP client :)

Answer 5

Navigate to the address and download the file with Links over a SSH connection. A GNU Screen could also be used to finish the download without having the SSH connection still open.

Answer 6

offhand the only software for downloading off one click hosters that you can run on a linux cli that i remember is ploughshare - its pretty much a dead simple, and handles loads of file hosts. Don't forget to run it in screen or tmux so you can detach the session rather than needing to keep the ssh session open

If you didn't mind a GUI i'd suggest looking at slimrat

Answer 7

Install owncloud server (https://owncloud.org/) on your linux box, and owncloud clients on your devices (smartphone, remote pc, whatever). Config, authenticate, and Voila! you can now save downloads to your owncloud drive...