1
1
I'm having a good deal of trouble getting curl or wget to download files from FTP servers through an FTP proxy. I have setup GUI FTP programs to work...
gFTP works with these settings:
I get output like below when just using curl/wget with the ftp_proxy variable set to the proxy hostname:
# curl -v ftp://ftp.astron.com/pub/file/file-5.05.tar.gz
* About to connect() to proxy blah port 21 (#0)
* Trying blah... connected
* Connected to blah (blah) port 21 (#0)
> GET ftp://ftp.astron.com/pub/file/file-5.05.tar.gz HTTP/1.1
> User-Agent: curl/7.19.0 (x86_64-suse-linux-gnu) libcurl/7.19.0 OpenSSL/0.9.8h zlib/1.2.3 libidn/1.10
> Host: ftp.astron.com:21
> Pragma: no-cache
> Accept: */*
> Proxy-Connection: Keep-Alive
>
220-
220-Enter an Internet ftp address at the Name prompt.
220 Type help for usage information.
500 Syntax error, command unrecognized.
500 Syntax error, command unrecognized.
500 Syntax error, command unrecognized.
Is there some way I can configure the shell environment and/or curl/wget with the same settings as gFTP for passing through an FTP proxy?
entropo
Posted 2011-03-30T21:02:50.833
Reputation: 655
1
Both curl and wget support FTP proxies... wget: http://www.gnu.org/software/wget/manual/wget.html#Proxies ... curl: http://curl.haxx.se/docs/manual.html
– entropo – 2011-03-30T21:11:15.713Ah no - they support proxies for FTP transfers using HTTP proxies - not FTP native proxies. The protocols are different. The "GET ftp://blah.blah HTTP/1.1" in your debug output is a dead giveaway - that's an HTTP proxy request that is being generated. – Majenko – 2011-03-30T21:15:26.740
Hmm, okay. Are there any command line tools which DO support native FTP proxying? – entropo – 2011-03-30T21:39:01.533
1You may not need them - the way the FTP proxy works you may be able to slip the details into the URL... f t p://ftp@remote.host.com@ftp.proxy.com/path/to/file without any proxy set might possibly work. It's worth a try anyway – Majenko – 2011-03-30T21:40:51.457
1If that doesn't work, try missing out the ftp@ or maybe changing the first @ to %40 – Majenko – 2011-03-30T21:42:53.890
Aha! Victory, thank you. That's stupidly painful, but here's my regex for reworking the URLs: perl -pe 's/(ftp://)(\S+?)/(\S+)$/$1ftp%40$2@proxy.blah.com/$3/' – entropo – 2011-03-30T22:17:16.960
For further reference, the "Types of firewalls" list here is useful: http://linuxnet.ch/groups/linuxnet/wiki/a6fa7/FTP_via_Proxy.html
– entropo – 2011-03-30T22:21:53.323Also, I found there is a feature request filed for wget to support FTP proxy auth: https://savannah.gnu.org/bugs/index.php?21439
– entropo – 2011-03-30T22:23:26.460curl is usable with ftp proxy as per the curl manual. The following command worked for me with pure ftp server with a ftp proxy in between.
curl -u "anonymous@myftp.server.com :anonymous" ftp://myproxy:21/path/to/server-ftp-file– GuruM – 2013-08-06T13:05:48.033Extract from the manual:
Most FTP proxy servers are set up to appear as a normal FTP server from the client's perspective, with special commands to select the remote FTP server. curl supports the -u, -Q and --ftp-account options that can be used to set up transfers through many FTP proxies. For example, a file can be uploaded to a remote FTP server using a Blue Coat FTP proxy with the options:
curl -u "Remote-FTP-Username@remote.ftp.server Proxy-Username:Remote-Pass" --ftp-account Proxy-Password --upload-file local-file ftp://my-ftp.proxy.server:21/remote/upload/path– GuruM – 2013-08-06T13:06:42.333Using other options of curl I was still seeing the GET ... HTTP 1.1 in the verbose output. Even forcing curl with --proto="=ftp" didn't work. But the above command did. Hope this helps someone else. – GuruM – 2013-08-06T13:08:06.763