cURL very long delay before starting download from ftp

0

I'm trying to write a script to download a file, convert it, then execute a program on the converted file.

For some reason curl takes over 1 minute to start the file transfer.

Command:

curl ftp://tgftp.nws.noaa.gov/SL.us008001/ST.expr/DF.gr2/DC.ndfd/AR.conus/VP.001/ds.wx.bin -o test.out

I've tried disabling active and EPRT modes with --ftp-pasv and --disable-eprt but that doesn't seem to make a difference.

I've tried --verbose and that tells me where but I still have no idea why.

It logs in quickly and changes to the right directory, but as soon as it tries to download the file, it just sits there for over a minute. Just before the delay here is the verbose output:

* FTP 0x1c2b268 state change from CWD to PASV
* Connect data stream passively
< 229 Entering Extended Passive Mode (|||64103|)

The funny thing is that wget starts downloading immediately. I know I can use that, but I'd like to figure out why curl is having problems.

Thanks in advance.

Shadow Man

Posted 2014-05-03T00:26:24.583

Reputation: 105

From here it is fast, so it might be related to your location. Maybe you have some kind of firewall/DPI device in between which does have problems with EPSV. Try to make curl use PASV instead with --disable-epsv. – Steffen Ullrich – 2014-05-03T06:16:26.757

I have no idea what we do for a firewall, but that did it. Thank you very much! If you'd care to put it in as an answer, I'd be happy to mark it correct. – Shadow Man – 2014-05-05T17:42:42.693

Answers

2

The problem was some kind of middlebox (firewall, deep inspection device etc) which produced problems with the EPSV command (passive mode for IPv6 and IPv4). Forcing PASV (passive mode IPv4 only) with --disable-epsv solved the problem. Similar problems could have been if the server had IPv6 and there were problems with IPv6 only, but in this case the server had no AAAA DNS record.

Steffen Ullrich

Posted 2014-05-03T00:26:24.583

Reputation: 3 897