Why is wget hanging?

Question

wget to a single specific url from one of my servers keeps getting timeouts. All other urls from this box work fine. This url works OK from any other boxes I have. Here's the output:

wget -T 10 http://www.fcc-fac.ca
--2011-07-14 14:44:29--  http://www.fcc-fac.ca/
Resolving www.fcc-fac.ca... 65.87.238.35, 207.195.108.140
Connecting to www.fcc-fac.ca|65.87.238.35|:80... failed: Connection timed out.
Connecting to www.fcc-fac.ca|207.195.108.140|:80... failed: Connection timed out

Can you tell me what might be wrong and how can I troubleshoot it? I'm using Ubuntu 11.04 (GNU/Linux 2.6.38-8-server x86_64)

Thank you very much in advance and forgive my noobish ignorance :)

ping, telnet, nc www.fcc-fac.ca 80 - all hang. However, some other urls that are easily wget'able though only some of their hosts are pingable.

traceroute doesn't tell me much:

7  rx0nr-access-communications.wp.bigpipeinc.com (66.244.208.10)  148.834 ms  149.018 ms  148.940 ms
8  sw-1-research.accesscomm.ca (24.72.3.9)  158.901 ms  159.805 ms  160.162 ms
9  65.87.238.126 (65.87.238.126)  150.069 ms  148.861 ms  148.846 ms
10  * * *
...
30  * * *

Thanks a lot for answers!

score 21 · Accepted Answer · answered Jul 14 '11 at 14:43

I think that the problem is that wget doesnt handle well IPv6 addresses and the DNS server is sending a IPv6 for that site. Sorry if I misunderstood your question. Check those tests:

hmontoliu@ulises:~$ wget -T10 http://www.fcc-fac.ca
--2011-07-14 16:44:34--  http://www.fcc-fac.ca/
Resolving www.fcc-fac.ca... failed: Connection timed out.
wget: unable to resolve host address `www.fcc-fac.ca'

If I force IPv6 because I believe that your problem is related to it, it fails:

hmontoliu@ulises:~$ wget -6 http://www.fcc-fac.ca
--2011-07-14 16:40:44--  http://www.fcc-fac.ca/
Resolving www.fcc-fac.ca... failed: No address associated with hostname.
wget: unable to resolve host address `www.fcc-fac.ca'

However if I force to use IPv4 it downloads right the index page

hmontoliu@ulises:~$ wget -4 http://www.fcc-fac.ca
--2011-07-14 16:40:56--  http://www.fcc-fac.ca/
Resolving www.fcc-fac.ca... 65.87.238.35, 207.195.108.140
Connecting to www.fcc-fac.ca|65.87.238.35|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 6554 (6,4K) [text/html]
Saving to: `index.html'

He resolves it just fine with an IPv4 address. – Jodie C Jul 14 '11 at 23:39 — Jodie C, Jul 14 '11 at 23:39
Another example of why I dislike IPv6 :-/ – PJ Brunet Aug 04 '19 at 16:44 — PJ Brunet, Aug 04 '19 at 16:44

score 1 · Answer 2 · answered Jul 14 '11 at 12:51

1

Run nc www.fcc-fac.ca 80, type in GET / and press Enter twice.

The remote sight might be down or blocking you.

answered Jul 14 '11 at 12:51

Jodie C

733
6
9

Hey, thanks for the answer - I updated the original question – Szczepan Jul 14 '11 at 13:10
If you can't netcat to the address then you are probably being blocked somewhere along the way. – Jodie C Jul 14 '11 at 23:40

score 1 · Answer 3 · answered Jul 14 '11 at 12:53

1

it seems wget isn't able to connect to your ip on port 80.

check if your server is pingable:

ping 65.87.238.35
ping 207.195.108.140

if the ip is pingable, check if u can connect to port 80 using telnet:

telnet 65.87.238.35 80

if the server does not respond to ping, it is probably down or unreachable for your source net.

test with traceroute tool to see where the connection fails.

answered Jul 14 '11 at 12:53

Goez

1,788
1
10
15

Hey, thanks for the answer - I updated the original question – Szczepan Jul 14 '11 at 13:10
a dig on www.fcc-fac.ca tells me that it has 2 DNS records of type A: ;; ANSWER SECTION: www.fcc-fac.ca. 120 IN A 65.87.238.35 www.fcc-fac.ca. 120 IN A 207.195.108.140 if both are unreachable the host is down, and naturally wget will not be able to retrieve the file u want. if there are some ip's reachable, wget will skip the down ip untill it comes across a ip that it can reach, and download the file from there. in your case, both hosts seem down due to the long response time I think (time-out). – Goez Jul 14 '11 at 13:12
u can use the -T option in wget to extend the timeout value The timeout is sec to seconds, so maybe a higher value will help – Goez Jul 14 '11 at 13:16
Ok. I see. Both ips are not pingeable (I checked from 2 different machines). Yet the link works fine in the browser and when wget'ed from different machines than the one that makes trouble :/ – Szczepan Jul 14 '11 at 13:21
Higher timeout does not help – Szczepan Jul 14 '11 at 13:21
are u able to load the site in tools like f.e.: lynx? – Goez Jul 14 '11 at 13:30
lynx hangs at "Making HTTP connection to ... " – Szczepan Jul 14 '11 at 13:43
I'm afraid for some reason your server is not allowed to make connections to the destination. if u really need to connect from that server u can try ssh forwarding. – Goez Jul 14 '11 at 13:55

score 1 · Answer 4 · answered Sep 05 '11 at 11:39

Fix for slow DNS resolution in PHP scripts using CURL library.

From the libcurl documentation:

CURLOPT_IPRESOLVE

Allows an application to select what kind of IP addresses to use when resolving host names. This is only interesting when using host names that resolve addresses using more than one version of IP. The allowed values are:

CURL_IPRESOLVE_WHATEVER

Default, resolves addresses to all IP versions that your system allows.

CURL_IPRESOLVE_V4

Resolve to IPv4 addresses.

CURL_IPRESOLVE_V6

Resolve to IPv6 addresses.

I believe these can be set as environmental variables. At least while using PHP, these settings made a huge difference to resolver speed.

Why is wget hanging?

4 Answers4