4
I am behind a very capricious proxy (squid) and it seems only http requests made with a particular user-agent are allowed.
When fetching google.com from Google Chrome, I can see a HTTP GET on WireSharc. However, wget tries to resolve the DNS first through the default DNS servers which never answer.
How can I make wget or even git use the proxy instead of a direct DNS query?
nowox
Posted 2016-03-11T10:37:15.080
Reputation: 1 779
2
How can I make wget or even git use the proxy instead of a direct DNS query?
Set the proxy environment variable and it will do this straight up:
export http_proxy='http://proxy.contoso.com:3128/'
wget http://www.example.net/
roaima
Posted 2016-03-11T10:37:15.080
Reputation: 1 006
I already did it. From chrome I get a HTTP REQUEST and from wget I still get a TCP request to the DNS. So Chrome is just asking the proxy and wget tries to resolve the DNS first. – nowox – 2016-03-11T13:46:48.527
@nowox I don't get any DNS lookup from wget in this configuration. What platform (OS) are you using? – roaima – 2016-03-11T15:49:34.073
1
there is no mechanism to specify that the proxy should try a particular IP address for a particular host. You could change the URL from, say, http://example.com/mypage to http://33.33.33.33/mypage, but then the proxy server won't know the hostname to request. The modern web (HTTP/1.1) depends on a Host header always being present in a request, allowing one web server to serve multiple sites identified by their hostname.
Your best bet would be to configure the proxy server to use the DNS resolver of your choice. This would only be possible if you use a local proxy server that you control.
Elizabeth Anderson
Posted 2016-03-11T10:37:15.080
Reputation: 101
So how is it possible that Chrome does not make a query to the DNS itself? – nowox – 2016-03-11T13:47:32.700
If you're seeing that DNS requests are never answered, why do you think it's the fault of the proxy? Please [edit] your question to clarify. – roaima – 2016-03-11T11:33:32.180
1@roaima I never said it is the proxy's fault. I said wget is not doing the name resolution through the proxy. – nowox – 2016-03-11T11:48:51.533
Please could you clarify the relevance of "it seems only http requests made with a particular user-agent are allowed" to the DNS lookup issue you're describing. I don't see the connection. Thanks. – roaima – 2016-03-11T13:45:18.790
if wget is still doing a DNS lookup for the target site (not the proxy) then it's not set to use a proxy. You could resolve the lookup problem with a hosts file entry, but that will just push the problem to the next stage, if it doesn't make a proper proxy request to the proxy, it will be rejected then. – Adrien – 2016-07-22T23:43:54.730