having with connecting to site from command line

Question

I am able to connect to following https://registry.terraform.io/.well-known/terraform.json using a browser

When I try to connect to it from command line

curl: (7) Failed to connect to registry.terraform.io port 443: Timed out

any ideas on how to resolve it thanks

Answer 1

issue was with incorrect settings of proxy on command window HTTPS_PROXY=proxy settings from browser

Answer 2

TL;DR

Try the curl command listed below in Step 7. If that doesn't work, you likely have another issue besides your curl command (such as a firewall or proxy).

Copying Browser Requests with cURL

If you can connect with a browser, one way to eliminate potential issues with your curl command is to "copy" the desired transaction from your browser. Chrome, Firefox, and Opera all have a Developer console which allows this.

1. Open an new blank tab.

2. In Chrome and Firefox, press F12 to open the Developer console. In Opera, press Ctrl + Shift + I.

3. Look for the Network option and select it.

4. Navigate to your desired URL e.g. https://registry.terraform.io/.well-known/terraform.json.

5. You should end up looking at something similar to the following:

   ex. Network Transactions

   

6. Select the desired transaction, right-click and select the option to Copy as cURL (for Chrome, use Copy as cURL (cmd)).

   ex. Copy as cURL

   

7. If you paste the contents of the clipboard, you should have an exact copy of the request for curl:

   curl "https://registry.terraform.io/.well-known/terraform.json" -H "authority: registry.terraform.io" -H "upgrade-insecure-requests: 1" -H "user-agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.132 Safari/537.36" -H "sec-fetch-mode: navigate" -H "accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3" -H "sec-fetch-site: none" -H "accept-encoding: gzip, deflate, br" -H "accept-language: en-US,en;q=0.9" --compressed
   

This curl command should replicate the request made by the browser exactly.

Other Issues

Assuming that this doesn't solve the problem, then you will have to look into other reasons why this request might be failing, such as firewall/proxy issues or certificate issues.

• Use the -k or --insecure option to ignore self-signed or invalid certificates with curl.

• Consider using the --verbose flag, as well as dumping HTTP headers with -D -, with curl to aid troubleshooting.

---

Emulating A Specific Browser

Based on your additional comment that your (generic) cURL requests are still failing, it's likely you will need to emulate the specific browser which can connect successfully.

Browsers identify themselves with a User Agent string associated with a given version of that browser. If you want to emulate the browser that can connect to your URL, you'll need to either:

1. Use the steps I give under Copying Browser Requests with cURL in the browser that can connect successfully.

2. Try replacing the user-agent value in Step 7 with the value returned from a site like What Is My Browser (visit this link in the browser which has no trouble accessing your link).

Example (Option 2)

"Generic" User Agent (above)

-H "user-agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.132 Safari/537.36"

Specific User Agent (e.g. from What Is My Browser)

-H "user-agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:54.0) Gecko/20100101 Firefox/54.0"

---

It's important to note that the first method (copying the request) is probably the most reliable, as it may catch headers other than the user-agent that are used to authorize a request.

---