6

i'm having an issue when trying to run devstack which is trying to get pip using the following command i added the verbose parameter

sudo -H -E python /opt/stack/devstack/files/get-pip.py -c /opt/stack/devstack/tools/cap-pip.txt --verbose

i get this output, and it just hangs there forever then tries four more times and quits.

Created temporary directory: /tmp/pip-ephem-wheel-cache-WNf2he
Created temporary directory: /tmp/pip-req-tracker-YNDzMb
Created requirements tracker '/tmp/pip-req-tracker-YNDzMb'
Created temporary directory: /tmp/pip-install-qbbm3e
Setting pip!=8,<10 (from -c /opt/stack/devstack/tools/cap-pip.txt (line 1)) extras to: ()
Collecting pip!=8,<10 (from -c /opt/stack/devstack/tools/cap-pip.txt (line 1))
  1 location(s) to search for versions of pip:
  * https://pypi.org/simple/pip/
  Getting page https://pypi.org/simple/pip/
  Looking up "https://pypi.org/simple/pip/" in the cache
  No cache entry available
  Starting new HTTPS connection (1): pypi.org:443

My machine is sitting behind a proxy, that uses a white list to allow some domain names, these domains are allowed.

# python
.pypa.io
.python.org
.pypi.org
.pythonhosted.org

and also we use our own certificate

i can get the page with a curl curl -v pypi.org:443 but this is the output i get with the verbose mode.

* Rebuilt URL to: www.pypi.org:443/
*   Trying X.X.X.X...
* Connected to X.X.X.X (X.X.X.X) port XXYY (#0)
> GET http://www.pypi.org:443/ HTTP/1.1
> Host: www.pypi.org:443
> User-Agent: curl/7.47.0
> Accept: */*
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 503 Service Unavailable
< Server: squid/3.5.12
< Mime-Version: 1.0
< Date: Fri, 24 Aug 2018 15:55:36 GMT
< Content-Type: text/html;charset=utf-8
< Content-Length: 3601
< X-Squid-Error: ERR_CONNECT_FAIL 101
< Vary: Accept-Language
< Content-Language: en
< X-Cache: MISS from dns
< X-Cache-Lookup: MISS from dns:XXYY
< Via: 1.1 dns (squid/3.5.12)
< Connection: close
< 
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html><head>
...
</body></html>
* Closing connection 0

but when i run an openssl i get nothing sudo openssl s_client -connect pypi.org:443

also when checking the squid access logs i can see a TCP_MISS

1535126136.880    137 X.X.X.X TCP_MISS/503 3966 GET http://www.pypi.org:443/ - HIER_DIRECT/2a04:4e42:2c::319 text/html

Please any help is appreciated.

this thing is driving me nuts.

Thank you.

update to use curl with https instead of http

when running the command curl -v https://pypi.org:443 this is the output i get

* Rebuilt URL to: https://pypi.org:443/
*   Trying X.X.X.X...
* Connected to X.X.X.X (X.X.X.X) port XXYY (#0)
* Establish HTTP proxy tunnel to pypi.org:443
> CONNECT pypi.org:443 HTTP/1.1
> Host: pypi.org:443
> User-Agent: curl/7.47.0
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 200 Connection established
< 
* Proxy replied OK to CONNECT request
* found 149 certificates in /etc/ssl/certs/ca-certificates.crt
* found 601 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* SSL connection using TLS1.2 / RSA_AES_128_GCM_SHA256
*    server certificate verification SKIPPED
*    server certificate status verification SKIPPED
*    common name: pypi.org (matched)
*    server certificate expiration date OK
*    server certificate activation date OK
*    certificate public key: RSA
*    certificate version: #1
*    subject: CN=pypi.org
*    start date: Tue, 15 May 2018 17:28:09 GMT
*    expire date: Fri, 12 May 2028 17:28:09 GMT
*    issuer: ******** info about the certificate issuer *********
*    compression: NULL
* ALPN, server did not agree to a protocol
> GET / HTTP/1.1
> Host: pypi.org
> User-Agent: curl/7.47.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Content-Security-Policy: base-uri 'self'; block-all-mixed-content; connect-src 'self' https://api.github.com/repos/ *.fastly-insights.com sentry.io https://2p66nmmycsj3.statuspage.io; default-src 'none'; font-src 'self' fonts.gstatic.com; form-action 'self'; frame-ancestors 'none'; frame-src 'none'; img-src 'self' https://warehouse-camo.cmh1.psfhosted.org/ www.google-analytics.com *.fastly-insights.com; script-src 'self' www.googletagmanager.com www.google-analytics.com *.fastly-insights.com https://cdn.ravenjs.com; style-src 'self' fonts.googleapis.com; worker-src *.fastly-insights.com
< Content-Type: text/html; charset=UTF-8
< ETag: "5fFqoiKHaa7+ibgDS4xg6Q"
< Referrer-Policy: origin-when-cross-origin
< Server: nginx/1.13.9
< Content-Length: 17391
< Accept-Ranges: bytes
< Date: Fri, 24 Aug 2018 16:32:00 GMT
< Age: 17
< X-Served-By: cache-iad2147-IAD
< X-Cache: HIT
< X-Cache-Hits: 1
< X-Timer: S1535128321.765578,VS0,VE1
< Vary: Accept-Encoding, Accept-Encoding
< Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
< X-Frame-Options: deny
< X-XSS-Protection: 1; mode=block
< X-Content-Type-Options: nosniff
< X-Permitted-Cross-Domain-Policies: none
< X-Cache: MISS from dns
< X-Cache-Lookup: MISS from dns:3131
< Via: 1.1 dns (squid/3.5.12)
< Connection: close
< 
<!DOCTYPE html>
<html lang="en">
<head>
...
</body></html>
* Closing connection 0

and this is the squid access log 

1535128320.770    105 129.6.112.80 TCP_MISS/200 18684 GET https://pypi.org/ - HIER_DIRECT/151.101.128.223 text/html
Mheni
  • 199
  • 1
  • 2
  • 8
  • 1
    `curl -v pypi.org:443` makes no sense since this means HTTP (not HTTPS) connect to pypi.org - which results in the error message of the proxy. Try `curl -v https://pypi.org` or `curl -v https://pypi.org:443` - the last one being the same as the first since 443 is the default port for https. `openssl s_client -connect pypi.org:443` instead is doing a direct access with TLS (HTTPS), not involving the proxy you've used with curl. – Steffen Ullrich Aug 24 '18 at 16:30
  • hi @SteffenUllrich thanks for the reply, i updated the post with the output i get for https. curl works fine but not openssl or the python command. just scroll down for the update – Mheni Aug 24 '18 at 16:43

1 Answers1

5

It looks like you need to use a proxy but don't setup the proxy with get-pip.py. That's why it fails. According to the documentation get-pip.py should be given the proxy in the following way:

python get-pip.py --proxy="http://[user:passwd@]proxy.server:port"

Taking the verbose output from your experiments with curl as the base you would need to use it like this:

python get-pip.py --proxy="http://X.X.X.X:XXYY"

Or, with your specific command line you've used so far:

sudo -H -E python /opt/stack/devstack/files/get-pip.py \
    --proxy="http://X.X.X.X:XXYY" \
    -c /opt/stack/devstack/tools/cap-pip.txt --verbose
Steffen Ullrich
  • 12,227
  • 24
  • 37
  • thank you this sort of worked, the traffic is now going through the proxy, but im getting a new error suggesting that the certificate is not valid. `Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)'),)': /simple/pip/` do i need to get a certificat from `pypi.org` ? – Mheni Aug 24 '18 at 18:13
  • @Mheni: Different problem. Most likely you don't have the relevant CA installed. There are [several similar questions with answers](https://www.google.com/search?q=CERTIFICATE_VERIFY_FAILED+%27get-pip.py%27+site%3Astackoverflow.com) you can follow. – Steffen Ullrich Aug 24 '18 at 19:10