Confusion about HTTPS proxies

Question

I know how HTTP proxies work, but I am unsure about HTTPS ones. More specifically, I am confused about the encryption part.

So, the client connects to the proxy using HTTPS. That I understand. Now say the user wants to connect to gmail through that proxy.

1. Does the proxy have to estabilish a HTTPS connection to gmail first?
2. And how is the client/server hello done? Does the client encrypt the client hello through https, send it to the proxy which in turn decrypts it, encrypts it with the encryption method agreed with gmail and sends it to gmail?
3. And then, after the client/server hello is executed and an encryption method is agreed upon, doesn't the client get confused? Because if the requests are all going through the proxy, shouldn't the encryption method be just one? Or does the client encrypt it's requests according to the host they are being sent to, regardless of ip address?
4. One more thing; if the proxy is supposed to replace the sender address of the packets with it's own address, then shouldn't the packets be completely decrypted? Because if they are not, then how is the proxy supposed to know where to insert it's IP if the data is all garbled? Or is the sender part not encrypted?

Answer 1

The comment 'There are no HTTPS forward proxies' is not true. They are very common. Any company of decent size will incorporate explicit forward proxies for SSL so that they can decrypt the https traffic for monitoring, logging and filtering purposes - a common euphemism being 'Compliance Reporting'.

Some background, which you may already know, for context:

The initial handshake of an HTTPS session contains three parts:

1. Greeting: ClientHello message containing what TLS versions and ciphers it supports, and a ServerHello message with similar info. This step and the following one are not encrypted.

2. Cert Exchange: Server provides it's TLS cert, which the client needs to decided if it trusts - typically implemented with implicit trust on those signed by certificate authorities and requiring user intervention for self-signed.

3. Key Exchange - The method decided on in step 1 is used; the simplest method is a RSA key exchange type transaction: Client generates a key using a random number and encrypts it using the server's public key. This is decrypted using the server's private key (it can only be encrypted with the public key, it cannot be decrypted without the private key, which is subtle but confusing to many).

From here on, entirely encrypted communications take place, with nearly all of the TCP/IP packet encrypted (except for headers/fields required for routing).

The answer to your questions is that a forward proxy requires that the proxy entirely decrypts the HTTPS communications. The client never handshakes directly with the intended server (sometimes without knowing that it isn't, which I touch on below).

Here is an explanation from Palo Alto Networks on how a Forward HTTPS Proxy works (PAN-OS refers to their network SW):

1. Client is trying to reach out www.google.com with HTTPS. Traffic is matching the decryption policy.
2. This traffic is handled by our SSL proxy engine, and a certificate for www.google.com is generated by our internal PKI (signed by the CA certificate).
3. PAN-OS is proxying the ssl traffic and setting up a new ssl connection with the Web Server.
4. Web Server is starting handshake with PAN-OS device.
5. PAN-OS device is completing its SSL handshake with client presenting generated certificate in Server Hello message.

This (if malicious) is an explicit man in the middle attack - it turns a 2 party transaction: PC ⟺ Google - into 2 separate transactions: PC ⟺ Proxy and Proxy ⟺ Google.

---

Side Note

As forward proxies are primarily used to monitor ssl traffic, it is interesting that the non-proxy method also discussed in that PAN link - inbound 'inspection / decryption' which requires that the firewall have the servers cert and keys, and is therefore able to entirely decrypt the communication with simple packet monitoring. As an aside, it's a bit unnerving that they advertise this as a key feature when one would hope it would have very few use cases - it requires that the operator of the firewall have the public and private keys of the 3rd party site - e.g. Google's private keys. One would think that wouldn't be common.

There are many different ways forward proxies can be implemented and I barely touch on them - many if not most forward proxies are implemented as 'transparent' forward proxies, meaning the clients are not configured to connect to a proxy at all, and the firewall / proxy interposes itself into the transaction. The forward proxy described above is a 'transparent' proxy. Once the client has accepted the cert from the proxy, which if it is a CA signed cert is usually automatic, the client machines will receive packets that the proxy modified to appear to be from the original server (e.g. google).

---

Update to pull in info from comments

1. Another method to implement a forward proxy uses TCP tunnelling, which effectively creates a wrapper around the encrypted data. The initial transactions here are not encrypted, and are initiated by the client using a CONNECT request to the proxy server. Once the connection is established, client to server communications are encrypted. See this for details.

2. HSTS (HTTP Strict Transport Security) is a protocol that was developed to prevent malicious attackers from intercepting requests from servers to transition the current transaction from http to https. The theory behind it is that it is trivial to implement a man in the middle attack on http, and this 'proxy' agent could then spoof the transition to https if it saw the transition request. It doesn't directly impact how https forward proxies work, but is related; here is a much better explanation

3. MITM attacks, HTTPS and proxy servers. Here's an excellent explanation on why MITM attacks using proxy servers are not currently effective when using 'straight' HTTPS (i.e. not a HTTP to HTTPS transition that is partially addressed with HSTS). Here is additional info on MITM attacks and HTTPS from a question on crypto.stackexchange.

I think that answers the original and updated questions. If I have anything wrong or unclear let me know and I'll update.

Answer 2

HTTPS via a proxy uses a command called "CONNECT". This is called tunneling, and uses an HTTP request message using the CONNECT verb to set up a relay between the client and the requested server:port. The proxy makes the connection, indicates the connection to the client (e.g. sends back 200 Connection OK) and then gets out of the way.

From this point, the client has a relayed connection to the end server, similar to if it had connected directly without a proxy. This is used to set up a SSL/TLS layer over which the subsequent encrypted http requests are made.

Answer 3

A HTTPS proxy server is nothing different from a HTTP proxy server, except that with HTTPS proxy, brower to proxy server connection is encrypted.

But you need to deploy a certificate on your proxy server, like how a https website does, and use a pac file to configure the brower to enable Connecting to a proxy over SSL.

The difference is for example, with https proxy server, a firewall can't not see which host you use CONNECT method connect to.

For more details and a practical exmaple, see my question and answer here https://stackoverflow.com/questions/56981993/https-proxy-server-only-works-in-switchomega