16

We are using Squid proxy server in our environment and we want to cache HTTPS requests.

Is there any way to configure Squid or in general a proxy server to cache HTTPS requests?

Supratik
  • 2,104
  • 10
  • 49
  • 66

6 Answers6

14

There is a way to do it, but it's fundamentally against the reasons for using HTTPS.

Here's how you'd do it.

  1. Generate a self-signed SSL Certificate for the site you'd like to intercept and cache the requests from.
  2. Install and run stunnel on your proxy server, telling it that the certificate it should present is the one generated in stage 1.
  3. Have stunnel forward the decrypted requests to squid.
  4. You might need to have stunnel on the other side, or openssl_client to re-encrypt the request to the upstream server.

Caveats:

  1. Your users will hate you. Every SSL request to that site will present an invalid certificate window.
  2. You're exposing yourself to potential lawsuits for doing naughty things. (IANAL)
  3. You'll only ever be able to get a self-signed certificate working for this, because of how the PKI web of trust for SSL Certificates is supposed to work. Saying nothing about compromised root CAs.

I'm not going to give you the exact details of how to do this, because a) I think it's somewhat unethical, and b) It's better for you to learn how to do it.

I suggest you research how stunnel and man-in-the-middle attacks work.

Tom O'Connor
  • 27,440
  • 10
  • 72
  • 148
  • 3
    Actually, if you're on a domain, it's far easier to generate your own CA, and deploy the public certificates for it with Group Policy. – Tom O'Connor Oct 11 '13 at 14:03
  • 1
    Trusting a SSC + MITM is useful for protocol debugging, caching, deep packet inspection and censorship/logging. :/ Other than those reasons, not so good. –  Sep 13 '17 at 22:03
8

Just to explain why this can't be done without MITM - a proxy only sees the DNS name of the server you want to connect to when using encrypted HTTPS. It does not see the URL, nor any response headers. It cannot determine which individual resource you are accessing on a site, whether or not it is cacheable, nor what it's modification times are. All it can see is someone wants something from a remote server using HTTPS.

This means that caching cannot work as the proxy does not know what cached objects to give you, or how to get them in the first place.

robf
  • 145
  • 1
  • 5
5

No, there are not: they are encrypted... A workaround would be something like a man-in-middle deployment, but that would defeat all the reasons behind https.

yrk
  • 2,347
  • 16
  • 22
  • 1
    Are there no workaround to achieve this or force the proxy server to decrypt and cache? – Supratik Jan 18 '12 at 10:22
  • A workaround would be somewhat which resempbles `man-in-middle` deployment, but that would defeat all the reasons behind https – yrk Jan 18 '12 at 10:35
  • 6
    I disagree that it would defeat all the reasons behind https. If you do this at home and you own the proxy, your data will still use https between your proxy and the web sites. – bbigras Feb 15 '16 at 19:54
  • @brunoqc that's a VPN's job. – yrk May 11 '16 at 18:58
  • 1
    If caching https payloads is important for some reason or debugging an https session, MITM is super useful. In fact, this is how Charles works. –  Sep 13 '17 at 22:04
  • @yarek, provided that there is a reason behind that https. Now it feels more like that https have been abusively used in many senarios. – Ben Jun 10 '19 at 15:13
2

Squid has a "hack" for just that:

https://wiki.squid-cache.org/Features/SslPeekAndSplice

Have fun. :)

user1050755
  • 121
  • 1
1

Zeus (Now Riverbed's) ZTM Traffic Manager can do this as it can translate http and https traffic both ways and cache unencrypted content - it works, we use it, but it's fearsomely expensive - as in the price of a Porsche per server.

Chopper3
  • 100,240
  • 9
  • 106
  • 238
  • 6
    But it still requires you to install a new root cert on the client, doesn't it? And you still have to trust the proxy, don't you? – phihag Jan 18 '12 at 15:37
0

No, there is no Squid configuration to achieve this in a clean way (it must be impossible by design of HTTPS).

But yes, there is a browser configuration (HTTPS Proxy) enabling the desired behavior. Like this, the clients can make the difference between a legit proxy and a "man in the middle" attack.

Your users must be aware and agree that you are intercepting HTTPS traffic. If they are collaborators of a company, the HTTPS Proxy can be set by a policy. Have the users sign an internet usage agreement stating that you are going to intercept HTTPS traffic.

If you are an internet service/café provider, incentivize your users to activate a legit setting more or less manually, especially for visiting social media with video. Your users probably want to opt out for doing e-banking and should be able to do so.

stonux
  • 1