We are looking to monitor inbound HTTPS connections for performance issues as well as errors. HTTP requests we are able to gather this information just fine, but we don't have a way to do this while maintaining PFS via HTTPS. Our HTTPS decryption tool does not support DH ciphers at all, for instance.
What are people doing for this?
So you want to do some analysis on inbound HTTPS traffic. A common way is to use reverse proxies (commonly Apache or nginx) in front of the actual servers, install the server certificates on the reverse proxies and actually do the SSL/TLS decryption on the proxies. It gives:
client <-> Internet <-> [ router <-> { reverse proxies <-> actual servers } ]
| |
corporate security
network zone
limit limit
In this scenario, the reverse proxies shall be in the same security zone as the servers, that means that only admins should be able to spy them and the traffic between the proxies and the servers. So the traffic between the proxies and the server can safely be unencrypted (but should transport optionaly client certificates). As reverse proxies are normally simpler than application servers, it is easier to add filtering and traffic analyzis there.
This is a really interesting question because when crypto that uses Perfect Forward Secrecy , such as TLS 1.2 with DH is in play, then tools that try and perform man-in-the-middle to inspect traffic get broken.
There are some things that you can do. You should look at something called the a SSLKEYLOGFILE. The full tutorial is covered in really good depth here, decrypting TLS browser traffic with wireshark the easy way , but essentially, you can export an environment variable called SSLKEYLOGFILE and point it to a file. The next time Firefox or Chrome starts up, they will check for this variable, and if it exists, they will start writing all the CLIENT_RANDOM keys and DSA keys to the file as the user browses.
Wireshark can then take the SSLKEYLOGFILE and use it to decrypt the TLS packets and inspect the traffic.
You will notice that all of this TLS/SSL key writing happens on the client side. You could have all your clients start writing their keys to a file share, collect their traffic and inspect it. This would not be in real time, and it would not work for every program, but it does solve your problem of not being able to inspect traffic that uses
DH.
So there are a few different ways to do this. But some of the things you are asking for you need more of the picture to discern. We'll look at the one you're proposing: break-and-inspect.
Break-and-inspect requires we be able to crack open the TLS, and EC(DH) ciphers are designed to be resistant to MITM. There was a great post a while back that went over the limitations (Decrypting TLS in Wireshark when using DHE_RSA ciphersuites.) tl;dr, you need the pre-master keys from your application and a way to consume them.
Another way to break-and-inspect is to move the TLS termination point to a device you can instrument (a apache/nginx proxy of some sort.) If you have another device proxying requests on your behalf you could then inspect that traffic through the proxy's application.
Finally the way I would attempt to do this is: I would look to the webserver itself. Almost all of the information you mentioned (4XXs, URI logs, tcp session timeouts etc.) can be gleaned from webserver logs.
If we have to perform this activity without any break and inspect, or access to webserver logs, and I had no end of time, I might try and (https://www.scip.ch/en/?labs.20160317) implement a practical Bicycle attack since I have access to the unencrypted text for things like 404 pages etc. You wouldn't be able to get meaningful information about the errors, only confidence that there were errors etc. Any sort of network inspection will capture TCP/IP level disconnects etc but you still will be limited to what you can actually see.
