18

I am working with a website that sends API requests. I would like to write a client to make the requests myself, but in order to do so I would need to first see the request payloads. However, the connection is secured and therefore I can't see the data in wireshark just like that.

I found out that Wireshark supports SSL decryption: http://wiki.wireshark.org/SSL

However, it doesn't explain how the private key file should be obtained or generated, so I went and looked around and found this blog entry: http://blog.stalkr.net/2010/03/codegate-decrypting-https-ssl-rsa-768.html

After finding the appropriate packets as shown in the images, I exported the certificate, but unlike the challenge they were doing, the connection established for me uses RSA-2048 and does not provide the factorization (I'm assuming real certificates do not provide that, only the ones for games and such).

Is it possible for me to decrypt HTTPS packets that are sent to 3rd party websites? How would I generate the key file required?

MxLDevs
  • 313
  • 1
  • 2
  • 8
  • 4
    As an alternative to decrypting the HTTPS traffic, you could potentially use a local proxy such as Fiddler or Burp Suite to achieve the same goal. Essentially you add a man-in-the-middle in your HTTPS connection. – itscooper Aug 06 '14 at 23:31
  • That sounds like an interesting technique to know. I'll check it out. – MxLDevs Aug 07 '14 at 01:04
  • 2
    If you only need to monitor your browser's requests you can use its built-in developer tools or an add-on such as Firebug to look at the requests. No SSL decryption required. –  Aug 14 '14 at 11:38

5 Answers5

14

Wireshark is a very powerful tool. In most cases, the (addon-less) debug consoles of the browsers firefox and chrome should be enough. Both have network monitors that are sufficient most time. Be aware that the firefox' monitor doesn't support websockets yet.

If you still wanted to use wireshark, then consider utilizing the SSLKEYLOGFILE file, more help at this and this, and your linked wireshark wiki page.

user10008
  • 4,315
  • 21
  • 33
  • 1
    Currently I am using chrome inspector, which sort of does the job, but the service sends binary-serialized data which chrome does not display properly, so I'm unable to reproduce the request bodies myself. I only thought of using wireshark because I've worked with the same protocol before but the connection was not secured and so it was easy to just read it off wireshark. – MxLDevs Aug 07 '14 at 01:00
  • OK, then you are not one of the "most cases" :-) – user10008 Aug 07 '14 at 18:57
  • 2
    I've selected this as the accepted answer because supplying the pre-master key list files obtained via SSLKEYLOGFILE variable did the trick. – MxLDevs Aug 07 '14 at 20:49
11

The private key is private to the webserver. If you don't control the webserver you shouldn't be able to obtain it. The certificate only holds the public key so it wouldn't be of much use to you. You could try to setup a proxy https server and do a man-in-the-middle attack - in that case you would have the key of your proxy server.

Are you trying to crack the protocol from a software or you are normally using a browser to access the service?

In the second case you can use the temporary for the session by configuring Firefox to log the key to a file, as it was suggested by user10008: look for SSLKEYLOGFILE in the wiki page you linked, and the links that user10008 posted.

pqnet
  • 297
  • 1
  • 5
  • yes, currently I am accessing it through a browser, but it would be save so much more time if I could just simulate the API calls. – MxLDevs Aug 07 '14 at 00:58
  • your best chance is to try and observe the API calls through the javascript debug tools, instead of trying to use wireshark or similar. Which browser are you using? – pqnet Aug 07 '14 at 01:00
  • Chrome. The API calls are simple enough, except it is binary-serialized, which the inspector cannot render properly. Do you know if there are any extensions that might be able to display request payloads as raw bytes? Then that would be straightforward to work with. – MxLDevs Aug 07 '14 at 01:02
  • as far as I know, the chrome inspector can show typed arrays too. Do you have any experience in javascript? – pqnet Aug 07 '14 at 01:08
  • Yes, but perhaps not with debugging. The service I am using is built on flash and uses the AMF serialization methods, which chrome does not seem to know how to render. Would javascript debug methods be useful here? I am not certain whether I would be able to break on, say, XHR requests and hope to find values in memory. – MxLDevs Aug 07 '14 at 02:02
  • Burp Suite (a local proxy) supports AMF - so can deserialise requests/responses as they are intercepted: http://releases.portswigger.net/2009/08/v1214.html – itscooper Aug 07 '14 at 09:22
  • @MxyL oh well, that makes sense. I don't know much about flash, not sure whether it uses javascript at all for http requests. – pqnet Aug 07 '14 at 10:37
  • @itscooper I am not sure if he can use a proxy because of the cryptography problem. If you used it and can share the method please put it in an answer – pqnet Aug 07 '14 at 10:41
  • @pqnet It doesn't answer the question he asked (so wouldn't be useful to others looking for the answer to that question), but it would solve MxyL's underlying problem, hence commenting. You technically just end up with two HTTPS connections. Browser --> Local Proxy --> Website. You need to trust the local proxy's CA naturally. – itscooper Aug 07 '14 at 10:48
  • @itscooper yes indeed it is not the asked question. I would argue that he didn't ask the right question because the answer to the one he asked is a plain "no, you can't get the private key if you aren't friend with the server", when the problem he want to solve is about sniffing https traffic on a flash webapp, which is also probably not on-topic on this website. But since you propose to workaround the lack of the private key with a man-in-the-middle ssl proxy this is related to both this question and this website I think. – pqnet Aug 07 '14 at 11:07
  • @pqnet, the problem I have involves a flash webapp, but what I want to do is to be able to read the request and/or response packets that are encrypted. My assumption is that if my browser can read it, then I must therefore have some way to read it as well, but maybe that's not how it works. I've updated the question title, but not sure if me being allowed to connect to a remote server makes me a "friend" or not. – MxLDevs Aug 07 '14 at 18:50
  • Currently trying out the SSLKEYLOGFILE. It looks promising, since the blog that @itscooper links makes it look pretty easy!\ – MxLDevs Aug 07 '14 at 18:59
  • 1
    @MxyL There are two kinds of "private keys": one connection-private known by your browser and the server only used for symmetric encryption, and another private certificate key only known by the server, used for assymetric encryption. For the first, you are a "friend" by opening the connection, the second you are none. No one should be, as then that person could read (or, with PFS, MITM) all HTTPS traffic the guy has. The SSLKEYLOGFILE transports the connection-private key from your browser to wireshark. – user10008 Aug 07 '14 at 19:05
  • @user10008 In my case the connection-private key worked, but I guess this could easily be defeated if asymmetric encryption was used. – MxLDevs Aug 08 '14 at 15:36
  • 1
    @MxyL Nothing to worry, you always get the content. Both keys are used on every connection, this is called hybrid encryption. This is part of TLS, see the second sentence of the [wikipedia page](https://en.wikipedia.org/wiki/Transport_Layer_Security), and [thats](http://security.stackexchange.com/questions/3657/symmetric-encryption-session-keys-in-ssl-tls) why. – user10008 Aug 08 '14 at 17:10
8

You can't, unless you have administrative control over the 3rd party web server, or retrieve the certificate via some other nefarious means. SSL/TLS is reliant upon the private certificate staying private.

Furthermore, even if you had the server's private key, you might not be able to decrypt traffic from an earlier session if Perfect Forward Secrecy was used. In that case you would have to know the specific private key used for your single session.

In principle, because you are the client, you are privy to the pre_master_secret which is what you need to derive the master_secret. The master_secret is the symmetric key that's actually used to encrypt your session. As @pqnet said, there are ways to utilise this in FireFox and Chrome, although it is dependant upon the client software, and I'm not sure about Flash. However, there are more trivial methods to inspect your own HTTPS traffic...

Local Proxies

Since it is your own connection, there is no reason why you can't pass the website through a proxy server. Many proxy servers are configured to allow SSL-pass-through, which still gives you end-to-end encryption, but you can break this by terminating your connection at the proxy server (if you trust the proxy's SSL certificate). The proxy will then establish it's own SSL connection to the 3rd party website, passing along any traffic you send. Companies sometimes use the same method to inspect their users' outgoing HTTPS traffic (that's a contentious one). Essentially it's a man-in-the-middle.

Browser <---> Local Proxy <---> Website
          ^                 ^
        HTTPS             HTTPS

Local proxies run on your own machine, and allow you to inspect and even modify traffic that passes through them. If an attacker tried to do this, the user would see a browser warning to show that it's not received a valid certificate for the requested website. However, you can setup your machine to trust the local proxy's CA certificate.

Two popular local proxies that work with HTTPS traffic are:

Burp Suite has AMF serialisation support built-in. Fiddler looks like it has an extension.

itscooper
  • 2,230
  • 13
  • 15
  • The master secret is not the symmetric key used to encrypt messages. That's the session key ('write key' in RFC parlance). – user207421 Mar 21 '17 at 04:29
2

SSL was built to prevent you from doing exactly what you're attempting. You need the key to get access to private communications, with or without Wireshark.

Several answers suggest a man-in-the-middle attack, which should work with a lot of effort... but if this is a public API, it's probably documented. Particularly, if it is a web service API, you can get the WSDL from the browser by appending the endpoint URL with ?WSDL. This will give a complete list of methods and return types.

Mashmagar
  • 121
  • 2
2

Adding to itscooper's message, you can also use Charles Proxy with a trusted certificate installed on the device/browser and allow Charles to decrypt SSL so you can read the traffic. I do this quite a lot when testing devices that are communicating over SSL.

http://www.charlesproxy.com/documentation/proxying/ssl-proxying/

JasonG
  • 121
  • 2