How do I know which padding was used in a cipher suite?

Question

I am trying to write a java program that decrypts tls packets. I want to use the Cipher class. From the Handshake I know that e.g. my CipherSuite is 0x00 0x2f which is TLS_RSA_WITH_AES_128_CBC_SHA. So when I call Cipher.getInstance(); I know to put in "AES/CBC/????" but what’s the padding?

I read in How does SSL/TLS work?, that:

...then some padding (depending on the encryption algorithm)...

And from Using Padding in Encryption on www.di-mgt.com.au I know there are at least 5 methods of padding.

So to my understanding there is more than one possibility for one encryption algorithm.

So how do I know which one was used? Or can I get that maybe out of the handshake?

Answer 1

Padding in TLS is almost the same as the PKCS#7 padding, but with some subtle differences.

When using CBC encryption, in TLS, things go the following way:

• The MAC is computed over the plaintext, and appended to the plaintext. In the case of TLS_RSA_WITH_AES_128_CBC_SHA, that's HMAC/SHA-1, so 20 extra bytes.

• What will be encrypted is the concatenation of the plaintext, the MAC and the padding. The total length must be a multiple of the block cipher size, i.e. 16 in the case of AES. The padding consists in appending n+1 bytes, such that 0 ≤ n ≤ 255. So there is at least 1 extra byte, at most 256, all have the same value, and that value is one less than the padding length.

  (In true PKCS#7 padding, there would be n+1 bytes who all have value n+1, not n.)

• Upon decryption, the contents of the padding bytes MUST be verified, as well as the MAC. Doing so securely is hard to do but if you are writing an inspection tool, then you are the attacker, and thus can afford to do things most insecurely.

In the older SSL 3.0 version, only the last padding byte had to have value n, the other bytes being random (and this is a weakness in the protocol). Conversely, in SSL 3.0, the total padding length was not allowed to exceed the block cipher length, whereas in TLS it can be longer (up to 256 bytes), as long as the total padded message length is a multiple of the block length.

For an inspection tool, the simplest implementation method would be to decrypt with padding "none", and then inspect and remove the padding and the MAC with explicit code that you write yourself. There is no "standard" padding method implemented in Java that would match the variant used in TLS.

---

If you are interested in decrypting SSL/TLS handshake, it might be a good idea to first write your own SSL/TLS library -- not for production use (these things are very hard to get right with regards to side-channel attacks), but more to gain experience in the fine details of the protocol. When your own library succeeds at completing a handshake, you know that you got all of them properly. Also, the code elements will be reusable for your inspection tool. Thus, writing your own library is a good pedagogical investment (as long as you remember the condition: not for production use !).

You may want to read this answer as an introduction to the protocol.