How to you manually decrypt a SOAP message using openssl?

Question

So I have a SOAP message like this (key data and encrypted data has been truncated):

<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/" xmlns:a="http://www.w3.org/2005/08/addressing" xmlns:u="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
  <s:Header>
    <o:Security xmlns:o="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" s:mustUnderstand="1">
      <e:EncryptedKey xmlns:e="http://www.w3.org/2001/04/xmlenc#" Id="_0">
        <e:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5"/>
        <KeyInfo xmlns="http://www.w3.org/2000/09/xmldsig#">
          <o:SecurityTokenReference>
            <X509Data>
              <X509IssuerSerial>
                <X509IssuerName>CN=blah, O=blah, L=blah, S=blah, C=blah</X509IssuerName>
                <X509SerialNumber>1</X509SerialNumber>
              </X509IssuerSerial>
            </X509Data>
          </o:SecurityTokenReference>
        </KeyInfo>
        <e:CipherData>
          <e:CipherValue>TiMPCLfQgfw==</e:CipherValue>
        </e:CipherData>
        <e:ReferenceList>
          <e:DataReference URI="#_2"/>
        </e:ReferenceList>
      </e:EncryptedKey>
    </o:Security>
  </s:Header>
  <s:Body xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" u:Id="_1">
    <e:EncryptedData xmlns:e="http://www.w3.org/2001/04/xmlenc#" Id="_2" Type="http://www.w3.org/2001/04/xmlenc#Content">
      <e:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#aes256-cbc"/>
      <e:CipherData>
        <e:CipherValue>1qsIPulqkVQ3==</e:CipherValue>
      </e:CipherData>
    </e:EncryptedData>
  </s:Body>
</s:Envelope>

So the body has been encoded in aes-256-cbc and then the session key that did this encoding has been encoded with my public key.

My question is how do I decode this manually?

I was trying the following:

1. Copy Encrypted Session Key to a file

   echo "TiMPCLfQgfw==" > sessionkey.enc

2. Format key to 64 chars max per line:

   sed -e "s/.{64}/&\n/g" < sessionkey.enc > sessionkey.hex

3. Convert session key to binary format for openssl (as rsautl command only works with binary):

   openssl enc -in sessionkey.hex -out sessionkey.bin -d -a

4. Decrypt session key using openssl and privatekey:

   openssl rsautl -decrypt -in sessionkey.bin -out sessionkey.dec -inkey myprivatekey.key

5. Copy Encrypted Message Body to a file

   echo "1qsIPulqkVQ3==" > messagebody.enc

6. Format Encrypted Message Body to 64 chars max per line (hex format):

   sed -e "s/.{64}/&\n/g" < messagebody.enc > messagebody.hex

7. Convert Message Body key to binary format for openssl:

   openssl enc -in messagebody.hex -out messagebody.bin -d -a

8. Decrypt Message Body using openssl and session key:

   openssl enc -aes-256-cbc -d -in messagebody.bin -out messagebody.dec -kfile sessionkey.dec

But I get "bad magic number" on this last step when I try this. Any ideas why?

Answer 1

(It is not "hex", but Base64. Not a critical point here, but if you get the terminology right then it becomes easier to search for solutions on the Web.)

When doing symmetric encryption, openssl enc, the command-line tool, uses its own non-standard, OpenSSL-specific format which does not match what SOAP uses. OpenSSL adds a specific header (that's the "magic number" it is complaining about) and some randomness (a "salt" -- salts make a lot of sense when keys are passwords, but that's not the case here). According to the specification, encryption with AES-256-CBC in SOAP simply adds the IV as the first 16 bytes, and there is no header and no salt. This means that the following should work (I have not tried):

1. Extract the IV:

   dd if=messagebody.bin of=iv.bin bs=16 count=1

2. Convert the IV to hexadecimal (real hexadecimal this time, not Base64):

   ivhex=$(od -t x1 -w iv.bin | sed 's/^[0-9A-Fa-f]*//' | sed 's/ //g')

3. Extract the actual encrypted data:

   dd if=messagebody.in of=edata.bin bs=16 skip=1

4. Apply decryption with explicit IV, and instructing OpenSSL not to play its own non-standard games with salts:

   openssl enc -aes-256-cbc -iv $ivhex -kfile sessionkey.dec -nosalt -in edata.bin -out cdata.bin

The last remaining thorny issue is the padding. OpenSSL, apparently, uses PKCS#7 padding, which means n bytes are appended to the data upon encryption, so that:

• n is between 1 and 16 (at least one, at most sixteen added bytes);
• the total padded length is a multiple of 16;
• the n added bytes all have value n.

SOAP, on the other hand, mandates something similar, except that they only require that the last added byte has value n; the n-1 other bytes can have any value. Thus, when OpenSSL will decrypt the data, it might find a padding which is not exactly what it would expect (it depends on whether OpenSSL insists or not on all padding bytes having value n, and on what padding byte values were used by whoever produced the encrypted data in the first place).

If padding seems to be an issue, try adding the -nopad option to the openssl enc command; then you will have to inspect the obtained binary data (e.g. with od), and remove the padding bytes yourself (again, with dd).

(All of the above has been written assuming a Unix-like system, e.g. Linux or FreeBSD.)

Answer 2

OK I finally figured this out myself after much trial and error.

The steps are:

1. Copy Encrypted Session Key to a file and base64 decode it at same time

echo "TiMPCLfQgfw==" | base64 -d -i > sessionkey.enc

1. Decrypt session key using openssl and privatekey:

openssl rsautl -decrypt -in sessionkey.enc -out sessionkey.dec -inkey myprivatekey.key

1. Copy Encrypted Session Key to a file and base64 decode it at same time

echo "1qsIPulqkVQ3==" | base64 -d -i > body.enc

1. Read your session key in hex (am sure there's an easier way to do this but here's one way using hexdump unix command):

hexdump -C sessionkey.dec

And let's say it looks like this (this too short a key for aes256 but is just for demonstration purposes):

00000000  8a bc f2 86 25 4c 76 c4                           |....%Lv.|
00000008

1. Read your body in hex (again need to script this up):

hexdump -C body.enc

and let's say it looks like this:

00000000  80 33 28 0f 7a fa 73 f3  7e 4a 93 e6 a7 5f fe 2d  |.3(.z.s.~J..._.-|
00000010  29 88 84 4e e5 23 33 b8  0e 1a 43 fb 43 b1 83 ce  |)..N.#3...C.C...|
00000020

The first block (80 - f3) is the encryption iv

1. Decrypt the message using the hex version of the key and the hex version of the iv:

openssl enc -d -aes-256-cbc -in body.enc -K 8abcf286254c76c4 -iv 8033280f7afa73f3 -out body.dec

Note you will get some garbage at the beginning as you should strip off the iv from the beginning of the body before decrypting.

Not pretty, and can definitely be massively improved with some scripting but this gives you the gist of what to do.