4

I have a problem between two ends of an SSL connection. The server sends ServerKeyExchange and the client says the server key exchange packet is not signed correctly, so I want to check the signature myself.
The packets I have contain:

Client:
    ClientHello -> client random
Server:
    ServerHello -> server random
    ServerKeyExchange -> EC Diffie-Hellman Server Parameters
        Signature Hash Algorithm: SHA256,ECDSA

My attempt:
To get the hash I use

sha256(client_random + server_random + server_params[:signature_index])

I extracted the server certificate and it has signature algorithm "sha256ECDSA".
I think I need to take the signature of the parameters, decrypt it using the server public key and compare to the hash.
The openssl command line either does not offer to decrypt ECDSA or I have not found how to do so.
How can verify the signature using command line tools or the least amount of code I can?

Daniel
  • 163
  • 1
  • 5
  • A simple way to check your server can be using the openssl sclient. Initiate a connection using the client certificate and see if it is able to create a connection. If yes, then at least your server is working fine. – Limit Nov 21 '16 at 18:23
  • @Limit: Most of the requests are fine, the problem happens rarely and I would be surprised if I could be able to reproduce it with s_client. All I have is saved packets from successes and from failures. – Daniel Nov 21 '16 at 22:15
  • @Limit: it might be worth noting that both of the ends are OpenSSL based, what I'm doing is an "exotic" transport layer. – Daniel Nov 21 '16 at 22:17
  • it's difficult to identify the cause without knowing the error flow. Unless ofcourse someone knows this off the top of their head. Can you boil it down to error flows and update the question? – Limit Nov 21 '16 at 22:24

1 Answers1

2

First, signing and verifying are NOT 'encrypting' and 'decrypting' the hash. That harmful trope is almost partly true for RSA but not really, and it is 100% totally wrong for DSA and ECDSA. See
Trying to understand RSA and it's terminology?
Public key authentication: what gets signed?
Is encrypting data with a private key dangerous?
If the public key can't be used for decrypting something encrypted by the private key, then how do digital signatures work?
Understanding digitial certifications
Digital Signature and Verification?
With GPG, can you "decrypt" a file that hasn’t been encrypted?
https://crypto.stackexchange.com/questions/15997/is-rsa-encryption-the-same-as-signature-generation

Also, the signature algorithm field in the certificate is irrelevant; that's the algorithm for the signature of the certificate by the CA, and not the signature on the data by the EE (here the server) using the certified key.

Those said, what is signed is the two random values, plus the encoded form of server-params.

Here's an example. I captured a (test) handshake in Wireshark and here I export the ServerKeyExchange message to temp3.raw: Wireshark display of ServerKeyExchange

I also exported the client and server hello nonces to temp1.raw and temp2.raw respectively, and the subjectPublicKeyInfo from the cert to tempk.raw and converted to PEM form (which OpenSSL prefers) with openssl pkey -pubin -inform der <tempk.raw >tempk.pem. Now:

$ xxd temp1.raw
0000000: ead0 1445 aec2 291d e316 8f77 b75c e0ec  ...E..)....w.\..
0000010: 696c ced0 9d45 65b3 78bd d6f8 5e11 287e  il...Ee.x...^.(~
$ xxd temp2.raw
0000000: 5834 4f06 be02 498d cba5 16b0 80bd d5f3  X4O...I.........
0000010: 4a0c 1476 d085 8c57 6fbf 5485 47cd 3937  J..v...Wo.T.G.97
$ xxd temp3.raw
0000000: 0c00 0091 0300 1741 04da 0889 2ed6 bdd0  .......A........
0000010: d140 30e8 417a 7d15 9d46 55b5 6eed 4fde  .@0.Az}..FU.n.O.
0000020: 814c a7aa fd1d 6617 2835 75c4 12d4 a2a4  .L....f.(5u.....
0000030: 213e 6141 112c fb8d 620b 21f9 e27f ea54  !>aA.,..b.!....T
0000040: 53ec 9ec1 bbbc b3df 2f06 0300 4830 4602  S......./...H0F.
0000050: 2100 fd47 429f 478b 3bd4 c8d4 ce4a 9e4c  !..GB.G.;....J.L
0000060: f6e0 c957 bf83 017d 0812 bb4a 8fda de06  ...W...}...J....
0000070: ddc0 0221 00e3 05b1 3046 3c32 a387 012f  ...!....0F<2.../
0000080: d149 e810 b423 b45c cce1 e4e9 62ce bd6a  .I...#.\....b..j
0000090: a4a0 b6d2 a3                             .....

This ServerKeyExchange consists of the handshake message header (4 bytes), the encoded curve_name and pubkey value here totalling 3+1+65=69 bytes, 2 bytes signature and hash identifiers (TLSv1.2 only, for earlier see the RFCs) here 06 03 meaning SHA512 ECDSA, 2 bytes length, and 72 bytes actual signature. I construct the input and separate the signature, and use OpenSSL commandline to (hash&)verify:

$ (cat temp[12].raw;dd if=temp3.raw bs=1 skip=4 count=69 status=none) >temp.dat
$ dd if=temp3.raw bs=1 skip=77 count=72 status=none >temp.sig
$ openssl sha512 <temp.dat -verify tempk.pem -signature temp.sig
Verified OK
Community
  • 1
dave_thompson_085
  • 9,759
  • 1
  • 24
  • 28