Why does my RSA DANE TLSA work, but my ECDSA DANE TLSA fail?

Question

I've purchased two single domain, wildcard SSL certificates from Namecheap/Sectigo/Comodo. I generated my CSRs in the typical fashion using openssl.

$ openssl req -newkey rsa:4096 -keyout example.com.rsa.key -out example.com.rsa.csr
$ openssl genpkey -genparam -algorithm ec -pkeyopt ec_paramgen_curve:P-256 -out example.com.ecdsa.pem
$ openssl req -newkey ec:example.com.ecdsa.pem -keyout example.com.ecdsa.key -out example.com.ecdsa.csr

I submitted the CSR's and was issued each certificate package including the .crt and .ca-bundle, all as expected.

I was able to install both certificates for use by Postfix. Each required an unencrypted version of the private key, secure permissions having been maintained.

$ openssl rsa -in example.com.rsa.key -out example.com.rsa.key.unencrypted
$ openssl ec -in example.com.ecdsa.key -out example.com.ecdsa.key.unencrypted

I have correctly setup SPF, DKIM, DMARC, and DNSSEC, each confirmed by third party tools and having sent and received mail with passing scores in the headers.

I again used openssl to generate the hashes of the certificates for my DANE TLSA records. I created two sets, one set for each algorithm, and for both port 25 and 587. These were added to my zone file, checked with named-checkzone, signed with dnssec-signzone, and published in the DNS.

I began by using two external tools to check the configuration, the first at danecheck and the second at dane.sys4.de. Both reported overall success using the RSA cert but specific failure of the ECDSA cert.

The former succeeded, reporting in part:

DANE TLSA 3 1 1 [9679fc29..]: OK matched EE certificate
DANE TLSA 3 1 1 [ecd29ffd..]: FAIL did not match EE certificate

The latter succeeded, reporting in part:

3, 1, 1 9679fc296960a23c[...]149b990a680cad8b *in green*
3, 1, 1 ecd29ffd76d61326[...]dadbcfa42eae9158 - unable to get local issuer certificate: (20) *in red*

I attempted to fix this by changing the various usage, selector, and matching fields, 3 0 1, 3 1 1, etc. I tried generating the hashes locally with openssl and by using online tools like gen_tlsa. Both tools produced identical hashed results for both certificates. Again, the RSA TLSA's succeeded while the ECDSA records failed. I eventually found the chaingen script, which generates all of the 3 0 1, 3 1 1, 3 0 2, and 3 1 2 hashes which I included for each port with no improvement for the ECDSA records.

I spent several hours trying different permutations of each certificate, TLSA records, ports, etc. I tried adding TLSA records (2 0 1) for the parent keys of the ECDSA records to the DNS. I tried changing the certificate available to Postfix to include the ca-bundle along with the certificate in a .pem file.

Researching I found this post at exim-users titled "How to get ec cert used with DANE and ec+rsa certs" by Viktor Dukhovni, who needs no introduction in the Postfix/DANE circle. He suggested that perhaps the online tools were opportunistic and as soon as they succeeded with the RSA certs didn't bother to test with the ECDSA certs. He provided very valuable calls to openssl to show that indeed the keys were correct in the poster's situation. I duplicated that effort with two short shell scripts using Viktor's code substituting my information, one called checkrsa and the other called checkecdsa.

The two scripts execute two commands each with the following setup for checkrsa (for IPv4):

$ cat checkrsa

echo "quit" | openssl s_client -starttls smtp -connect mail.example.com:25 -4 -verify 9 \
    -dane_tlsa_domain mail.example.com \
    -dane_tlsa_rrdata "3 0 1 c487cdb079...57aaec4ac9" \
    -dane_tlsa_rrdata "3 1 1 9679fc2969...0a680cad8b" \
    -sigalgs rsa_pkcs1_sha256:rsa_pkcs1_sha384:rsa_pkcs1_sha512:rsa_pss_rsae_sha256:rsa_pss_rsae_sha384:rsa_pss_rsae_sha512:rsa_pss_pss_sha256:rsa_pss_pss_sha384:rsa_pss_pss_sha512

And checkecdsa (for IPv6):

$ cat checkecdsa
echo "quit" | openssl s_client -starttls smtp -connect mail.example.com:25 -6 -verify 9 \
    -dane_tlsa_domain mail.example.com \
    -dane_tlsa_rrdata "3 0 1 87a8c75581...87fb13bfc5" \
    -dane_tlsa_rrdata "3 1 1 ecd29ffd76...a42eae9158" \
    -sigalgs ecdsa_secp256r1_sha256:ecdsa_secp384r1_sha384:ecdsa_secp521r1_sha512

The results:

root@server:~/bin# ./checkrsa 
verify depth is 9
CONNECTED(00000003)
depth=0 CN = *.example.com
verify return:1
---
Certificate chain
 0 s:CN = *.example.com
   i:C = GB, ST = Greater Manchester, L = Salford, O = Sectigo Limited, CN = Sectigo RSA Domain Validation Secure Server CA
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIHNDCCBhygAwIBAgIQAzfJZrX65NjG2FvLmk0I0jANBgkqhkiG9w0BAQsFADCB
...
Xw8UgZDYihDaIxT8SUQUgV9weQg5Lkru
-----END CERTIFICATE-----
subject=CN = *.example.com

issuer=C = GB, ST = Greater Manchester, L = Salford, O = Sectigo Limited, CN = Sectigo RSA Domain Validation Secure Server CA

---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 2904 bytes and written 386 bytes
Verification: OK
Verified peername: *.example.com
DANE TLSA 3 1 1 ...99d48c44149b990a680cad8b matched EE certificate at depth 0
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Server public key is 4096 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---
250 CHUNKING
DONE
root@server:~/bin# ./checkecdsa 
verify depth is 9
CONNECTED(00000003)
depth=0 CN = *.example.com
verify return:1
---
Certificate chain
 0 s:CN = *.example.com
   i:C = GB, ST = Greater Manchester, L = Salford, O = Sectigo Limited, CN = Sectigo 
ECC Domain Validation Secure Server CA
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIEqDCCBE6gAwIBAgIRAIIKQBNWh2R9wwvn2/j30PgwCgYIKoZIzj0EAwIwgY8x
...
YemreHq/Cd5HPgIgE6InSF5ko6mWo9GMpR7w1ijpbsnShlS6EiYrpZozD0s=
-----END CERTIFICATE-----
subject=CN = *.example.com

issuer=C = GB, ST = Greater Manchester, L = Salford, O = Sectigo Limited, CN = Sectigo ECC Domain Validation Secure Server CA

---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: ECDSA
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 1811 bytes and written 348 bytes
Verification: OK
Verified peername: *.example.com
DANE TLSA 3 1 1 ...50d47bccdadbcfa42eae9158 matched EE certificate at depth 0
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Server public key is 256 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---
250 CHUNKING
DONE

So, both certificates checkout locally, but continue to fail externally.

Seeing that Viktor's message indicated that perhaps the tools, having success with RSA bailed on the ECDSA hashes, I also tried just publishing the ECDSA TLSA records. The results showed a complete failure with just the ECDSA hashes on their own.

So, I'm let with this situation: I have two sets of SSL certificates, one RSA, the other ECDSA. Both certs succeed in Postfix. Both certs checkout with openssl. Publishing both certs passes overall, based on the RSA certs, noting that the ECDSA certs fail. The ECDSA certs succeed locally, but fail externally.

I'm not sure where to go from here and am looking for help in getting my ECDSA certificate setup in parallel with my RSA records for DANE.

Update 6/5/22

I've added additional information provided with the ECDSA cert by the CA from the ca-bundle file.

First I created a pem file including both the cert and the contents of the ca-bundle.

$ cat example.com.ecdsa.cert example.com.ecdsa.ca-bundle > example.com.ecdsa.pem.combined

I made this file available to postfix in main.cf:

smtpd_tls_eccert_file=/etc/ssl/certs/example.com.ecdsa.pem.combined

Next I used a script called chaingen to generate the various TLSA records and added them to the DNS.

;; subject=CN = *.example.com
;; issuer=C = GB, ST = Greater Manchester, L = Salford, O = Sectigo Limited, CN = Sectigo ECC Domain Validation Secure Server CA
;; notBefore=May 17 00:00:00 2022 GMT
;; notAfter=Jun 17 23:59:59 2023 GMT
;;
_25._tcp.mail   IN      TLSA 3 0 1      87a8c75581...87fb13bfc5
_25._tcp.mail   IN      TLSA 3 1 1      ecd29ffd76...a42eae9158
_25._tcp.mail   IN      TLSA 3 0 2      d09f24a28f...cc8e69b13d
_25._tcp.mail   IN      TLSA 3 1 2      b83f332b52...4634ec6dce

;; subject=C = GB, ST = Greater Manchester, L = Salford, O = Sectigo Limited, CN = Sectigo ECC Domain Validation Secure Server CA
;; issuer=C = US, ST = New Jersey, L = Jersey City, O = The USERTRUST Network, CN = USERTrust ECC Certification Authority
;; notBefore=Nov  2 00:00:00 2018 GMT
;; notAfter=Dec 31 23:59:59 2030 GMT
;;
_25._tcp.mail   IN      TLSA 2 0 1      61e97375e9...8f5f65825f
_25._tcp.mail   IN      TLSA 2 1 1      e98044f242...db029d719a
_25._tcp.mail   IN      TLSA 2 0 2      2d44d5fbb5...33970c93c9
_25._tcp.mail   IN      TLSA 2 1 2      2042bc624d...a0aef921fe

In total there are four sets of TLSA records, my ecdsa cert and three parent CA chained certs. And, I've added additional sets of records for ports 465 and 587. Unfortunately, these additions haven't improved my result.

You might find [Hardenize](https://www.hardenize.com/) and [crt.sh](https://crt.sh/) more useful for their per-certificate(-chain) output. Are you quite certain your server is sending along the intermediate - It should, because what is the point of having it signed by *Sectigo* otherwise? — anx, Jun 01 '22 at 05:57
I'm not at all sure. I assumed the chain of trust was such that my certificates as leaves would be close enough to a known/published record to be useful "out of the box". I have attempted to included gap information of the parent keys to fix the problem without knowing the term "intermediate" or how it relates to my situation. I'll try to research further about how to provide such information. — synacksoldier, Jun 02 '22 at 04:52
The CA gave you two things: The certificate itself ("leaf", .crt) and others to send along with it ("intermediates", .ca-bundle), those they believe will connect the chain to known&trusted roots. If the CA provides each separately, you have to, for each algorithm, create a file containing both (1: your ecc cert + the ecc intermediates 2: your rsa cert + the rsa intermediates). I *suspect* you only gave Postfix the cert. Then your DANE setup will continue to work, but *additionally* your certificate will be easily verifiable even for clients only trying to see if it chains to a trusted root CA. — anx, Jun 02 '22 at 14:54

anx · Accepted Answer · 2022-06-06T08:57:23.093

Yup, the tools you mentioned are not appropriate to test your setup. Use a test that checks step by step and return clear messages.

A good test needs to individually test the Cartesian product of the possible combinations, netting results for:

for each IP
for each algorithm
for each of the methods, PKI and DANE
and possibly for each TLSA usage, TLS version, SNI, ..

Unfortunately, some tests only try the first IP, some tests only try the first negotiated cipher suite. Most tests fail to report in unambiguous language whether the DANE or the PKI match failed.

Your s_client -dane_tlsa_... command is just fine. When I checked, your DANE setup was good.
Because your certificates imply you also want PKI validation to work, you also want to configure Postfix to send the intermediates, which now seems to have worked. Verify with s_client -CApath /etc/ssl/certs

RE: edit: I suspect you now added more TLSA records than you need. Rather than offering both usages 2&3, both selectors (0&1), and offering multiple has algorithms (1&2), I would suggest just having one primary and one backup for each of RSA and EC, not a whole barrage of certificate associations. (I don't expect any of the records generated this way to hurt, though).

Good copy on the need for two calls to openssl s_client for each ip version (v4,v6). I've updated the text to include an example for each version. But, what would I need to include prior to that? What am I missing before expanding for ip versions? — synacksoldier, Jun 05 '22 at 04:06

Why does my RSA DANE TLSA work, but my ECDSA DANE TLSA fail?

1 Answers1