18

On my Mac, HTTPS connnections to certain sites fail using the built-in curl binary of macOS 10.14. They work fine with different browsers, as well as other builds of cURL on the same system. One of the affected sites is https://kapeli.com/, the download site for the utility "Dash". Another one is https://electroncash.org.

cURL complains about an expired certificate:

curl: (60) SSL certificate problem: certificate has expired
More details here: https://curl.haxx.se/docs/sslcerts.html

I encountered this during installations with Homebrew-Cask, which uses macOS' built-in cURL to download software.

F30
  • 451
  • 1
  • 3
  • 10

5 Answers5

17

Root CA certificates used by the mentioned sites (Comodo and USERTrust) have expired this morning (UTC time). While I find it remarkable that two different Root CA certs would expire at the exact same second, this may be explained by USERTrust being affiliated with Comodo (now Sectigo).

Edit: These two actually never were Root CA certificates, but rather intermediate CAs signed by "AddTrust External CA Root". Therefore, their expiration date was determined by the validity of the "AddTrust External CA Root" certificate, which also happens to have expired at the exact same second.

Now, updated certificates (sharing their private keys with the expired ones) have been issued back in 2010 (Comodo, USERTrust). These certificates are part of the common Root CA stores these days (including Apple's system trust store), therefore browsers establish the connections perfectly fine. The same is true for most variants of cURL (e.g. from MacPorts or Homebrew), which are built against custom OpenSSL installations.

The built-in cURL variant of macOS 10.14 is built against LibreSSL and uses /etc/ssl/cert.pem as its Root CA store, which also includes the new certificates. However, something appears to causing cURL or LibreSSL to prefer the old certificates for its validity check. I suppose cURL is at least somewhat involved in the problem, since I couldn't get the connections to fail using /usr/bin/openssl s_client (/usr/bin/openssl is actually built from LibreSSL).

My hypothesis would be that the problem is caused by the sites sending the expired Root CA certificate as part of their certificate chain. Including the Root CA in such chains is allowed, but not required, and in this case appears to break certificate validation.

Edit: This is part of a series of issues around the "AddTrust External CA Root" expiration. See this blog post by Andrew Ayer or this Twitter thread by Ryan Sleevi for the bigger picture. Ryan Sleevi also has a collection of things failing due to the expiration.

On macOS 10.15, where cURL uses OpenSSL 0.9.8 by default, the issue apparently may be mitigated by setting the environment variable CURL_SSL_BACKEND=secure-transport. This does not work on 10.14 with its LibreSSL which, according to Christian Heimes, is affected by the issue in general.

F30
  • 451
  • 1
  • 3
  • 10
  • 3
    Removing the `AddTrust` certificate from `/etc/ssl/cert.pem` fixed it for me, which confirms your theory. See this patch which I applied to remove that certificate: https://gist.github.com/jmibanez/d3a5c97c44e5077edf00b9b68d60ed25 _Edited to add_: This is for kernel.org specifically; I also queried the certs, and AddTrust was indeed being sent by the server as part of the certificate chain, so as well that adds more data to confirm your theory. – jmibanez May 30 '20 at 15:04
  • Sectigo talks about the new certificate being cross-signed by the old one in their [support article about the expiration](https://support.sectigo.com/articles/Knowledge/Sectigo-AddTrust-External-CA-Root-Expiring-May-30-2020). However, I don't understand the role of cross-certification, since both certificates share the same key anyway. – F30 May 30 '20 at 15:58
  • 2
    Useful post about it by an expert: https://www.agwa.name/blog/post/fixing_the_addtrust_root_expiration – Z.T. May 30 '20 at 20:19
  • Regarding the comment about root certificates: Not only did the `AddTrust External CA Root ` certificate expire today, but the `USERTrust RSA Certification Authority`, which was signed by ``AddTrust External CA Root` also expired. So many people got bit by this even if they didn't include the root certificate. I have a 2-week old cert that includes `USERTrust RSA Certification Authority`. It was provided by InCommon, which is used throughout academia. – Stefan Lasiewski May 30 '20 at 22:04
  • 1
    To fix homebrew, add this snippet to your zshrc (or etc.): https://gist.github.com/ypresto/d6148ec94345cd7b869a3c70f66911f5 . HOMEBREW_NO_ENV_FILTERING=1 is important to pass ssl backend env: https://github.com/Homebrew/brew/issues/6274#issuecomment-507937736 – ypresto May 31 '20 at 18:29
  • @StefanLasiewski: The cross-signed variant of "USERTrust RSA Certification Authority" expired, but it can be drop-in replaced with the new variant, which is a Root CA itself. – F30 Jun 01 '20 at 07:16
4

Just comment out the AddTrust entry in /etc/ssl/cert.pem, since the end certificates are cross-signed, they will be validated against USERTrust.

In theory there should be no need to comment out that entry, but in practice, the LibreSSL version that ships with mac (2.8.3 on Catalina) has broken certificate path validation because it's based on an older version of OpenSSL that also contains the same bug (< 1.1.1).

According to the LibreSSL documentation (https://www.libressl.org/releases.html), they started incorporating OpenSSL 1.1.1 functionality in their 3.x.x series, I could find a way to update it manually but I am lazy and will wait for Apple to fix it.

Ivan Rodriguez
  • 41
  • 2
3

At least a workaround for macOS 10.15.4:

I encountered the same issue today in conjunction with the codecov bash script. My quick fix: brew install curl and do what brew link curl suggest. You can check if you've picked the right curl with which curl (should point to /usr/local/opt/curl/bin/curl).

I have no time or patience to wait for to fix those things.

menelik
  • 31
  • 1
  • 1
    But `brew cask install` (as well as `brew install`) explicitly runs `/usr/bin/curl`, not just `curl`. The OP is about `brew cask install` failing. – Noach Magedman Jun 29 '20 at 15:02
2

All these sites that I have found seems to have the same expired CA cert in their chain:

openssl s_client -connect kapeli.com:443
CONNECTED(00000003)
depth=3 C = SE, O = AddTrust AB, OU = AddTrust External TTP Network, CN = AddTrust External CA Root
verify error:num=10:certificate has expired
notAfter=May 30 10:48:38 2020 GMT

I see issues popping up on many different sites about this right now. Editing the ca cert file like @jmibanez suggests will probably work when the site isn't sending the expired certificate in the response. I tried the latest CA cert file from https://curl.haxx.se/ca/cacert.pem using curl --cacert path/to/cacert.pem which didn't work. Browsers seems fine so they seem to ignore the expired CA cert included in responses from web sites.

EDIT: My bad here. I was using curl 7.54 by mistake. Newer versions are working. The error does not exist when using curl 7.67/7.70.

Johan Andersson
  • 21
  • 3
  • 1
    To add, I can confirm the issue exists up to at least curl 7.64.1 which is what I'm running with macOS 10.15.4. – Alex Taylor May 30 '20 at 22:47
2

I had to fix this issue on a debian based server

here is how it went:

  1. remove AddTrust_External_Root.crt from your system (usually found in /etc/ssl/certs)
    1. remove or comment the "mozilla/AddTrust_External_Root" line from /etc/ca-certificates.conf
    2. run sudo update-ca-certificates to update the certificates used by openssl

may it can help you ?

Elvandar
  • 21
  • 3