33

Is there an easy way to test an SMTP server to check for configuration issues associated with STARTTLS encryption, and report on whether it has been configured properly so that email will be encrypted using STARTTLS?

Think of the Qualys SSL server tester as an analogy: it is a great tool to quickly check a webserver to see use of SSL has been properly configured, and identify opportunities for improving the configuration to provide stronger encryption. It knows how to recognize many common configuration errors and gives a grade. Is there anything like that for STARTTLS on SMTP servers?

In particular, given a SMTP server, I would like to tell:

  1. whether it supports STARTTLS,
  2. whether its STARTTLS configuration has been set up properly so that email with other major email providers will end up being encrypted,
  3. whether it supports perfect forward secrecy and whether it is configured so that the perfect forward secrecy ciphersuites will be used in practice (where possible),
  4. whether it provides a suitable certificate that will pass strict validation checks,
  5. whether it has any other configuration errors.

How can I do this?

Facebook and Google have recently highlighted the state of STARTTLS usage on the Internet and called for server operators to enable STARTTLS and configure it appropriately so that email will be encrypted while in transit. Are there easy-to-use tools to support this goal?

mc0e
  • 491
  • 2
  • 14
D.W.
  • 98,420
  • 30
  • 267
  • 572

5 Answers5

33

Here are a several websites that provide tests that you may be interested in.

  • SSL-Tools is a web-based tool that tests a SMTP server for each of the items you mentioned; it tests for STARTTLS support, a certificate that passes strict validation checks, support for perfect forward secrecy, and other stuff:

    https://ssl-tools.net/mailservers

  • StartTLS is a web-based tool that tests a SMTP server and provides a simple grade, along with many details on the configuration of the SMTP server (though no testing of whether perfect forward secrecy is used):

    https://starttls.info/ (see the about page information about the service, or statistics about sites checked with their service)

  • CheckTLS is a web-based tool provide a way to test a SMTP server for STARTTLS server as well as whether the certificate is "ok" (i.e., it passes strict validation) and partial information on what cipher was negotiated when they connected to that SMTP server (but no information about perfect forward secrecy support):

    https://www.checktls.com/

  • The following web-based tools check whether a SMTP server support STARTTLS, but do not perform any of the other checks mentioned in the question:

If you have to check only one or two, try SSL-Tools and StartTLS.

StackzOfZtuff
  • 17,783
  • 1
  • 50
  • 86
MrBrian
  • 704
  • 6
  • 5
  • 2
    StartTLS does not appear to be in service. SSL-Tools and CheckTLS both still work, although they both don't recognize Letsencrypt certificates as legitimate. – Bert Peters Aug 21 '16 at 21:12
  • Both SSL-Tools and CheckTLS recognize my Letsencrypt certificates. I thought they didn't, but it turns out I hadn't configured the server correctly (with Postscript I had configured the SMTPD TLS settings, but not the SMTP settings). – orrd Jan 21 '17 at 16:45
12

You can check support for starttls with openssl s_client -starttls smtp ....

  • With the right settings of -CAfile/-CApath you can also check the certificate chain.
  • What it does not check is the hostname, e.g. you have to manually check it.
  • It will also print out the cipher used, so you can check if it's a ECDHE or DHE cipher to see if forward secrecy gets used.
  • Maybe you might want to explicitly specify a cipher list with the -cipher option to find out if the server prefers FS ciphers even if the client put them at the and of the preference list.

Alternatively you might use Perl with a recent enough IO::Socket::SSL like this:

use strict;
use warnings;
use IO::Socket::SSL 1.968;
use Net::SSLGlue::SMTP;

my $host = 'mx.example.com';
my $smtp = Net::SMTP->new($host, Debug => 1) or die "connect failed";
$smtp->starttls(
    # where your CA are, has usable defaults
    # SSL_ca_file => ...,
    # SSL_ca_path => ....,
    # to restrict ciphers and set preference
    # SSL_cipher_list => '...',
) or die "starttls failed: $@|$SSL_ERROR";
print "cipher=".$smtp->get_cipher."\n";
print "cipher=".$smtp->get_sslversion."\n";

This will do a proper certificate checking, does hostname verification, gives you the cipher to find out if it is forward secrecy and gives you the SSL version too. And with the latest IO::Socket::SSL versions you can also do OCSP checking to see if the certificate is revoked (see documentation in IO::Socket::SSL).

Steffen Ullrich
  • 184,332
  • 29
  • 363
  • 424
  • Thank you! Is there a relationship between the cipher that is negotiated between `openssl s_client` and SMTP server X vs the cipher that will be negotiated between some random major email provider and server X? If they have different preferences, might they end up with a different ciphersuite (thus messing up the heuristic for telling whether ECDHE/DHE will be used)? I like the idea of specifying a cipher list; can you suggest a cipher list or lists that will be representative of what's used by major email providers? – D.W. May 27 '14 at 23:38
  • The final cipher depends on the ciphers offered by the client and supported by the server, and if the server prefers its cipher order or the clients. From my tests most servers now decide based on their own cipher preferences. You can find out the supported ciphers and if client or servers order is important with https://github.com/noxxi/p5-io-socket-ssl/blob/master/util/analyze-ssl.pl. As for recommendations have for ciphers look at the sslabs.com or at the discussion at http://security.stackexchange.com/questions/51680/optimal-web-server-ssl-cipher-suite-configuration. – Steffen Ullrich May 28 '14 at 04:38
  • This IO::Socket::SSL recipe will test whether a usable and correct ssl connection can be negotiated. I'm thinking though that it will not alert you if an unsafe cipher is offered, or if the SSL connection that is negotiated can be abused. It's useful, but It fails to provide for requirements (3) and (5) in the OP's request. – mc0e Jun 07 '16 at 04:45
2

Here are several tools that gives Qualys SSL Labs like results and support STARTTLS

  • testssl.sh (https://testssl.sh/)

    Its a command line tool which checks a server's service on any port for the support of TLS/SSL ciphers, protocols as well as recent cryptographic flaws and more. Its quite comprehensive and major advantage is that you can scan your intranet servers as well.

    eg. ./testssl.sh -t smtp aspmx.l.google.com:25

  • HTBridge SSL test

    This is web based tool allows email and other ports.

    https://www.htbridge.com/ssl/

  • Cryptosense Discovery

    This tool allows scanning on any port. Default ports scanned (21, 22, 25, 110, 143, 389, 443, 465, 587, 636, 993, 995, 5222, 5223, 5269)

    https://discovery.cryptosense.com/

bhushan5640
  • 381
  • 3
  • 12
2

CryptCheck is just like Qualys SSL server tester, only then with SMTP support. CryptCheck checks your ciphersuites grading C/B/A/A+...

CryptCheck: https://tls.imirhil.fr

The website is in French https://imirhil.fr

Ronald
  • 21
  • 1
  • Thanks, this is useful! However, it seems to be a bit unstable: I seem to frequently get "Error during analysis: Too long analysis (max 2 minutes)" errors. – D.W. Feb 25 '19 at 19:47
0

For the TLS part there are some other online tools that you can use:

and also some offline ones:

Szilárd Pfeiffer
  • 111
  • 3