DTLS vs direct use of AES. What are the threats unique for direct use of AES instead of DTLS?

Question

For regular traffic in mesh network (between Internet of Thing devices) customer decided to use one of two options:

1. DTLS PSK ciphersuite - DTLS based on pre-shared symmetric key
2. or direct use of AES - customer wants to minimize traffic between sleepy devices (battery should be alive for several months)

Questions:

1. What are the additional threats of second option (AES) that do not exist for first option (DTLS-PSK)?
2. How to mitigate these threats (say using AES in CBC or another mode and change IV, etc)? Not reinventing DTLS protocol manually and not with the same overheads as DTLS has.

There is mandatory requirement to share symmetric key between IoT devices within mesh network for regular traffic. This key fits for both options. (First joining of new device will use DTLS-PAKE - I mentioned this just in case - this is not important for this question).

Let's specify DTLS-PSK mode more exactly - I'm going to use mbedTLS crypto library with define MBEDTLS_TLS_PSK_WITH_AES_128_CCM_8 is specified in cipher suite. Where CCM is mode for block ciphers that provides both authentication and confidentiality.

Answer 1

What are the additional threats of "plain" AES in contrast to DTLS?

Before I start explaining the additional threats, I will first describe what DTLS does and what it does you protect against.

The PSK "Handshake" of DTLS consists of sending nonces (both sides; server random, client random) and the identity (so the server knows to which client he is talking to and vice versa). The pre-shared key is used as the so called premaster key. Together with the nonces, a unique session key is derived for each session. This procedure is described in RFC 4279 for more details. There is implicit authentication of both participants in this handshake.

With this session key, every message is encrypted and - as you intend to use CCM - protected from manipulation. Every message has an explicit authenticated sequence number and a proper nonce for CCM. This protects your application from message replay. Improper nonce use can lead to disastrous failures, as described on Crypto SE.

In short DTLS gives you:

• Unique session keys (but no forward secrecy)
• Authentication
• Confidentiality (= encryption)
• Integrity (= protection against manipulation of messages)
• Replay protection
• Proper use of the blockcipher mode of operation.

If those properties are relevant to you is ultimately your decision.

In contrast your alternative proposal of using AES directly may lead to problems. At first, you do not say anything about integrity protection of messages. Encryption by itself does not protect from manipulation, even if an attacker does not know the plaintext. You can find an example how this manipulation is possible on Crypto SE. When using the AES-CBC manually, you also have to establish a way to have an unpredictable initialization vector (IV) - while not as problematic as in other modes of operations, it can still lead to security vulnerabilities. You also do not have any replay protection in your "plain" use AES: Thus an attacker may replay valid, but old data which you do not detect.

Last but not least, there is possibility of other security relevant mistakes you can make when writing the code for your custom protocol (time safety, padding oracles to only name two), leading to side channel attacks. mbedTLS is a well established library and is likely written with those threats in mind or has been audited to be safe against them.

Returning to your question, your plain usage of AES has the following threats in contrast to DTLS:

• Spoofing and manipulating of messages
• Replay protection
• Potentially incorrect usage of the block cipher mode (leading to loss of confidentiality)
• Side channel attacks by improper writing code

How to mitigate these threat if not using DTLS?

To protect against those attacks you have to at least:

• Add integrity protection by using a MAC (e.g. HMAC) or AEAD cipher modes (like GCM, EAX or CCM instead of CBC).
• Add a replay protection e.g. by using an authenticated counter or an authenticated timestamp.
• Use a proper nonce or IV for each message (proper depends on the mode you use)
• Know about the risks and possible mistakes of writing cryptographic software and security relevant software (or hire experts to do it)

You should derive session keys from the pre-shared one like DTLS does, so somehow leaked session keys do not endanger the long term key.

Recommendation

Try to use DTLS. It is usually considered a bad idea to write (or at least to actually use) your own crypto. Most if not all arguments valid against writing a custom ciphers can be applied for writing custom cryptographic protocols as well.

The overhead of the DTLS handshake can be greatly reduced, by using features like session resumption. This only requires a very small amount of storage for saving the session state. There even does exist a variant for constraint environments (see RFC 5077) to reduce the storage requirements on the server that is implemented by mbedTLS. There is also RFC 7925 about (D)TLS for IoT devices in general which may be interesting for you to further minimize the resource usage of DTLS.

Another option would be to search for other protocols (and implementations) having similar properties as DTLS, but with less overhead.

Answer 2

[D]TLS provides quite a bit of protection beyond just confidentiality (encryption). It verifies the server identity (authentication), protects against message corruption or tampering (integrity), prevents replay attacks, and can provide forward secrecy (no ability for an attacker to decrypt a recorded message even if all persistent keys later become known to the attacker). Modern versions also include protections against weaknesses around not-widely-understood areas of cryptography such as padding.

There are several forms of [D]TLS-PSK, but all of them potentially reduce the security somewhat. If you're not using asymmetric keys at all then any node in the network can impersonate any other node, so you lose strong authentication. The simplest form of PSK also does not provide forward secrecy, though it is possible to use ephemeral DH key exchange (enabling FS) with symmetric keys as well.

However, [D]TLS-PSK is still almost certainly more secure than any hand-rolled system. (The saying around cryptography "don't roll your own" applies to cryptosystems - at least, whenever possible - not just to primitives.) Trying to build your own cryptosystem on top of AES (or any other primitive) means you must consider stuff like padding oracles, IV/nonce generation, length, and rotation, integrity protection, replay prevention, and so on. Using an well-reviewed protocol with a well-reviewed implementation avoids all these concerns.

Answer 3

At the protocol level, the best you can probably hope for is a secure end-to-end connectivity with no man-in-the middle and perhaps with perfect-forward-secrecy added.

If the server needs the client authenticated one option at the protocol level is still dual-handshake.

From what I've understood (the PSK concept is a new one for me), the keys are pre-configured in case of PSK (how is an implementation detail which will need to be figured out). But the idea is that using a cipher suite is always better than using a single protocol to mitigate against vulnerabilities that may be identified subsequently.

For example with regard to the CCM that you plan to use as per wikipedia entry regarding it, the implementation needs to generate a nonce that will never repeat. That depending on your usecase may or may not be feasible. For example if you have a 32 bit integer, the counter will rotate after about a couple of billion cycles - which depending on your usage may be far-fetched or could be a factor.

To make a long story short, using a cipher-suite is always a better approach - in that if some vulnerability is identified in the approach, we may be able to configure the application to blacklist the given cipher-scheme. If we only have one that would not even be an option from a design perspective.

Poodle Attack and Heartbleed are two instances I can think of right now.

Given that you are looking for answers from a reputable source, may be this paper has it (i have not read it - so its free advice !) and you could consider taking a look at it.