3

I have an app that receive data from third parties using various protocols: MQTT, HTTP, AMQP, ...
Some clients cannot use TLS (iot devices without TLS support), so we need to find some compact and easy way to receive encrypted data.

My idea is to use a symmetric algorithm (AES256 or similar), but how can I formalize how data are trasmitted?

Along with encrypted data we need also to pass an InitializationVector. There is a common "container" format that explain how you should package encryptedData+IV?

Of course we can just create our own format, maybe using ASN.1 or some other serialization format, but if something exists it will easier to explain to all developers.

UPDATE
Can Cryptographic Message Syntax (CMS PKCS 7) be a valid solution? Or is too complex to implement on small iot devices?

UPDATE 2 As pointed out by @AndrolGenhald I don't need to pass the salt if I store the key inside the device. I have removed the salt requirement.

UPDATE 3 We know that using TLS is the best solution, and in fact for most devices we can use it. Unfortunately some third parties developers tell me that they don't have TLS on a device type. I don't know exactly the detail (I probably should investigate better...). They ask us to provide a way to send encrypted data without TLS.

Davide Icardi
  • 33
  • 8
  • JSON is handy, or just a fixed-length concat you can slice up on the server. you can use a reserved char delimiter to split upon too. It's unclear why you need salt and in IV, but 1 question at a time... – dandavis Jul 29 '18 at 17:46
  • @dandavis Our idea is to use a binary format, because we have a limited bandwidth and most fields are not string... About salt and iv I thought it was a best practice to use a different salt for each encrypted data, to reduce the possibility of a rainbow table attack. It is correct? For passwords I always do that...thanks – Davide Icardi Jul 29 '18 at 21:55
  • Independently of the transmission format - how do you plan to do the actual AES encryption? OpenSSL, GPG, own implementation, ...? – TheWolf Jul 31 '18 at 09:36
  • @TheWolf I don't have software constraint, I can use any library compatible with Java 1.8 on the server. Also the actual algorithm can be changed if necessary. The only limitation can be on the device due to hardware limit...We can use AES-CTR or other "simple" algo... We put inside the device a "key", how to use it is up to us. – Davide Icardi Jul 31 '18 at 09:43

4 Answers4

3

To answer the question of how to package the ciphertext and IV, concatenation should suffice: IV || ciphertext || tag. This format is convenient because it allows streaming of encryption. You generate and send the IV, stream the encryption, then calculate and send the authentication tag (decryption output can't be streamed however as the tag must be checked first).

The downside of course is that it makes it more difficult to change the algorithm or to support multiple algorithms. You could include an unencrypted algorithm identifier, but you would have to be very careful, as an attacker could change it to whatever they like, opening the door to downgrade attacks.

But you're wanting to do this with a pre-shared key as an alternative to TLS, so there's a lot more to it than that. TLS doesn't just provide confidentiality, it also handles key management and authenticity (among other things). The problem with a pre-shared key is that you have to be very aware of the constraints. RFC 4107 is worth a read.

Nonce/IV Reuse

ChaCha20-Poly1305 and AES-GCM both use a 96-bit nonce, meaning there is a 50% chance of repeat after 248 messages when using random nonces, so the key should be changed well before that (NIST says before 232 messages for GCM). It's also possible to use a counter instead of a random IV for certain algorithms, but there are difficulties with that as well. Due to this risk, the RFCs for ChaCha20-Poly1305 and AES-GCM as used in CMS both require that an automated key management system be used.

The simplest key management system is symmetric key-encryption keys. A content-key is randomly generated for each message and used to encrypt it, and a pre-shared key is used to encrypt the content-key to send along with the message.

Salt

Your question mentions a salt, which implies that you're using a password. This is a bad idea. To derive a key from a password requires a slow KDF like Argon2, bcrypt, or PBKDF2, and a constrained device is likely not fast enough to run such a KDF with a high enough cost to be secure. It would be much better to generate the pre-shared key with at least 128 bits of entropy. Confusingly you also want to transmit the salt, which implies that the password is changing, but this contradicts your comment that you put a "key" on the device.

Authenticity

TLS provides authenticity, which is important because commonly used encryption modes like CBC and CTR are malleable. It would be best to use an AEAD mode like ChaCha20-Poly1305 or AES-GCM, or, failing that, to use an HMAC with Encrypt-Then-MAC.

Constrained Devices

On constrained devices that aren't capable of generating cryptographically secure randomness (and thus can't generate a random nonce/IV or a per-message key), a pre-shared key with a stored nonce counter might be the best you can do. You just have to be very careful to make sure that nonce reuse is prevented even when the device is power cycled, and make it as easy as possible to change the key if/when that becomes necessary.

Community
  • 1
AndrolGenhald
  • 15,436
  • 5
  • 45
  • 50
  • I have to heavily disagree with this answer. Encryption is easy to screw up, but harder to screw up if you use an actual standard. Too many degrees of freedom is often a path to failure. – Steve Sether Jul 31 '18 at 16:12
  • @SteveSether What specifically do you disagree with? If it's the "it doesn't matter" part I agree that's probably to vague and I'll change it. – AndrolGenhald Jul 31 '18 at 16:28
  • I disagree with the entire idea that you should roll your own key exchange and higher level encryption container. As I say, it's too easy to screw up. – Steve Sether Jul 31 '18 at 16:54
  • Key exchange I agree, but that's not the question. It's simply about how to format the ciphertext message with the IV. I'll be adding CMS as an option though once I get a better understanding of it. – AndrolGenhald Jul 31 '18 at 16:56
  • @SteveSether I think I answered the actual question too quickly without taking the context into account. I've reworked it to include info on nonce reuse and key management. – AndrolGenhald Jul 31 '18 at 19:06
  • @AndrolGenhald Thanks very much for your explanation! As you can see I'm not expert on security ... I have fixed also the question removing the Salt, you are right, I dont' need it. – Davide Icardi Jul 31 '18 at 23:02
1

As usual, it is probably not a good idea to roll your own crypto algorithm (and that includes the transmission format). I would recommend using the OpenPGP format, which has standardized the transmission of the Initialization Vector, seed for key derivation from the passphrase, the ciphertext itself, etc.

You can either install gpg and use it from a command line (see for example here), and/or use BouncyCastle's implementation. There are some questions about BouncyCastle and GPG on SE, see for example this one specifically covering Java/GPG interoperability. There is also a simple byte[] decryptor/encryptor service implemented in BouncyCastle's examples which can serve as a starting point.

TheWolf
  • 1,069
  • 7
  • 12
  • I'd add that S/MIME offers similar features as OpenPGP. Both are good and accepted solutions. https://en.wikipedia.org/wiki/S/MIME – Steve Sether Jul 31 '18 at 16:10
  • Both PGP and S/MIME are both designed for asymmetric encryption, when just using symmetric encryption I don't believe either offer authenticity (though PGP has an optional MDC I don't believe it has the same properties as a MAC). – AndrolGenhald Jul 31 '18 at 16:46
  • @TheWolf Thanks for the info. OpenPGP Bouncy Castle implementation seems to be a good fit. Do you known similar library for C? I found GnuPG/gpgme but it seems to don't have functions to handle the transmission format with symmetric keys, or I cannot find it... I see mainly command line examples or asymmetric keys. – Davide Icardi Aug 01 '18 at 08:43
  • @DavideIcardi Sorry, apart from gpgme I don't know about any other C implementation either. – TheWolf Aug 03 '18 at 06:15
0

My idea is to use a symmetric algorithm (AES256 or similar), but how can I formalize how data are trasmitted?

AES256 is a fine symmetric algorithm to use. However, you also have to choose a mode and use it correctly. I'd suggest you use an authenticated encryption mode of AES (such as GCM), in which case the encryption is supplemented with a MAC (like GMAC in GCM). An authenticated encryption mode performs both encryption and integrity tagging (via the MAC) for your data so that not only is it kept secret, but it can not be altered without you knowing about it.

Along with encrypted data we need also to pass an InitializationVector. There is a common "container" format that explain how you should package encryptedData+IV?

You must create a new random IV each time you encrypt the data. The IV is simple concatenated on at the begining or the end of the cipher text output of the AE encryption. No specific container is required, per se, just send IV+ciphertext (the MAC tag is implied in the ciphertext output of the AE mode) or send ciphertext+IV, it doesn't really matter.

What matters mode is that you have some secure way to store the encryption keys on the device and that you generate a new random IV every time you encrypt data to send.

In the system you have described (where you cant use TLS) you will presumably have to distribute secret symmetric keys with the endpoints and keep them safe. This will be the difficult part of your problem. If you have TLS you don't really need to worry about this as much because you only need the public key of the server, which does not need to be kept safe on each and every device out in the world.

hft
  • 4,910
  • 17
  • 32
  • Thanks for your info. About TLS I have added an update, just to explain better the situation. About the container my only doubt is the need to define a custom format like `IV+ciphertext` and explain to all parties. Having a solution based on a well know standard is easier and more extensible for the future I think. – Davide Icardi Aug 01 '18 at 08:14
  • What do you mean by "well known standard?" AES *is* a well-known standard (https://nvlpubs.nist.gov/nistpubs/fips/nist.fips.197.pdf). GCM *is* a well-known standard (https://nvlpubs.nist.gov/nistpubs/legacy/sp/nistspecialpublication800-38d.pdf). You want a well-known standard for *what*? For how to package together two byte arrays into a data structure? – hft Aug 01 '18 at 16:27
  • Yes, I'm looking for a way to package two arrays... Maybe it is a stupid question, but having a common way to package data can be useful in my experience. Also because in the future I need to compress data, hashing, signing, ... Things can get quickly complex. I thought that exists something like [digital container](https://en.m.wikipedia.org/wiki/Digital_container_format) but for encryption... But maybe you are right, I should just define my format. – Davide Icardi Aug 01 '18 at 20:39
  • It's just kind of hard to say without knowing what your platform is. Like, what type of containers do you already have? What tools do you have to work with. Of course, it's also hard to describe all that in a short question. Good luck with your work. – hft Aug 01 '18 at 21:26
0

In the comments you say that there is no software restriction. My advice is to include the OpenSSL library - or similar encryption library - within your software and let it handle the TLS channel, there is no need to roll your own encryption scheme

Furthermore, using just AES for symmetric encryption leaves the problem of how to exchange the symmetric keys for that encryption. Hardcoding it is a bad advice and making a key exchange scheme derives in re implementing TLS or rolling your own, in both cases it's a bad advice too

Mr. E
  • 1,954
  • 9
  • 18
  • Thanks, I agree with your about TLS. I don't have software restriction on the server, on the device we don't have direct control. For most third parties we just used TLS. Unfortunately some third parties developers tell me that they don't have TLS on a device type. I don't know exactly the detail (I probably should investigate better...). They ask us to provide a way to send encrypted data without TLS. – Davide Icardi Aug 01 '18 at 08:22