Why are there few (none?) easy to use encryption libraries?

Question

If I search stack overflow for how to safely encrypt data, one of the first hits is someone's custom encryption scheme. I've seen several similar questions on this site, and in general they are all doomed to be severely flawed at best. Most people naive enough to even try to invent their own encryption have no idea how many ways there are to screw it up.

Everyone noding their heads up to this point?

Now, what surprises me then, possibly because I belong with the naive crowd mentioned above, is that there are no good recommendations for those of us that don't know better. While the correct answer may be, "Don't try this at home", that's not helpful.

Need a production ready web server? There's Apache and Nginx. Or a database? For most use cases that don't need to scale to Google/Facebook/Twitter scales, there are so many production ready choices that will all solve your problem that there no point in even listing them. Something more complicated? High quality machine learning libraries are readily available in most languages you can think of and probably a few more.

Want to encrypt something? To bad.

For example, lets say I search for "ruby encrypt data" in google. I end up finding OpenSSL Cipher documentation.

One might think, "Awesome, I've heard one should never invent their own encryption schemes, and here's a simple copy/paste ready piece of code", so I probably copy it, and merrily move on.

With my fairly limited understanding of the subject though, even I can see what looks like a huge flaw: the word integrity is never mentioned anywhere. There's no HMAC of the data, so the application ends up vulnerable.

If you do a similar search for php, you may be lucky enough to pick up the warning about integrity, but there no solid explanation for why it's important, nor does it mention that using an unprocessed password is unsafe, unlike the ruby docs.

So for all the developers that want to encrypt information, we're pretty much destined to get it wrong.

I'm sure there's a good reason for the complexity, but what is it?

Answer 1

Because encryption is complex. There are many pitfalls to avoid and dire consequences that can happen if you get it wrong. The answers to one of my question explores some of the reasons pretty well. At what level of abstraction should a developer work with with regards to cryptography?

That said, there are some nicely abstracted crypto libraries out there. The Fernet library provides a easy to use interface for symmetric encryption in both Ruby and Go. The Keyczar library has implementations in several languages as well. Lastly, a personal plug for a project I am involved with, the cryptography library for python aims to provide both high level crypto recipes and low level primitives with ample documentation on best practices.

Answer 2

I agree with you up to a point that there are no easy solutions. A famous recent example is Edward Snowden's inability to teach Glenn Greenwald how to use a public key encryption tool.

The landscape has been made more treacherous by deliberate(?) backdoors. Apple, Linux and RSA have all been in the news lately for broken crypto.

So for all the developers that want to encrypt information, we're pretty much destined to get it wrong.

I don't agree. While there are no easy solutions, there are plenty of solutions. The Java world has built in crypto libraries that are tried and true as well as third party libraries like Bouncy Castle (which includes HMAC).

You cite lack of message integrity checks in the library you found. Message integrity (cryptographic authentication) is a closely related but separate problem than encrypted communication. Many libraries do both. If a crypto library does not support it and you need it, then it might be found in a separate library. The wikipedia page about HMAC has a list of implementations in many languages.

I'd like to point out that implementing message integrity check yourself by following established methods is not the same as inventing your own. Just like if I wrote java code that implemented a SHA-256 all by my self, it would not count as rolling my own. The algorithm is the real vetted and proven invention here, not the implementation.

Answer 3

As you and the others noted, encryption is complex. But more importantly, security is complex; and the problems aren't all technical, and can't all be solved with encryption. Encryption algorithms are just one tiny piece of the security puzzle. By itself, encryption solves only one attribute of security: confidentiality. There are other factors, including integrity, availability, authorization, and non-repudiation, to be concerned with.

To be useful, encryption has to be performed in the context of a protocol. The protocol specifies exactly which bytes go here, and which bytes go there. If you get the encryption right but the protocol wrong, you can fail at protecting the integrity of the message. The protocol can be designed to protect the integrity of the message, but that's not the same as encryption. If the system is so complex that your users at either end can't make it work, it's not available. And key management remains the largest problem: if you get the encryption, protocols, and complexity solved, but you fail at key management or key distribution, you fail at protecting the authorization, non-repudiation, and even integrity and confidentiality. An encryption library that successfully addresses all these aspects doesn't exist.

Another part of the problem is that security is defined by the absence of successful attacks - proving a system is secure is almost impossible, because it's impossible to prove a negative. You can test for correct behavior, but you can't test for every possible instance of incorrect behavior. Novel attacks on protocols have come from replaying messages out of sequence, changing certain bytes, changing orders of operations, changing message version numbers, rearranging fields, etc. And more attacks are to come. No systems are perfectly tested against the unknown.

A large part of the problem is there are competing interests at play: if I sell "SecuriFoo 2000, your all-in-one security solution", I have little economic incentive to make it compatible with "Protect-O-Matic 8.1", my direct competitor in the business (think iMessage). There is a patchwork of encryption laws around the world, prohibiting certain kinds of encryption in certain places. And there are government agencies who have a vested interest in keeping the marketplace so complex and fragmented that most ordinary people are unable to adequately select and use protected communications, leaving insecure cracks in the messages that can be intercepted.

The field is simply immature. The practitioners are learning new things every day, but the attackers are trying new things every day, too. As long as value can be derived from breaking security, people will keep trying to break it.

Answer 4

Use HTTPS.

HTTPS is the Nginx of encryption. Most languages have a library that can do it with no configuration. It just works. If you have to configure a server, SSL Labs can tell you whether you've done it right. You could get your hands dirty and use TLS directly, but that's already getting into tricky territory.

The reason you don't find convenient building blocks at a lower level than TLS is that it's impossible to use them correctly. Even TLS 1.2, the sixth and current version of the protocol, contains known major mistakes.

Of course I'm cheating a little bit, because you didn't actually say what you needed encryption for. Maybe you're not doing transport at all? In any case, HTTPS is an example of what solutions to this sort of problem look like.

Answer 5

This is not a complete answer, but a point that seems to have been skipped over by the other answers (John Deters touches on it, but addresses it from a very different angle) here is this:

Usually the point of encryption is to package some information that can be decrypted or verified. It's quite possible to write a library which is very simple to use which provides both encryption and decryption facilities. See, for example thia Javascript TEA implementation which has 2 functions: encrypt and decrypt and 2 arguments: a password and some data.

But in many cases we don't control the software performing both ends of the operation - in order to accomodate different standards and protocols, the API the developer must interact with gets significantly more complicated.

There are also issues about how data is represented for storage and transmission.