1

I know this is usually frowned upon, but supposing you were to roll your own encryption cipher (completely on your own or with a small group of friends, with no peer review possible), what could you do to make your custom algorithm as safe as possible (or rather, to keep the danger of using it to a minimum)? Similarly, what would you do for something like a hash function to make the probability of it having some trivial collision or preimage attack as low as possible?

I understand that there would be no way to approach the security of algorithms like AES, and I have no plans to use custom algorithms for anything important, but I thought it would be informative to think of what some best practices would be for rolling your own cryptosystem if you were to do so.

DEP
  • 35
  • 1
  • 2
    "keep the danger of using it to a minimum" is a phrase that could be interpreted a few different ways. One way to answer that is to make it, then shred all evidence of its existence. Then there is no danger of it being used. I think "use it safely" might work as an alternative? – schroeder May 02 '18 at 15:36
  • 1
    @schroeder Well, sending it to /dev/null is the correct way to use your own crypto. But I think what he wrote makes sense. He explained he wants to use it and is asking how to minimize the danger created by that. – Peter Harmann May 02 '18 at 15:38
  • @PeterHarmann sure, it's just poor phrasing: "dangers from using it" is what I think was meant. – schroeder May 02 '18 at 15:40
  • Another way to limit risk when using it is just not to use it for anything that's actually secret. But then the endeavour's only purpose is the academic pursuit of learning about cipher algorithms. If that's your goal, https://crypto.stackexchange.com is a much better site for you. – nbering May 02 '18 at 15:41
  • 5
    @DEP do ***not*** cross-post on StackExchange: https://crypto.stackexchange.com/questions/58876/rolling-your-own-crypto – schroeder May 02 '18 at 15:43
  • I think your answer is capture here: https://security.stackexchange.com/questions/18197/why-shouldnt-we-roll-our-own/18198#18198 It's guiding you to not roll your own, but it outlines many things to think about in order to make your own crypto safer. – schroeder May 02 '18 at 15:47
  • 1
    I think the question contradicts itself. On the one hand the OP knows to better not create own cryptography unless one has a considerable expertise in this field - which the OP does not have. On the other hand the OP wants to to it anyway and asks how to do this with the fewest problems. Only, in order to really understand the problems and to make sure you don't run into these when designing your own cryptography you would already need to have some serious understanding of cryptography, which you don't have. Therefore I propose to close the question as unclear what you are actually asking. – Steffen Ullrich May 02 '18 at 16:52
  • May be better just to customize here and there existing ones. – VovCA May 02 '18 at 16:58
  • Would using existing crypto primitives (e.g. using a hash function to make your own cipher) be considered cheating? If not, then that would probably be the safest and most conservative way. – forest May 03 '18 at 01:26
  • @SteffenUllrich I would surmise that the point of stating that OP understands not to roll his own crypto is to prevent individuals such as yourself from screaming at him not to. – Stack Tracer May 03 '18 at 03:00
  • @StackTracer: looking at the answer which currently exist I feel assured in my opinion. It essentially points the OP to things like proving uniformity, using NP hard problems etc which the OP cannot really do without already having lots of expertise in the field - only he does not. In my opinion you simply cannot give what the OP expects, i.e. simple to understand and implement rules to create his own ship (instead of using an existing one) for sailing safely in the complex waters of cryptography. – Steffen Ullrich May 03 '18 at 04:30
  • @SteffenUllrich, (to continue the boat analogy) OP may not be able to build a great enough ship to cross the sea, he might be able to build a small boat, which is enough to have some fun down at the lake with his friends. – Stack Tracer May 03 '18 at 05:54
  • @StackTracer: while the OP is asking for some fun the OP explicitly wants to keep the danger low, i.e. avoid risks. Given how complex the waters of cryptography are this is simply not possible when using only a simple raft (i.e. the current knowledge of the OP). It is also not possible to make this raft robust enough to mostly withstand the dangers with some easy to understand and implement tricks. – Steffen Ullrich May 03 '18 at 07:12
  • Why not study TrueCrypt's source ? – Overmind May 04 '18 at 04:47
  • 2
    Possible duplicate of [Why shouldn't we roll our own?](https://security.stackexchange.com/questions/18197/why-shouldnt-we-roll-our-own) – jk - Reinstate Monica May 04 '18 at 09:24
  • 1
    I've voted this down because rolling your own cryptography should not be a goal in itself. Maybe you need an algorithm that is not vulnerable to certain attacks on standard ciphers. But then the algorithm must be *significantly different* from a standard cipher. In that case you need to protect at least against all known attacks, but you'd still be vulnerable to ones specific to your scheme... You're better off spending time on a specific key derivation scheme (and you could replace your hash function with one that uses specific initial values or HMAC with a public key). – Maarten Bodewes May 05 '18 at 20:23

3 Answers3

5

Revised answer

In general, you may want to start by basing your cipher (mostly relevant for asymmetric ciphers) on a known hard problem. You may want to make mathematical proofs verifying your cipher has the desired properties and breaking it indeed requires solving the known hard problem. What kind of problems to choose was explained more on Crypto SE. By creating these mathematical proofs, you will make sure your cipher is not easily breakable directly.

The next step may be to look at side channels. Most notably timing attacks. If your cipher takes different amount of time for different inputs and keys, you may want to at least add code to either make it take the same amount of time always or just add random amount of time by active waiting.

There are many other side-channels and considerations a widely used cipher has to take into account, but this should get you started. Of course, making all these mathematical proofs is very difficult and even then, it does not give you any guarantee the cipher can't be broken. Even if the cipher is secure on its own, in combination with other cryptographic primitives, it may turn out to be easily breakable. That is why rolling your own crypto is such a big taboo. Getting it right is very very difficult and you will not find out you did something wrong until someone breaks it, or if you are unlucky, not even after that.

Peter Harmann
  • 7,728
  • 5
  • 20
  • 28
  • 2
    This is a quibble, but I am not aware of many practical asymetric cipher that actually use an NP-hard problem (lattice-based cryptography being a notable exception). – cody May 02 '18 at 18:37
  • @cody Right, for some reason I thought integer factorization is NP hard, but it is not known whether it is. – Peter Harmann May 02 '18 at 20:10
  • -1 because hashes should be using confusion and diffusion, not some hardness problem (which is a lot harder to prove). And as cody mentions, asymmetric cryptography requires a _hardness problem_, not necessarily a proven _NP-hard problem_. You also contradict yourself: If you can prove a hash has certain desirable traits, you can do the same for a symmetric cipher, since a hash can be used to trivially create a stream cipher (e.g. Salsa20, which uses a hash at its core) and vise versa (e.g. the Miyaguchi-Preneel hashing scheme, as used by hashes like Whirlpool). – forest May 03 '18 at 01:28
  • But you are right in your overall message that it takes a lot of skill to work with low-level crypto, and that asymmetric cryptography benefits from a reduction proof (proving that breaking the cipher is at least as hard as the underlying trapdoor function). – forest May 03 '18 at 01:31
  • Revised my answer to be less specific on the exact proofs and problems to use as it was better explained on crypto SE where it is more on-topic and I don't really know enough about it anyway. @forest Do you think I should leave the old answer as it is now or just delete it? – Peter Harmann May 05 '18 at 20:48
  • @forest is there even a way to verify symmetric ciphers? I can't imagine a proof that would guarantee confidentiality in any way. – Peter Harmann May 05 '18 at 22:38
  • 1
    @PeterHarmann Even for symmetric ciphers, the proof is not that it's hard, but that the difficulty is reduced to at least the difficulty of a [hardness problem](https://crypto.stackexchange.com/a/50581/54184). Like the Rabin cryptosystem is proven to be as hard as integer factorization. For symmetric ciphers you don't need to do any reductions like that, although you can create proofs for other things (such as certain properties holding in certain conditions, like having a flat keyspace or being IND-CCA secure, or having desirable properties if an underlying core function is a good PRP). – forest May 05 '18 at 22:40
  • 1
    Crypto is extremely complex. With very few exceptions (such as the OTP for confidentiality or poly1305 for integrity), there is no way at all to prove that a cipher is information theoretic secure, only that certain properties hold given certain assumptions. – forest May 05 '18 at 22:43
  • 1
    Er, I meant "even for asymmetric ciphers". Curse the 5 minute editing limit... – forest May 05 '18 at 22:50
  • @forest figured and I know how reduction works, was just trying to have as simple answer as possible. The general point is not going to be a guide of how to do this anyway, that is way to complicated for a simple answer on SE. I was mostly trying to explain why it is so hard to make own crypto. – Peter Harmann May 05 '18 at 22:58
2

Obligatory disclaimer.

You really shouldn't roll your own.

It's not like normal programming, where you can reinvent your own wheel with a different shape or color.

Any small detail can introduce a weakness you don't even know about, and which can completely break your scheme.

As your skill in designing crypto is (understandably) low, the likelihood is high. The more sensitive are the data you entrust to your scheme, or the longer you use it, the higher is the impact.

High likelihood + high impact -> high risk.

Not even professional cryptographers invent schemes and use them straight away.

Plus, even good algorithms can be implemented badly. Again, implementing algorithms (even good algorithms) is something you shouldn't do, unless it's really necessary. If anything, inventing your own algorithms is even (by far) less necessary.

Do not put this stuff in production, period.

Do not use this stuff for any information you care about, period.

That said, if you want to design something for learning purposes only, @PeterHarmann's advice is valid.

I'm just going to expand a bit on symmetric algorithms.

Most block ciphers are based on either of two approaches, SPN or Feistel networks. Choose one.

If you go with SPN, the round number can be relatively low, but remember you need a quite large P-box.

You also need to design a way to obtain the round key.

Now comes the fun part... break it!

I don't design crypto, but if I had to, for whatever unlikely reason, I would check that the scheme is reasonably secure at the very least against linear and differential cryptanalysis.

It's not a piece of cake, but you're doing it for learning purposes, right?

A. Darwin
  • 3,562
  • 2
  • 15
  • 26
0

There are certain attributes that ciphers need. The Ciphertext indistinguishability property says that pairs of output of a cipher can't be distinguishable from each other. So perform tests on the outputs, comparing them.

Another property is that of diffusion: a one bit change to the input of a block cipher should have a ripple effect that results in an average of 50% change of bits to the output. That should be testable.

Use fuzz testing on the inputs to provide a variety of samples to compare.

Basically, what you're looking for are all the attributes that make ciphers secure, and trying to find ways to write tests against those.

Next, follow that up with attacks. What are the classic attacks on ciphers similar to yours? Start with reduced round attacks and learn how many rounds it's going to take to adequately diffuse your ciphertext. Understand how your diffusion works, and structure your rounds and mixing accordingly.

And like everyone else here, I do feel compelled to end with Shneier's Law: "Anyone, from the most clueless amateur to the best cryptographer, can create an algorithm that he himself can't break." This isn't just a bunch of web amateurs telling you not to roll your own; this is one of the most preeminent cryptographers on the planet reminding other talented cryptographers not to roll their own. Heed his warning.

John Deters
  • 33,650
  • 3
  • 57
  • 110