265

Please Note: I'm aware that the proper method for secure password storage hashing is either scrypt or bcrypt. This question isn't for implementation in actual software, it's for my own understanding.

Related

Background
As far as I know, the recommended/approved method for storing password verifiers is to store:

$verifier = $salt + hash( $salt + $password )

Where:

  • hash() is a cryptographic hashing algorithm
  • $salt is a random, evenly distributed, high entropy value
  • $password is the password entered by the user

Some people advice to add a secret key into the mix (sometimes called pepper). Where the pepper is a secret, high entropy, system-specific constant.

The rationale seems to be that even if the attacker gets hold of the password verifiers, there is a good chance he or she does not know the pepper value. So mounting a successful attack becomes harder.

So, my question is:
Does adding a pepper value in addition to a salt when hashing passwords increase the overall security?

Or is the perceived increased security based on false assumptions?

Quick Update
I know the purpose of the $salt (I wrote quite a long answer on StackOverflow about it) the additional $pepper key is not improving upon what the salt does.
The question is, does the $pepper add any security other than what the salt does?

Community
  • 1
Jacco
  • 7,402
  • 4
  • 32
  • 53
  • 8
    You salt a hash to avoid them being cracked by a rainbow table. And then threes not really a lot of need to add an additional constant to the mix. – KilledKenny Apr 22 '11 at 10:30
  • Related but not quite the same as http://security.stackexchange.com/questions/211/password-hashing – AviD Apr 22 '11 at 13:17
  • Like @WzeberaFFS said, there's no extra cryptographic strentgh coming from 'publishing' your salt like that. I'm thinking thought you're asking about why do hashes in like /etc/passwd have a format of $1$saltsalt$realhashhereblahblah, is that right? If yes, then it's so the verifier can take the password, add the visible salt to it, then hash it. Without the salt being visible you could never recreate the hash. – Marcin Apr 22 '11 at 13:19
  • @Marcin, No, I'm asking exactly what's written in the question. I'm not interested in the purpose of the salt. I'm looking for the possible increase in security by adding a secret key to the hash. – Jacco Apr 22 '11 at 13:29
  • Then the answer is simple: nothing gets better with the 'extra' salt. Making the proper salt longer would make it better, in case people start pre-computing hash tables for small salts. – Marcin Apr 22 '11 at 13:56
  • 15
    If I understand correctly, and this was discussed in the q I linked to, a pepper can "...help mitigate certain compromise scenarios (eg, database backups going missing) by having a piece of the puzzle stored elsewhere" (@RoryMccune). Also can differentiate your hashes from someone elses - IF a big enough rainbow table is ever constructed... – AviD Apr 22 '11 at 14:15
  • 4
    This question asks specifically about pepper. For the broader picture, see a more complete discussion elsewhere on IT Security StackExchange of the more general topic of [password hashing](http://security.stackexchange.com/questions/211/how-to-securely-hash-passwords) – nealmcb May 14 '11 at 19:23
  • plenty arguments against using a pepper: https://stackoverflow.com/a/16896216/4031815 – CommonSenseCode May 20 '18 at 17:31
  • 3
    @commonSenseCode, yes, plenty of arguments can be brought against using a pepper. But having a lot of arguments does not automatically make a better point. In the case of the pepper, it is by now quite widely accepted as a method that increases security, when used in the right context. Since 2017, even NIST started to recommend it. – Jacco May 22 '18 at 07:01
  • [Dropbox blogged](https://blogs.dropbox.com/tech/2016/09/how-dropbox-securely-stores-your-passwords/) that they add a pepper. [Filippo Valsorda already blogged in 2014](https://blog.filippo.io/salt-and-pepper/) that he thinks a pepper is a good idea -- he is a well-known security guy, currently working on a cryptography team at Google. If you don't take it from almost two hundred upvotes, take it from them. – Luc Feb 07 '19 at 20:49
  • We run with a different pepper per environment (and 'stored securely'). This also ensures that the hashes cannot be 'transferred' across environments, similarly as encrypting with an environment-specific key. (There is also a pepper applied to key-based encryption, with the similar intent: an additional piece of the puzzle is required making it much harder to enable 'transferring' of data between environments.) – user2864740 Aug 04 '20 at 20:00

7 Answers7

217

In some circumstances, peppers can be helpful.

As a typical example, let's say you're building a web application. It consists of webapp code (running in some webapp framework, ASP.NET MVC, Pyramid on Python, doesn't matter) and a SQL Database for storage. The webapp and SQL DB run on different physical servers.

The most common attack against the database is a successful SQL Injection Attack. This kind of attack does not necessarily gain access to your webapp code, because the webapp runs on a different server & user-ID.

You need to store passwords securely in the database, and come up with something on the form of:

$hashed_password = hash( $salt . $password )

where $salt is stored in plaintext in the database, together with the $hashed_password representation and randomly chosen for each new or changed password.

The most important aspect of every password hashing scheme is that hash is a slow cryptographically secure hash function, see https://security.stackexchange.com/a/31846/10727 for more background knowledge.

The question is then, given that it is almost zero effort to add a constant value to the application code, and that the application code will typically not be compromised during an SQL Injection Attack, is the following then substantially better than the above?

$hashed_password = hash( $pepper . $salt . $password )

where $salt is stored in plaintext in the database, and $pepper is a constant stored in plaintext in the application code (or configuration if the code is used on multiple servers or the source is public).

Adding this $pepper is easy -- you're just creating a constant in your code, entering a large cryptographically secure random value (for example 32byte from /dev/urandom hex or base64 encoded) into it, and using that constant in the password hashing function. If you have existing users you need a migration strategy, for example rehash the password on the next login and store a version number of the password hashing strategy alongside the hash.

Answer:

Using the $pepper does add to the strength of the password hash if compromise of the database does not imply compromise of the application. Without knowledge of the pepper the passwords remain completely secure. Because of the password specific salt you even can't find out if two passwords in the database are the same or not.

The reason is that hash($pepper . $salt . $password) effectively build a pseudo random function with $pepper as key and $salt.$password as input (for sane hash candidates like PBKDF2 with SHA*, bcrypt or scrypt). Two of the guarantees of a pseudo random function are that you cannot deduce the input from the output under a secret key and neither the output from the input without the knowledge of the key. This sounds a lot like the one-way property of hash functions, but the difference lies in the fact that with low entropy values like passwords you can effectively enumerate all possible values and compute the images under the public hash function and thus find the value whose image matches the pre-image. With a pseudo random function you cannot do so without the key (i.e. without the pepper) as you can't even compute the image of a single value without the key.

The important role of the $salt in this setting comes into play if you have access to the database over a prolonged time and you can still normally work with the application from the outside. Without the $salt you could set the password of an account you control to a known value $passwordKnown and compare the hash to the password of an unknown password $passwordSecret. As hash($pepper . $passwordKnown)==hash($pepper . $passwordSecret) if and only if $passwordKnown==$passwordSecret you can compare an unknown password against any chosen value (as a technicality I assume collision resistance of the hash function). But with the salt you get hash($pepper . $salt1 . $passwordKnown)==hash($pepper . $salt2 . $passwordSecret) if and only if $salt1 . $passwordKnown == $salt2 . $passwordSecret and as $salt1 and $salt2 were randomly chosen for $passwordKnown and respectively $passwordSecret the salts will never be the same (assuming large enough random values like 256bit) and you can thus no longer compare password against each other.

Community
  • 1
  • Hmm, the edit removed the much of the uncertainty present in the answer as written by @Jesper Mortensen. Maybe the edit should be rolled back and moved to a separate answer? – Jacco Apr 23 '11 at 22:08
  • 14
    The gist of the argument here is pretty good and I tend to agree. However don't put a constant in the code if you can avoid it, make it a configuration variable of some kind (obv. not stored in the DB, but not in the code either). – frankodwyer Apr 24 '11 at 09:30
  • @Jacco: I agree that the tone of this answer has changed with the edits done. But then, looking at who made the edits, I'm now more comfortable that this answer is correct (within its scope), so I'm happy to leave it as it is now. :-) –  Apr 24 '11 at 11:27
  • 16
    I agree with the statement: **that it is almost zero effort to add a constant value**. It really is, so albeit not as critical as salting a password, I see no reason to exclude peppering in some form to provide an application code level protection. – The Thirsty Ape Feb 11 '14 at 18:20
  • 5
    @frankodwyer, **You could even have both.** Constant in the code on the webserver plus another constant in a "config" entity on another server. – Pacerier Dec 02 '14 at 07:40
  • What if you had a saved byte[] for another RNG, and then as well as the salt, added another block of data from that? Would that also work as a "changing pepper"? –  Apr 12 '16 at 19:14
  • @frankodwyer what would be the security advantage of placing the pepper in configuration? Configuration is typically stored in the same source control repository, deployed as plaintext alongside the binaries, and loaded into memory. I suppose if you wanted to change the pepper config is easier, but you'd have to worry about a migration path anyway and really that's an engineering concern, not a security one. – Ohad Schneider Aug 12 '17 at 11:34
  • Having said that, there is a concept of "secure configuration", like Azure Key Vault (which uses HSMs). You could put your secret there and get it securely at runtime, possibly even retaining it as a `SecureString` in memory. That would definitely create a serious hurdle for the would-be attacker. – Ohad Schneider Aug 12 '17 at 11:36
  • Regarding comparing `passwordKnown` hashes, I doubt it's a feasible approach seeing as the rate at which you could change passwords in the web app (presumably using the user-facing interface, even if you somehow script it to use the underlying HTTP/API) would probably be far too slow for you to actually mount any attack. – Ohad Schneider Aug 12 '17 at 11:38
  • is there any advantage to storing the salt in the database? Drupal systems, for instance, only store salt in the filesystem. – Capi Etheriel Apr 05 '18 at 17:02
  • @CapiEtheriel As long as you generate a new salt every time a new password is hashed and you store that in the filesystem the net result is the same (with the exception of the filesystem solution being unwieldy). But it sounds like misinterpreting the goal of salt - slowing down cracking of the passwords by making it so that each password has to be worked on separately. The same password for two different users will be different, and whatever hashes you find in one user won't matter for another because of the difference in salt, so salt can be "public" without security risk. – Maurycy Apr 24 '18 at 05:38
  • my mistake: drupal does not store salt in a file. the drupal_hash_salt in settings.php is meant for other things, not password hashing. it seems not to use pepper, but stores salt in the database. – Capi Etheriel Oct 09 '20 at 03:00
97

(Note: using a salt is only half of the job; you also need to make the hash function slow -- so that attacking a single low-entropy password is still difficult. Slowness is usually achieved through multiple iterations, or hashing the concatenation of 10000 copies of the salt and password.)

What your "pepper" does is that it transforms the hash into a MAC. Making a good, secure MAC out of a hash function is not easy, so you'd better use HMAC instead of a homemade construct (the theoretical way of putting it is that a collision-resistant hash function is not necessarily indistinguishable from a random oracle).

With a MAC, you may gain some security in the following sense: possibly, database read access by the attacker could cease to be a real problem. The MAC key (the "pepper") may concentrate the confidentiality need. However, this relies on the MAC being also a one-way function, which is a property which you will get from many MAC constructions (including HMAC) but which is not really guaranteed cryptographically speaking (there are subtleties).

The "pepper" implies that you have a key to manage, including secure storage in a way which resists to reboots. A key is small and fits in RAM, but, due to the storage requirements, it is unclear whether it actually improves security. An attacker who can read the whole database usually can also read the whole harddisk, including any "protected" file. The key small size may allow for some advanced setups, e.g. the key being stored on smartcards which are used at boot time but not left connected afterwards. To sum up, whether peppering is worth the effort thoroughly depends on the context -- on a general basis, I would recommend against it, in order to avoid the added complexity.

Community
  • 1
Thomas Pornin
  • 320,799
  • 57
  • 780
  • 949
  • 1
    I'm trying to wrap my brain around "Adding a key the hash transforms it into a MAC". Am I correct in understanding that, by adding a constant value (along with the password and salt) into the hashing algorithm, the changed use of the primitive affect the stength of the outcome? And that, if the key is known, the overall scheme is likely to be less secure? – Jacco Apr 22 '11 at 20:19
  • 4
    @Jacco: if the hash function is any good, then no, you do not weaken it in any way by adding a known prefix or suffix. However, being a good hash function is not fully equivalent to being a good MAC (the difference is subtle but real), hence the need to follow well-studied constructions such as HMAC. As for a good book, you can try the Handbook of Applied Cryptography ( http://www.cacr.math.uwaterloo.ca/hac/ )(not the same book than "Applied Cryptography" by Schneier). – Thomas Pornin Apr 22 '11 at 20:54
  • @Thomas Pornin: Please see my answer below -- I'm not sure if we are talking about the same thing. If we are, then I'm using words more common to webapp developers below, so your comment on what I'm describing would be valuable. –  Apr 23 '11 at 08:40
  • 5
    @Jesper: we are talking of the same thing, but with distinct viewpoints. The "pepper" increases security only insofar as it is not known by the attacker -- whether this is true or not is not easy to guess, especially in your setup in which the webapp and the database are on distinct machines. The pepper will not _decrease_ security by itself, except that it increases installation complexity (a value which must not be known by the attacker is a key, and key management is known to be non-immediate). Also, with a secret pepper, security increase depends on whether the construct is a good MAC. – Thomas Pornin Apr 23 '11 at 14:09
  • "_The "pepper" implies that you have a key to manage, including secure storage in a way which resists to reboots._" And if you can do that, why not encrypt the password database? – curiousguy Oct 15 '11 at 16:02
  • @curiousguy: well, you can. Some may view the hashing as an additional security layer: even if the attacker gets the "pepper", he can still not recover the passwords immediately (he _can_ do an offline dictionary attack, but that's not as easy as having a list of decrypted passwords). – Thomas Pornin Oct 15 '11 at 19:37
  • 48
    -1 for the comment "An attacker who can read the whole database usually can also read the whole harddisk". Clearly you are not a penetration tester, don't make such assumptions. – rook Jul 19 '12 at 09:23
  • 1
    Good that Rook pointed out the flaw in Thomas' comment (thanks @Rook); still, I have upvoted the answer because the general idea makes sense, and that's what matters: I agree that complexity is security's enemy. – Lex Nov 22 '13 at 15:56
  • 8
    @Lex, **Unneeded** complexity is security's enemy. Security itself is inherently complex. – Pacerier Nov 02 '14 at 21:29
  • 7
    @rook, it's better to point out the flaw in the answer than to attack the writer's credentials, as pointing out the flaw increases the information with which other users can judge the answer. I will do that here. It is not true that database credentials are usually equal to whole OS access. DB servers are frequently separated from the application running them, which connects over the network. Or flaws in the DB software might lead to full access to the DB, even if the DB user cannot break out of its sandbox. – madumlao Mar 19 '16 at 10:17
  • "you have a key to manage" that can just be one line of code in my app (e.g. `const key = "76fc67a8-dde0-4f49-9e5e-84fb186b97b9"`) so I don't see why not. As others have mentioned, having read access to my DB might be quite far from access to my binaries / sources. – Ohad Schneider Aug 12 '17 at 11:10
  • One could also add the key to one's HSM-based secret vault (which he has to manage and retrieve secrets from anyway) like Azure Key Vault. Then even if the attacker retrieves the code and binaries he doesn't get the secret. One could even go a step further and store the key securely in memory (e.g. .NET's `SecureString`) in which case even access to the server's memory might not easily lead to key leakage. The added complexity would be quite low. – Ohad Schneider Aug 12 '17 at 11:52
35

I would like to point out what a pepper really can do.

When does a pepper help?

As the others already pointed out, adding a pepper is only an advantage, as long as the attacker has access to the hash-values in the database, but has no control over the server, and therefore does not know the pepper. This is typical for SQL-injection, probably one of the more often used attacks, because it is so easy to do.

What does a pepper improve?

$hashValue = bcrypt('12345', $cost, $salt);

This password you can get easily with a dictionary attack, even if you correctly used a slow key-derivation function. Put the most used passwords into a dictionary and brute force with this weak passwords. It's very likely that we find the password in (too) many cases.

$hashValue = bcrypt('12345anm8e3M-83*2cQ1mlZaU', $cost, $salt);

With the pepper, the weak password grows in length, it contains now special characters, and more important, you will find it in no dictionary. So, as long as the pepper stays secret, it does prevent dictionary attacks, in this case it can protect weak passwords.

Edit:

There is a better way to add a server side key, than using it as a pepper. With a pepper an attacker must gain additional privileges on the server to get the key. The same advantage we get by calculating the hash first, and afterwards encrypting the hash with the server side key (two way encryption). This gives us the option to exchange the key whenever this is necessary.

$hash = bcrypt($passwort, $salt);
$encryptedHash = encrypt($hash, $serverSideKey);
martinstoeckli
  • 5,149
  • 2
  • 27
  • 32
  • 1
    I would suggest bcrypt(hash_mac('sha256', $password, $pepper), $salt) – Jacco Nov 02 '12 at 16:06
  • 1
    @Jacco - Just added an example of using a hmac. – martinstoeckli Nov 02 '12 at 21:42
  • I'm not sure in what language your example is, but it would be more clear if it where language agnostic – Jacco Nov 05 '12 at 07:23
  • @Jacco - It is PHP, but the function `bcrypt` is fictitious, because PHP itself does not provide such a function yet (before 5.5), the function `hash_hmac` exists though. I tried to edit the answer, so all confusing parts are removed, what do you think i could improve more? – martinstoeckli Nov 05 '12 at 08:00
10

The paper on the invention of salting and iterations, for Unix passwords (Password Security: A Case History, Morris & Thompson, 1978), also described the equivalent of a pepper:

The first eight characters of the user’s password are used as a key for the DES; then the algorithm is used to encrypt a constant. Although this constant is zero at the moment, it is easily accessible and can be made installation-dependent.

I haven't heard of it being used though. Has anyone else?

Bono
  • 165
  • 8
nealmcb
  • 20,544
  • 6
  • 69
  • 116
  • I think all modern distros use something based on a hash function (not an encryption function like DES), so this ability isn't available anymore. – Brendan Long Nov 15 '12 at 18:16
9

Just a BTW, the new NIST Digital Idendity Guidelines (Draft) strongly recommend to use Pepper as well:

https://pages.nist.gov/800-63-3/sp800-63b.html#sec5

5.1.1.2 Memorized Secrets Verifier:

... A keyed hash function (e.g., HMAC [FIPS198-1]), with the key stored separately from the hashed authenticators (e.g., in a hardware security module) SHOULD be used to further resist dictionary attacks against the stored hashed authenticators.

eckes
  • 962
  • 8
  • 19
  • 1
    Wow, I've been advocating for pepper for years but people rarely implement it for no apparent reason. I didn't know NIST now also recommends it! Thanks for this answer, this should be higher up. – Luc Oct 11 '18 at 08:43
3

Consider this scenario:

I am about to break into a website X using an SQL injection to retrieve a list of users, with their password hashes and salt. Suppose website X is also using a global pepper.

All I would have to do would be to register a user at website X with a user name and password known to me prior to the SQL injection. I would then know, for a particular record in the database, the password hash, the plain text password, the salt (stored as plain text) and it would be computationally trivial for me to crack the global pepper on the basis of this one record.

So really, a pepper would be a way of slowing an attacker down for a trivial amount of overhead time. They wouldn't have to brute force the password + salt + pepper, as intended, only the pepper.

The above is a form of chosen plaintext attack. As long as attackers know the algorithm (hash()), the output ($hashed_password), and all but one of the inputs ("constants" $salt & $password and "variable" $pepper), they can "solve for x" like a linear algebra equation (h=s+p+x == h-s-p=x), but by brute force of course. Making the pepper longer than 56 bytes (448 bits), bcrypt's limit, increases the time cost and is as good as bcrypt but still may not be as good as scrypt. So, as long as the pepper is sufficiently long, it is an improvement.

The Grumpy Hacker
  • 3
  • 3
Alex R
  • 71
  • 1
  • 1
  • 18
    What if the pepper is a 128 bit random value? Is it still trivial to crack it on the basis of salt and plaintext password value? The reason why salt and pepper values are helpful is because you can easily make it long enough (128 bit) and you don't have to remember it that is why cracking it through brute force is never trivial. – void_in Aug 31 '13 at 10:25
  • 1
    If the pepper is a 128 bit random value then it certainly won't still be trivial to crack. However you have to remember the following: - you are adding this value to the overhead of the hashing function on **every single validation** - the pepper is global and only needs to be brute force cracked once, not once for every record - you are much better off increasing the length of the salt than the pepper – Alex R Mar 10 '15 at 22:09
  • 7
    The cost of any even partially appropriate hash function like bcrypt, script, or even multiple iterations of some sha, does not depend on the length of the key, so this is wrong. If the hash function is chosen correctly, and not just using a single iteration of `sha256`, the pepper merely needs to boost the initial password to 50 bits of entropy to make it bullet proof for decades if not centuries. Of course, like @void_in said, we boost it to 128 bit or 256 bit, just because we can. – Peter Dec 10 '15 at 20:00
1

Not very familiar with how a server would be able to hide a global pepper constant but my take is that sooner or later a hacker that has penetrated the server will figure out how to capture the pepper value. To make a pepper value totally secure would require special hardware. One way to do this would be to use a FPGA board installed in the server. The FPGA would contain the code used to perform the hash including the pepper value and all hash calculations occur inside the FPGA. With the FPGA, the programming can be a one way function. The pepper can be programmed in but there is no instruction that can be sent to read it back out. The pepper would be stored on a piece of paper locked in a safe. If the pepper is 128+ bits generated randomly, there would be no practical way of determining it.
Not sure how practical this would be as it would increase server hardware cost.

Brian Sparks
  • 11
  • 1
  • 2
    There are cryptographic modules available which do something like this - one-way devices for storing+verifying passwords and/or keys. – Mark K Cowan Jul 31 '16 at 17:07