42

Aren't API keys considered usernames and API secrets considered passwords? Why is it that API servers like Amazon Web Services allow you to view your API secret in plain text? It makes me think they store it in plain text or at least in a decrypt-able format.

Isn't it better if API secrets were treated as passwords that you should type-in to create then hashed in the database instead of being handed to you in plain text? If for some reason their API secret database were compromised it will easily open the flood gates for many applications that are using their API. However if it was hashed in a non decrypt-able manner then all is not easily lost.

IMB
  • 2,888
  • 6
  • 28
  • 42
  • 1
    The short answer to your question is this: API secrets are *symmetric keys*, not passwords. What makes you think they don't store it in a decryptable format? – David Schwartz Aug 13 '12 at 10:47
  • 3
    @DavidSchwartz I didn't say that. I said it's either plain text or decrypt-able. – IMB Aug 13 '12 at 11:28

8 Answers8

19

No. The API keys need to be stored in cleartext.

They are not passwords: they are cryptographic keys. These keys are used for things like authenticating requests (using SHA1-HMAC). The server needs to know the crypto key to apply the cryptographic algorithms.

Therefore, the API key needs to be stored in cleartext on the server. If the server stored only a hash of the API key, it could not verify the authenticity of messages from the client or authenticate messages sent to the client.

D.W.
  • 98,420
  • 30
  • 267
  • 572
  • 3
    What is preventing an API server from hashing a key for storage since it can hash the user input (i.e., the client) for comparison the same way password storage works? – IMB Aug 13 '12 at 11:31
  • 6
    @IMB, If you hash the API key, and both sides use the hash of the API key as the actual cryptographic key (say, for use with a message authentication code), then you've gained nothing: the hash now becomes the effective key, and both sides are storing that in plaintext. The point is that a cryptographic key is not the same as a password. A password can be stored hashed; a cryptographic key generally cannot. – D.W. Aug 13 '12 at 18:02
  • It's not clear to me why nothing is gained. In HTTP Digest auth you can store the `HA1` in `md5` in your database. So the password (which can be used as API secret key) is never stored in plain text. AFAIK in Digest auth, the client only supplies username and plain text password, while server only stores the hash so the database is protected with hashed passwords. Didn't this example just gained password secrecy? If the attacker gains the database of hashes, he can't simply use it since he must crack it first to know the plain text version. – IMB Aug 13 '12 at 18:26
  • 2
    @IMB, Amazon Web Services doesn't use HTTP Digest auth. And, HTTP Digest auth is insecure against a network attacker. Instead, Amazon Web Services uses a symmetric-key message authentication code to "sign" messages. That is more secure, but it requires a cleartext crypto key. – D.W. Aug 13 '12 at 19:26
  • I know my example was AWS but I also mean other APIs, and other APIs implement HTTP Digest auth. You mention it's insecure but that is only true if the API server is not using SSL/TLS right? – IMB Aug 13 '12 at 19:39
  • 1
    @IMB, If you want to know about HTTP Digest auth over SSL, I'd encourage you to create a new question and ask about that one separately -- that's a different topic, and the answer for HTTP Digest auth over SSL is different from the answer for AWS. (Do note: HTTP Digest auth does not use an "API key".) – D.W. Aug 13 '12 at 19:57
  • Yes I understand it may not be called API key or API secret literally but the purpose is quite the same: API key = Username, API secret = Password. BTW, I do have a separate question that you may want to answer: http://security.stackexchange.com/questions/18563 Note that I pointed out not using SSL in that question but you can note about SSL in your answer, if ever. – IMB Aug 13 '12 at 20:06
  • 1
    @IMB, again, please move any questions about storage of the secrets for HTTP Digest Authentication to a separate question. Please avoid extended discussion of it in this comment thread. – D.W. Aug 13 '12 at 20:09
  • _They are not passwords: they are cryptographic keys_ @D.W. do you have any formal reference for this? – sw. Apr 11 '16 at 16:36
  • @sw.: https://docs.aws.amazon.com/general/latest/gr/signing_aws_api_requests.html – Lie Ryan Feb 28 '17 at 13:13
  • If you hand wave terminologies a little bit, there is actually a way to "store" cryptographic key in "hashed" form. Using public key encryption, if the authentication server is compromised, then all the attacker massage to obtain is a public key. The attacker can't use the public key obtained in a such a way to make request to the application/resource server. – Lie Ryan Feb 28 '17 at 13:19
  • @D.W. I realize that you penned this answer 10 years ago, in 2012. Do you still feel this answer applies today, in 2022? If so, how does this square with Anders' answer at https://security.stackexchange.com/questions/180345/do-i-need-to-hash-or-encrypt-api-keys-before-storing-them-in-a-database, or with the answer below by Nick Sloan? – mti2935 Aug 01 '22 at 17:11
17

There seems to be a bit of confusion here and other places, so I'm adding this answer in hopes that it will clarify the situation for other users.

There are two types of API secrets that might be used.

The use case that most of the answers on this page are discussing is a secret key that is used to sign and verify messages using an algorithm such as HMAC. In this case, both the client and API need the actual value of the key to create a signature of the message. The server then compares the signature that it generated to the signature that the client sent and is able to verify the message. As mentioned above, if the original key is not reproducible on the server this cannot work. After initially sharing the key with the client (over an encrypted channel) the key does not need to be transmitted between the client and server ever again.

The other common API client secret is merely a secret credential (as you might see in a typical OAuth2 access token request). This credential is transmitted on each request, and therefore should only be transmitted over an encrypted channel (such as HTTPS). In this case the server only needs to retain enough information to verify that the secret value supplied by the client is correct, so the secret can and should be hashed. In this case, the secret is effectively acting as a password, and so the same precautions should be taken.

Disclaimer: I am not a security researcher, and you should absolutely do more research on this and any subject before blindly following any of the advice on this StackExchange.

tl;dr The most important takeaway is that different APIs may use the secret in different ways. It is important to understand how your API is using the client secret in order to evaluate best practices for storing it.

Nick Sloan
  • 271
  • 2
  • 4
  • Excellent answer (+1). I would hate to think what would happen in the case of the first solution, if an attacker was able to dump the database table containing the plaintext secret keys. The second solution seems much more secure to me, being that the hashed key (instead of the plaintext key) can be stored. – mti2935 Aug 01 '22 at 15:10
13

Hashing is not storage; it irreversibly destroys data. We can get away with calling password hashing as "password storage" because when we actually need the password, we have a handy human operator to type it in. Indeed, when we hash the password we do not store the password, but only a token sufficient to verify the typed-in password.

An API key must be stored in such a way that it can be used again in an unattended fashion. The server must really store it, instead of just remembering a ghost of it, because it needs the genuine key to access the API.

Tom Leek
  • 168,808
  • 28
  • 337
  • 475
  • 3
    Is there any reason why an API server can't hash user input (i.e., hash the key)? And compare it to the hashed key stored in the server. Seems to me there's no special reason why keys can't be hashed. – IMB Aug 13 '12 at 11:34
  • @IMB, see my response at the other place where you posted this same question. – D.W. Aug 13 '12 at 18:04
  • @user359996, http://security.stackexchange.com/q/18572/971 – D.W. Feb 08 '13 at 21:47
9

Whilst it would be more secure to hash the API tokens, it presents a usability problem: the tokens are long and random, and would have to be told to the user only once before hashing. This makes it a little difficult to secure them with a hash.

My suggestion for a secure scenario would be the following:

  1. Generate a random salt value and store it in the database. This salt must not be equal to the user's password salt.
  2. Take the user's plaintext password and compute KDF(password, salt), where KDF is a key-derivation function such as bcrypt. Call the resulting value k.
  3. Generate an API key.
  4. Encrypt the API key value with AES, using k as the key, and store the ciphertext in the database.
  5. Discard the plaintext API key and k.

When the user logs in, the webapp knows their password and uses it to compute k, which is then used to decrypt the API key and display it to them.

When the user wishes to use the API, they must send k as well as the plaintext API key, over SSL. The webapp can then decrypt the stored API key value using k and compare it to the given API key. If they match, it's valid. This allows API key use without the app needing to store the user's password.

If an attacker breaches the database, they must crack the user's password in order to compute k.

Polynomial
  • 132,208
  • 43
  • 298
  • 379
  • 2
    And what in case of password change? – Andrew Smith Aug 12 '12 at 23:51
  • Just a suggestion the unique salt can be anything from the user's password or something that remains constant like the userID. Point being, encryption better than plain text anyday. – Rohan Durve Aug 13 '12 at 01:45
  • 1
    @AndrewSmith During password change the user should always be required to enter their existing password, to prevent walk-by attacks. This allows the computation of `k`. – Polynomial Aug 13 '12 at 05:44
  • @RohanDurve-Decode141 In this case, yes, it could be the user ID. However, I normally give the advice to avoid using externally known values as salts in all cases, because the differences are subtle and things get complicated when dealing with password hashes. In this case, though, user ID is fine. – Polynomial Aug 13 '12 at 05:46
  • Yeah, yeah! ...a missing part of the sentence caused that. I meant, "The answer is just a rough suggestion..." and was aimed at Mr. Smith. :) Ps. I didn't downvote. – Rohan Durve Aug 13 '12 at 12:25
7

Putting in my own two cents because of one issue that has not yet been mentioned. In particular, I agree with what @NickSloan has said, but with one more caveat.

There is one more important difference between an API key and a password. API Keys are unique to your site. The part that make password security so critical is password sharing. If someone gets a hold of the password that a user saved on your site, it isn't just your site that is compromised: it is every other site that the user used that password (and email/username) for. Many people reuse passwords across critical sites: banks, social media, email, etc, which means that good password security isn't so much about protecting access to your site as it is about protecting your users on other sites. We can't force users to do security better, so we have to assume the worst and take extra steps for them to protect their own privacy.

An API Key is a completely different scenario because it is unique to your site. As a result, if it is stolen, the attacker gains access to your site, but gains nothing that they can leverage against that person's bank/email/etc. This means that the risk level for API keys is actually lower than that for passwords. They aren't quite in the same ball park: API keys are more akin to session identifiers than passwords.

Understanding that these are more akin to session identifiers than passwords gives some practical insights into how to properly protect such keys. Note that the following discussion is applicable to API Keys for things like OAuth credentials, and not encryption credentials (as discussed by @NickSloan in his answer).

The biggest threats to passwords are phishing attacks and hacked databases, which is why we store them hashed in our databases. The biggest threats to API keys are the same as for session keys: weaknesses in front-end applications such as XSS vulnerabilities, MIM, and the like. It's important to note that the front-end application needs the password in unencrypted form, so hashing the API key doesn't really gain you anything when the person who is most likely to have it stolen needs it in plain text.

Instead, you protect a session identifier (or OAuth token, or client-specific API key) primarily by keeping track of the client. If a session identifier get stolen via XSS or other means from the client, a typical result is for that identifier to be used by the attacker on a new device. That means that all of a sudden you will see an old key coming up with a new user agent. Such a change is easy to detect and fix: immediately invalidate all API keys. Especially in the case of OAuth requests, this is very painless for the user: they just log back in and immediately get a new one, while someone who only stole the key (and doesn't have the password) is back to the drawing board.

This is a common enough defense measure that a smarter hacker will also try to record the user agent associated with the stolen key and use it in all future requests, but you can still easily see something fishy going on when the same user has requests coming up from multiple IP Addresses. Like everything else in the world, this can be a bit of a cat-and-mouse game, but there are some important take-aways:

  1. An API Key/OAuth Token/Session ID is really just a way of remembering a user on a particular device, so they don't have to put their password on every page. As a result, when in doubt, you can just kill all authorized keys and force the user to log back in.
  2. In all three cases, since losing Key/Token/ID leads to an account being compromised, it is important to have some active security around these things and detect/respond if anything fishy happens. If you've ever gotten notices from facebook/google/etc about someone trying to login to your account from Mexico, it's because a security guy somewhere is doing their job right.
  3. The primary attack vectors for Keys/Tokens/Ids are different than those of passwords. You may still want to hash them in your database (again, we're not talking about encryption credentials which need to be in plain text), but you can't just hash them in your database and call yourself secure. You need to make sure and secure them against a whole new range of attack vectors. If you think of them as passwords only, you'll miss some important parts of the big picture.
Conor Mancone
  • 29,899
  • 13
  • 91
  • 96
  • I agree with your view of reduced risk for an API token when compared with the password, from a user's perspective. From a businesses perspective, the risk remains the same. You have stored all your API bearer tokens (akin to a psuedo random string of certain length) in plain text. when this db is stolen or data exfiltrated, a dos attack can be mounted against your enterprise and bring down your platform/services. So i am more inclined to hash and store these bearer tokens as hashes. As discussed by @NickSloan in case 2. – Raj Mar 31 '21 at 15:43
  • @Raj By all means, go for it! We all have different internal threat models/risk acceptance levels. Security does not work from a "one-size-fits-all" approach. – Conor Mancone Mar 31 '21 at 19:21
2

One of the main reasons passwords are hashed before storing them, is to protect against an attacker gaining a read-only access to the database. In this way, if an attacker gets hold of the login credentials list, they still won't be able use them directly because the passwords are hashed/salted.

And this is not just a theoretical problem we're trying to solve here, these type of attacks really happen (including to major players in the industry), and proper hashing does help mitigate the impact of an attack.

If you're lettings users authenticate using an API secret then the API secret has effectively became a password therefore, from a security standpoint, they should have the same protection mechanisms as regular passwords have.

Amazon itself now doesn't allow retrieving the API secret. If you forget it, you have to ask for a new one.

Pedro Rodrigues
  • 121
  • 2
0

No it's not okay for some kind of services.

Implementation

1. Upon user connection with login password, generate a key derived from his/her password to encrypt data for this user and keep it in a session on the server.

That way, each user will have a unique key. This derived key must not be reversible, this is the main point. Using hash_mac or other hashing method with salt and/or master key will allows this.

2. Use this key to encrypt the generated API Secret using AES 256 with a random iv for each user.

Or

2.bis Use this key to generate the real encryption key at the very last moment and encrypt the API method using the same way as in 2. to avoid in memory floating passwords.

3. Store the iv in the data base along with the user id and app id in a dedicated table

4. Store the encrypted API Secret in the database in the appropriate table.

Usage

Now when the user connect to his control panel you can recreate the key, fetch the iv, decipher the API Secret for this user and display it in clear text. You also display the key and ask for both when making API Call so that you can verify the App Secret.

If you want to avoid to display the key and not have to request it on API call you can also for authentication use the same method as for login. Meaning you can create another table to store a random key/salt for each client app and store a hash_mac version of the API Secret. Thus only the API ID and API Secret are required to authenticate API Request. but it's more work.

Note

This is practically the same as @Polynomial answer.

This way it's very very difficult for someone hacking your server and database to steal your users credentials and make api call on their behalf. You can now assess that if a request has been accepted from a API client it's the right client or it's their leakage.

If the user loses his password then you regenerate a new API Secret and Key from his new password.

Nicolas Manzini
  • 103
  • 4
-2

Well in this backend you can also provision server and read all the data anyway. You need to make sure that nobody can access your backend, otherwise he will read all of your data at once including backups.

Andrew Smith
  • 1
  • 1
  • 6
  • 19