API message verification without storing private key?

Question

I'm in the process of implementing a public API for my web service. Security is a key concern as money is involved.

It seems the current common practice for "authenticating" users in API requests is by confirming message authenticity, having the user send along a HMAC of the parameters (including a unique nonce), for each request.

This is fine, but it uses a shared secret: both the server and the client have to store the private key in order to generate the HMAC.

I don't like this. What's the point in going to all the hoopla with salting/hashing passwords, if I'm then going to start storing unencrypted shared secrets in the database?

Is there a better way? I was thinking of using GPG, and having the user provide their own GPG public key for me to store; they could then sign messages using their private key. However, I fear this might be a bit of a pain for Windows users to play with.

In general, isn't public/private key signing a much better way to do this? There must be some reason why everyone is choosing HMAC with a shared secret... what am I missing?

I will of course set limits on customer interactions via APIs, provide notifications and limit removing money from the system, but a great deal of damage could still be done if the shared secrets were leaked.

Answer 1

I asked the same question just yesterday, but in the meantime I have found what I believe will answer both of our questions: SRP. This question has the best answer I've found so far. Here is a reference to SRP. The main problem with SRP is there are no modern implementations that can be used at the application level.

Edit to add more information from my research:

I have decided to use a database backed session store, so it still meets the definition of stateless. This requires SSL to be the only protocol for communication, which eliminates MITM and replay attacks.

• Use bcrypt to store passwords in DB
• use Two Factor Authentication, such as Authy or Google Authenticator - this does not meet your requirement of no external dependencies, but in my case I'm ok with that. It prevents a user leaking his password - attckers still need the user's TFA device.
• Have a session ID and Session length (in secs) field in each user record
• Upon successful authentication (typical auth, user sends userid and password, use bcrypt to hash password using stored hashed pw from db), compare hashes, etc)
• Upon successful password, generate cryptographically sound guid, and store in db, along with how long you would like to keep session alive
• send this session id as response to auth
• if you want true statelessness, require client to HMAC hash every request with this ID - he can keep the ID, he doesn't have to send it to you
• verify request by hashing request with stored sid, compare to hash he sent you
• on each request, verify that expire-time hasnt passed - you dont need a process that checks expires, just check it each time the client attempts an API call.

I have compromised and used express-js's secure session storage, which DOES use a client side cookie, but the cookie is signed with a shared secret that all of my application servers know, so it meets the stateless definition in that the cookie is not unique to the server the client is connected to, since the cookie is still in my app server's domain

Answer 2

Public key cryptography is slower and requires more resources to obtain the same cryptographic strength as a smaller shared secret.

See: https://stackoverflow.com/questions/4299795/why-public-key-algorithms-are-slow

This probably won't be a problem if you're building a low-volume system, but once you get to large scale systems, you might have to justify the extra cost to your organization and your customers/clients.

If you're worried about storing symmetric secrets, you should probably encrypt them anyways.

Answer 3

user26802 is correct, the answer is speed. Imagine an API like Amazon SQS where clients are dropping off and picking up lots of small messages at high rates, having a slow encryption on the client and slow decryption on the server would slow down that service considerably.

You could imagine a blended approach where clients made a call authenticated with private key signing to establish a short-term symmetric key. If your keys were compromised you would only have a short window of vulnerability before keys expired. Now if the attacker has persistent access to your system, you're no better off.

Another thing to keep in mind, if an attacker has compromised your key-store and can steal all of the signing keys they may also be in position to replace a public key and still be able to create requests as another user.