1

I have a number of seemingly contradictory requirements:

1) We obtain content for users from third parties; content for certain users should be stored encrypted at rest for compliance reasons

2) The user should be able to decrypt and display their content via a secured webapp/api

3) There are many users on a given account, and all should be able to decrypt and view each other's content (implying a shared key)

4) A backend worker process should be able to decrypt and reprocess the user's content at any time, if we want to (for example) improve our analytics

If we had to, we could sacrifice #4 (or perhaps have the user authenticate to allow it.)

Naturally I don't want to store the encryption key in plaintext anywhere. It would be most convenient to store it at the user level, encrypted by a hash PBKDF2'ed from their password - but that introduces a cumbersome workflow if they forget their password and need to reset it (since we now no longer can derive the key.)

I can't just use the password either, because access to content that was encrypted already should survive change password requests. Also, of course we don't have access to the plaintext password in async backend worker jobs.

Is there a generally accepted way of doing this? E.g. could I use some kind of 2FA-ish workflow to retrieve a user-secret that I never store locally, but which is used to protect the key?

fridgepolice
  • 183
  • 5

2 Answers2

3

The usual approach is to use a master key for each account that does the actual encryption/decryption/integrity check, but that is never stored in plain text anywhere. This master key should be generated using a secure random number generator and there should be no way to retrieve its plain-text value.

You encrypt the master key with per-user keys derived from the user's credentials (you can use a key derivation function - use something stronger than PBKDF2 if possible, like scrypt or argon2 - on the user's password, or require that the user have some other sort of cryptographic key such as a TLS client certificate). Each user thus stores their own encrypted copy of the master key for their account. These encrypted master keys can be stored on server, because they are useless without the per-user keys to decrypt them. The per-user keys are never stored on the server in any form; they are either supplied by the client directly, or ephemerally derived from client-supplied data (such as a password), used, and then deleted.

If necessary (for your #4) you can also encrypt each account's master key with other keys, such as for your background worker process. Of course, then you've just passed the buck one step; where is the key / credential-that-generates-a-key stored for that worker? How is it protected such that an attacker can't just grab it, decrypt the corresponding master key(s), and go to town?

Password reset does become somewhat tricky. Merely changing a password is easy: get the old password, decrypt the master key, re-encrypt it with the new password, and save the newly-encrypted master key back to the database. Resetting a password when the old one is unknown is harder. The usual options there (aside from "you can't, not without losing all your encrypted data", which I assume is unacceptable) are to give the user one-time-use recovery keys that are way too long to be used for normal passwords but can be used to decrypt the master key at need (meaning you'd have one version of the master key encrypted with the standard per-user key, and one with the user's recovery key), and require one of those for a password reset, or to require that some other user authorized to access that data vouch for the user who is resetting their password, and use the vouching user's credentials to unlock the vouching user's copy of the master key, then encrypt it with the resetting user's key and store that newly encrypted master key (since you have multiple users per "account", this would work well enough as long as no accounts have everybody forget their passwords at the same time).

CBHacking
  • 40,303
  • 3
  • 74
  • 98
1

How about a dedicated encryption service / demon running on your web server?

The service would take a master password (known to you) and would need reentering on server reboot. You can the use a key derivation function to create an in memory encryption key.

Data can be then be encrypted or decryted at will by the service which is consumed by the web app and the back office worker process.

ste-fu
  • 1,092
  • 6
  • 9