On the client side you could use PBKDF2 or a similar key derivation
function to generate a key to then use to encrypt/decrypt the
symmetric key before using it. Would this be considered (acceptably)
safe?
This really depends on a few things:
1. What is the threat model?
Who are you trying to prevent from attacking you? A nation state? A jealous significant other? A script kiddie? Will this person have physical access to the client's device? Etc.
2. What are you using as an inputs to your KDF (PBKDF2 in your example)?
Hopefully the input to the KDF is something that is not persisted on the device (except for the salt and rounds, you can leave those hanging around) but rather something of sufficient entropy that the user is forced to enter once (e.g., a passphrase) and then is wiped from memory once it is no longer needed anymore.
3. How are you encrypting and storing the keys?
If trusted dedicated hardware is available (e.g., TPM or HSM [assuming you trust those]) then those will provide you with a way to store your symmetric keys (some store them on the dedicated hardware and some encrypt them with a key that is stored on the hardware but store the resulting ciphertext on your storage device). If no such hardware is available then you will usually store the keys in a keystore which is basically a file (or set of files) that stores the keys in an encrypted manner. There are several implementations already out there but if you are going to roll your own (which I do NOT recommend) then you need to consider how you will protect the confidentiality and integrity of your encrypted keys. Some things to consider are:
- Will each key to be encrypted be wrapped with a different KEK?
- How will you derive each KEK?
- What algorithms/modes will you use?
- What kind of performance do you need?
4. What do you do with the keys in memory when you don't need them anymore?
Every copy of each key should be wiped from memory once they are no longer needed. There are debates on how to do this that range from just writing a bunch of zeros over the bytes in memory to doing more fancy things that involve multiple passes that each write different bytes to the key's area in memory. What you do should depend on your threat model.
What possibilities are there on the server side? What procedure would
be considered safe to store such a key in a database?
The server side has very similar options. Ideally the server would use an HSM and store the keys in there (see #3 above for more details) and your server side application would never have access to the raw key bytes (only the HSM would). In the database you would just store the key's alias (you could use the HSM to provide integrity on the binding between the database record and the alias).
How does for example Amazon store their symmetric keys used for their
query string authentication for S3?
I am not sure what Amazon does specifically for query string authentication for S3 but I do know that they provide a Cloud HSM service.