2

Due to new laws in my country, any personal information must to be stored encrypted in the database. So, to be able to perform querys over this data I was thinking to keep an in-memory copy of these tables with decrypted data.

For example, to get everyone that are more than 18 years old will be very hard if the DateOfBirth column is encrypted.

I'm using aspnet.core on a linux machine.

Beetlejuice
  • 123
  • 4
  • 1
    "safe" from what? Who/what else would have access to it? – schroeder Jun 25 '20 at 13:40
  • 1
    Since the root of this question is legal, you really should be trying to determine how to meet your business needs while complying with the law. That likely means consulting a lawyer in your country. – Steve Sether Jun 25 '20 at 13:48
  • @SteveSether I'm not seeing this as a legal question. Due to the laws requiring the database on disk to be encrypted, can a copy in memory be "safe", not can it satisfy the legal requirement. – schroeder Jun 25 '20 at 13:54
  • @schroeder I think the underlying question is legal, even if it was stated as otherwise. The driving force behind this is legal, not security. What's PII? In an index PII? These are all legal questions, and getting a handle on them will the user solve the underlying intersection of legal, and business requirements. – Steve Sether Jun 25 '20 at 17:25
  • It's a legal question and comes down to the definition of a "database". You can have a database in multiple forms of storage, cloud storage, server farm storage, local hard disks, SSDs, slow DRAM, faster L1/L2/L3 caches, register memory. Any of these can be "snooped" by a sufficiently hacked machine. – Mark Lakata Jun 25 '20 at 22:15

2 Answers2

3

The short answer is: Just don't.

The long answer is: It depends on the rest of the system.

As ram0nvaldez said, you have to decrypt the data to work with it. Some level of decrypted in-memory is necessary. But!

Are you storing your cached DoB data in a Redis cache? Redis writes its cache data out to disk on a regular basis to facilitate fast restarts. That would need to be encrypted, too.

Is swap encrypted? If it's not, then any of your cache that gets swapped may break compliance. At the very least, it opens the door to a lot of explaining why it's not a problem.

Not to mention storing all that cache creates new problems for your app: memory pressure, cache coherence, and scalability among them. New devs on the project need to be told to use the cache-search object, and not instinctively write direct database queries. Is it just DoB? How many other columns in other tables need the same? Really, you're better off finding a way to make the database do the work.

AES-NI instructions in modern CPUs make short work of encrypting and decrypting data. Try to copy a file over the network with scp -o Compression=no and you'll likely see it copy over at wire speed, thanks to AES-NI. Encryption simply isn't the bottleneck it used to be. The database can use those instructions to decrypt the index and the data page-by-page, not row-by-row.

If the application has to fetch and decrypt each and every row to find all the records with age >= 18, then you may as well cache it as you'll be pulling the whole table into memory with every query anyway.

Leveraging the encryption features of modern databases makes it easier to be compliant with less overhead.

1

In any case, all data will be unencrypted to be processed, once finished the processing will be encrypted again in storage.

The regulations mainly have to do with storage, so you are good. Remember there is no way to not decrypt the data to make computations.

To decide how long or in what circumstances keep the data unencrypted, you may need a risk assessment, since there is a vulnerability (keep the data unencrypted in memory), you need to identify if there is a threat to the data, that may determine if there is a risk or not, once you know that, you may assess the likelyhood of the risk to occur and the impact should it occur.

The results may let you decide how long may the decrypted data be and stay in decrypted status in memory.

ram0nvaldez
  • 204
  • 1
  • 2
  • 9
  • 1
    "The regulations mainly have to do with storage" -- which regulations? Are you sure they don't also affect data in memory? I'm not sure you have enough data from the question to make this claim. – schroeder Jun 25 '20 at 13:55
  • "To decide how long or in what circumstances keep the data unencrypted ..." if the OP is talking about HIPAA, then the standard to use is not a risk assessment but “reasonable”, “appropriate”, and “equivalent” controls. – schroeder Jun 25 '20 at 14:03
  • I'm not sure about it, the country where this applies is not disclosed, but regulations "mainly" address the storage data, so I'm not taking it for granted, but it is likely to be the case. Maybe I didn't explain myself, sorry for that. – ram0nvaldez Jun 25 '20 at 14:04
  • The conclusion "so you are good" is too definitive based on available data. I'd add some qualifiers there. – schroeder Jun 25 '20 at 14:05
  • well, @Beetlejuice tells us that "... any personal information must to be stored encrypted in the database", so if he is encrypting the storaged data, he's in compliance with that requirement. – ram0nvaldez Jun 25 '20 at 14:13
  • Invariably, some time, data will be in memory, for processing or just for be sent to presentation. So keep this data in memory was my first idea. How long? I don't know yet. Maybe while serve is on. – Beetlejuice Jun 25 '20 at 16:58
  • But encryption at rest is not the question and neither is it if storing in memory is compliant. And since we still don't know what regulations we are working with, and their scope, your blanket statement is too broad to be made. – schroeder Jun 25 '20 at 20:20