6

A cloud operator such as Google can take a snapshot of a normal VM. This includes CPU state, RAM and disk. This can then be copied to another physical and resumed there. Or it can be analyzed off-line, and any cryptokeys in memory or in the CPU state can be extracted.

This means that if you do not trust your cloud VM provider (maybe your cloud VM provider is owned by your worst competitor), you should not process confidential data on those VMs.

https://cloud.google.com/confidential-computing seems to use AMD's Secure Encrypted Virtualization which includes hardware RAM encryption: https://developer.amd.com/sev/

If the RAM is encrypted, it will make it harder to use attacks like https://rambleed.com/

But will it also protect against Google?

It seems the RAM is encrypted with a key, that lives in the CPU. But is this key included when Google takes a snapshot of the CPU state of the VM?

In theory I could see it work like this: The CPU has a small web server with a TLS certificate signed by AMD. I access the web server, verify AMD's certificate, and now I have a secure connection to the CPU that Google cannot access.

Then I give the CPU a secret key to encrypt RAM with. Then I give it a disk image encrypted with the same key. Then I boot the VM.

If the secret key physically cannot leave the CPU, then it should be impossible for Google to access my data: The RAM is encrypted, data to the disk and to the network is encrypted. So I do not need to trust neither the RAM, the storage, nor the network. It will, however, also mean Google cannot snapshot my VM and restore it on another CPU.

This would also mean that this answer is outdated: https://security.stackexchange.com/a/215927/84564

Currently I see no way to do something similar to verifing the AMD certificate in Google's current solution. And thus I see no way to securely set a key that Google does not have access to.

Can Google take a snapshot of a running confidential computing VM and restore it?

Using AMD's SEV can CIA safely process their most secret data on North Korea's Confidential cloud (assuming they have that) without North Korea being able to access the data - assuming that AMD is trustworthy, but all other hardware apart from the CPU is made in North Korea?

Ole Tange
  • 301
  • 1
  • 9
  • What if google backdoors it before you have set all this up? – yeah_well Dec 01 '20 at 16:39
  • @VipulNair Where would you (Google) put the backdoor? Let's us assume Google does not change the AMD CPU. We can assume all other hardware than the CPU is "evil". – Ole Tange Dec 01 '20 at 17:33
  • @reed I think I can answer all of those by watching the first half of https://www.youtube.com/watch?v=0-ISmJNxGiY (especially 9:26-15:30). Have you watched that? Did you understand it? – Ole Tange Dec 05 '20 at 03:04
  • @OleTange assuming that OS will be stored in disk,when the disk is unencrypted,whats going to stop google from backdoorinng it when they have the physical access to the disk? – yeah_well Dec 05 '20 at 06:59
  • @yeah_well You have to distinguish between the hypervisor and the guest VM's OS. The hypervisor is untrusted. See the section on hypervisor in my answer. – Ole Tange Dec 05 '20 at 11:39
  • 1
    Ok, I watched part of the video and thought about this a bit more, although I'm not sure I understand some important details yet. Correct me if I'm wrong: all your data (i.e. guest OS images, applications, etc.) never leaves your local machine unencrypted, right? Everything is encrypted with keys that, ultimately (at the end of the chains), can be trusted because they are signed by AMD that guarantees that only one of their CPUs can decrypt your code. If so, then you really only have to trust AMD and no one else. – reed Dec 05 '20 at 12:13
  • @reed Pretty much. The keys are either (chain)signed by AMD or the cloud customer, or they are generated in the CPU and never leave the CPU unencrypted. – Ole Tange Dec 05 '20 at 12:24
  • I think there is a big and important disconnect between your question and your title. Your title is "Can google access data in their VMs" to which the answer is clearly "Yes". However, it seems that your actual question is "Would this tool from AMD make it impossible for google to access the data running in their cloud?". That is a completely different question. Which are you trying to ask? – Conor Mancone Dec 11 '20 at 14:38
  • @ConorMancone Google's **Confidential Computing** VMs *are* VMs which are meant to be run using AMD's Secure Encrypted Virtualization, so I fail to see how those are two separate questions. – nobody Dec 11 '20 at 17:48

2 Answers2

2

I found: https://www.youtube.com/watch?v=0-ISmJNxGiY

Watching 11:00-15:30 it seems my theory with the internal web server is pretty close to how it is designed.

So this confirms that it is indeed possible. This is HUGE: In time we will be able to run on untrusted hardware as long as we trust the CPU. So this answer is indeed outdated: How to prevent a hosting company from accessing a VM's encryption keys? The answer is not entirely wrong, because a dedicated attacker could open the AMD CPU and attach wires inside the CPU, but it would make the attack much harder - especially if AMD actively made the CPU tamperresistant.

But since Google does not seem to provide the information from the slide at 15:00 (i.e. I do not get the PDH and do not get to talk directly to the CPU) it seems Google still has control over the server: They have an (undisguised) Man-in-the-middle, namely their control panel, which could be lying about your VM being spawned on an AMD CPU.

This leads me to the conclusion that Google Confidential computing VMs are probably well guarded against attacks from other VMs running on the same server (like https://rambleed.com/), but they are not currently guarded against an attack from Google.

If, however, Google gave me access to the PDH this could change.

With the certificate from the AMD Secure Processor I can set up a secure channel to this unique CPU using Diffie-Hellman (the chip endorsement key https://youtu.be/0-ISmJNxGiY?t=728 and the platform Diffie-Hellman https://youtu.be/0-ISmJNxGiY?t=660) - pretty much like https, where you can also check the certificate is from the correct website by following its chain of trust.

The hardware outside the CPU will thus be similar to the internet when talking https: You do not need to trust the hardware (just like you do not need to trust the internet), because it only sees encrypted data.

Said differently: With normal computers we have no problem connecting them to internet that we do not regard trustworthy: We use encryption, so that anyone spying on the data going into and out of the computer will only see encrypted data. We can still check that we are communicating with the correct recipient using digital certificates.

AMD SEV just moves this limit inside the CPU: Everything going in and out of the CPU will be encrypted. And a cloud customer can check he is talking to the unique AMD Secure Processor by using the digital certificates.

The hypervisor (which is the first OS that boots on the machine and which is used to provide internet access to the AMD Secure Processor) does not have to be trusted. The hypervisor is isolated from the VM: The hypervisor can read and write the encrypted memory, but not the key for this encryption (https://youtu.be/0-ISmJNxGiY?t=86). A bit like VPN: You can have an untrusted connection carry a trusted connection on top of it.

The hypervisor can control VMs, but it cannot set the encryption key for the VM. A bit like an ISP who can throttle your internet connection without being able to decrypt your VPN.

The trusted VM starts by having a unencrypted, untrusted BIOS (https://youtu.be/0-ISmJNxGiY?t=563). This BIOS is then hashed and encrypted by the trusted CPU, and the hash is given to the customer. This way the customer will know if the BIOS for the VM is changed. If the customer accepts the hash as correct, he will then inject a secret (e.g. an encrypted disk image or a key to open an encrypted disk image stored on untrusted storage) (https://youtu.be/0-ISmJNxGiY?t=639), and then the VM boots in the encrypted RAM.

Summing up the similarities between https and AMD SEV:

Feature HTTPS SERVER AMD SEV
Untrusted features
Fast storage Full disk encrypted SSD Encrypted RAM
Slow storage Encrypted remote backup Full disk encrypted SSD
Control channel ("network traffic") https encrypted Diffie Hellman encrypted
Trusted features The whole server AMD Secure Processor
Certificate TLS-certificate for https certificate for secure processor
Certificate authority e.g. Thawte AMD
Key for fast storage full disk encryption key RAM encryption key
Key for slow storage backup encryption key full disk encryption key
Key storage small boot disk secure encrypted NV-storage

Would I recommend CIA do their confidential data processing in North Korea's confidential cloud (assuming they have one)?

No: AMD's SEV makes it much harder for an attacker. An attacker cannot simply take a snapshot and use that. He can, however, still corrupt memory, and if he is willing to open the CPU he may be able to bypass some security by attaching wires. But these attacks are out of reach for a normal cloud hosting provider.

So it does not protect against a truly dedicated evil hosting provider, but it does seem to protect against a sloppy hosting provider, who does not have major resources for attacking the customer's data.

Bruno Rohée
  • 5,221
  • 28
  • 39
Ole Tange
  • 301
  • 1
  • 9
  • You're badly over thinking it. Can google access data in their confidential computing VMs? Absolutely, because it is their computing VMs. This is like asking, "If I hand you a box to put in the safe at your house, do you have access to my box?" Unless you are encrypting your data with a key that is never sent to Google's servers, they could access your data if they *really* wanted to. – Conor Mancone Dec 04 '20 at 18:25
  • @ConorMancone You use a bad analogy. Please show how Google would get the key if I can communicate directly with AMD's Secure Processor. – Ole Tange Dec 05 '20 at 01:42
  • 1
    @ConorMancone the OP proposes public private keys for communicating with the said box that will be signed by third party(AMD).How would google tamper with that? – yeah_well Dec 05 '20 at 07:01
  • @yeah_well It's as secure as the key, which is sitting inside Google's datacenter. Granted, Google would have no motivation to try to extract the keys if such a thing was in use, but could they if they wanted to? Probably eventually. There have been plenty of cases of keys being extracted from hardware that was designed to keep it secure. Here's the first of many examples from google: https://www.nccgroup.com/us/our-research/private-key-extraction-qualcomm-keystore/ The point though is that if Google is a "normal" cloud hosting provider then these attacks are not out of their reach. – Conor Mancone Dec 11 '20 at 15:04
  • @ConorMancone So basically your argument is: If there is a design flaw in the CPU, Google _may_ be able to use that and extract the key. Basically the same argument I include in the section on CIA's use of North Korea's data center. – Ole Tange Dec 11 '20 at 20:15
  • @yeah_well Google might simply buy AMD. Or the NSA might suggest AMD and Google to cooperate – Hagen von Eitzen Jul 03 '21 at 09:53
-1

TL;DR: No, it's impossible to secure. They own the computer that does your work. They could have modded it. They control the network access. They can read your TLS communications because they're the recipient.

Edit: The OP wanted information about future computing tech and whether this may become true later on. It's simply not feasible. At some point you would need to trust someone that they are not lying about your data being inaccessible to them. Whether that is the CPU maker, the hosting provider, or your ISP to get there, you have to assume that they are giving you a key to a machine that does what they say it does. You can't guarantee that, short of actually obtaining one and then using that specific machine. Even then you'd need as much equipment and knowledge as the hypothetical attacker, because they can hide it at least as well as you can find it. And at that point, your test machine probably wouldn't survive the discovery anyway, so you'd need a new one...

From the linked AMD site, the key is generated in hardware... if it's enabled in the BIOS. Otherwise, the OS is at minimum involved in the process, and possibly even creates that key (during which time it's vulnerable) and talks to the memory controller to encrypt the data directly.

However, with the case of virtualization, the hypervisor (and the OS) interact with the memory controller and its collection of keys. I'm not certain of this, but it does sound like it has been configured such that the hypervisor can't actually see the keys, merely say which ones to use (and they're generated in a CPU register that can't be imaged with the VM).

That sounds awesome, but would I trust it? Maybe a bit more, but it's still not necessarily safe. The key isn't stored in RAM, and may not be accessible to the OS or Google, but how do you know that they are actually using the system at all? There almost have to be vulnerabilities in the system (it's new and complex), or maybe Google performed hardware mods of some sort (they physically hold the machines, and you can't see them). Maybe it's even possible to physically halt it like with Intel's ME engine, so it thinks it's running but really does nothing, or set up a man-in-the-middle (MITM), where another CPU is used to get proper responses and your request is run on a regular VM.

Just as you would before this technology, you have to actually trust Google to "not be evil." If you're that concerned, perhaps you should also wonder if AMD has a way to extract those keys "just in case" and it's not just (or only) Google that you'd need to be concerned about. Remember that Intel ME thing I mentioned? It's an always-on backdoor into all of your computer's hardware. AMD has one too, and it has even more tendrils into your system and is harder to disable. It's almost certain to be able to talk to that encrypted memory controller. Google has access to the BIOS ROM that runs the software that does the encrypting, for that matter, simply by owning the machine.

And, lastly, they control network access to the VM. They can try to attack the software and commands you send it or that it downloads. They can MITM your encrypted connection (they OWN the certificate that you're using to connect securely). They may be able to use unpatched bugs in the OS that's running on that "secure" VM. Why bother to use a hardware attack if your app is running in Flash on Windows XP (I really hope not, but that's an example)?

The rule you mention still applies, even if you get a fancier computer. Google has full access to that machine. It does not matter what you do with it; there is no way to keep that sort of scenario safe, especially against a hypothetical adversary with as much knowledge, hardware, and money as Google.

RDragonrydr
  • 491
  • 1
  • 3
  • 6
  • If we assume Google is not trusted, and AMD is trusted, and that the customer will be able to communicate directly with the AMD Secure Processor (which it seems Google does not allow currently), could you extend your answer with how Google will be faking AMD's signed certificate (https://youtu.be/0-ISmJNxGiY?t=660)? – Ole Tange Dec 05 '20 at 01:39
  • In your answer you make some wrong assumptions about AMDSEV: That the hypervisor is trusted (it is not), that the BIOS is trusted (it is not), that you cannot avoid a man-in-the-middle (you can). I cannot really blame you, because I made the same assumptions before watching youtu.be/0-ISmJNxGiY I have re-watched it 10 times now, and EVERY SINGLE TIME I have understood yet another detail. May I suggest you re-watch it until you can explain every word on the slides and every concept presented? – Ole Tange Dec 05 '20 at 12:57
  • I am not very clear on how exactly the user communicates with the processor, but couldn't Google just echo your commands to it to a mirrored VM that is using that CPU? If you can't query it at all, though, the point is a bit moot. My main point, however, was that you'd have to trust Google when it says that you are actually using that CPU in your VM, and you also have to trust that all the other security to and from that secured processor is perfectly safe. That's what I was aiming at. I wouldn't think that just a secured processor and memory will help when it's Google configuring the VM. – RDragonrydr Dec 06 '20 at 04:05
  • As it is today, Google can indeed do that, and that is why Google's current solution is not confidiential. But if Google lets you talk directly to the AMD Secure Processor (and not just through their control panel), then you can setup up an encrypted connection. The Diffie-Hellman key exchange (seen in the slide at https://youtu.be/0-ISmJNxGiY?t=775) not only encrypts the communication so that Google cannot spy on it, it also gives you the PDH which is unique for each CPU and is signed by AMD, so you know Google did not just invent their own, and you know only one CPU will be able to decrypt. – Ole Tange Dec 06 '20 at 08:28
  • Google will still carry the conversation to the AMD Secure Processor, but they do not have to be trusted. Just like your ISP does not have to be trusted, when you are using a https website. – Ole Tange Dec 06 '20 at 08:31
  • Hmm. I'm not actually sure, then. If that's the case, then maybe I should remove this answer, since it might be inaccurate. – RDragonrydr Dec 07 '20 at 16:34