Protection of Keys/Passwords on Virtual Hardware (XEN, KVM, VMWare, etc.)

Question

Is there any reasonable way to secure data on virtual hardware? What I mean by "virtual hardware" is the various hosted systems that one can rent from various online hosters, that run on top of XEN, KVM, VMWare ESXi and so on.

With actual dedicated hardware one can somewhat safely assume that the system itself has not been compromised if:

1. the software can be assumed as safe, that is the software running on that machine (ssh, webserver, etc.) can be assumed to not have been exploited or brute-forced
2. the hardware can be assumed as not-compromised, that is the machine hasn't gone offline so that an attacker could've changed essential hardware-parts like HDDs/RAM/etc with compromised ones. (Yes I know about hot-swappable hardware, but let's assume that will leave sufficient traces, be it in logs or software-malfunctions, if it's not done by a legitimate authority.)

But with virtual hardware the assumption that the hardware hasn't been compromised doesn't hold any more, because the hoster can pause, copy and restart the machine at will, without it ever rebooting or even going offline. Furthermore the hoster has direct access to the entire virtual machine's RAM and disks - while it is still running.

With some effort the hoster might even be able to change the authorized_keys or /etc/passwd file on the System to gain (root-)access to it and thus possibly even be able to view the contents of encrypted partitions that are currently mounted.

Given these vulnerabilites or possible attacks my questions now are:

1. How realistic are these vulnerabilities, are they an actual threat or just "science-fiction"? Is there anything that makes them impractical in reality? Do the VM-software vendors (Citrix, VMWare, etc.) implement any security mechanisms that make these attacks more difficult? (Is there even anything they can do to make this harder, other than security-by-obscurity)? Or do the VM-vendors maybe even provide tools to the hosters (which after all are their direct clients) that make these attacks easier?

2. Is there any effective way a guest-os can protect itself or the data in the RAM and on the disks from this kind of attacks? Would trusted computing approaches (signed code, etc.) be of any help? I know about DEP and ASLR, but that won't really stop the host from still reading every bit of memory available. Also I guess RAM-encryption is out of the question with conventional x86 hardware/VMs .. since that would no longer be "random-access" memory any more.

3. Has there been any research on this? I'm surprised about how little discussion there is on this topic. Is everything so obvious or unsolvable already, or is there just too little interest yet?

Answer 1

To me it seems clear, just by considering hypotheticals, that a virtual server has no practical technical protection from the administrator of the real host.

I'll try to be specific. It doesn't have to be impractical to crack a virtual server, given control of the real host. Here's the kind of strategy that I would imagine. Disassemble the virtual machine host software and locate the virtual keyboard driver. Then modify, not necessarily the VM host software, but perhaps a dispatch table in the operating system kernel. In that way, some sequence of system calls that is characteristic of the virtual keyboard driver could be made to trigger a logger for the virtual input keys. The result is a keylogger for the virtual server that can't clearly be detected from the virtual side.

A host system that would prevent this kind of manipulation probably wouldn't work very well. It's hard enough to debug and monitor software without intentionally adding obstacles for analysts.

Answer 2

As far as resources, I'm sure a google search / scholar search can yield many results about the theoretical and practical security implications of virtualization. Here's one I found (but there are many more): NIST Full Virtualization Security Guidelines

Yes, virtualization opens up a lot of attack vectors that are less accessible (but still possible) when you run straight on the hardware. The same principles could be said however about writing your entire code to run off the hardware, instead of using an existing operating system. Do your own low-level memory access and I/O and process management to avoid the chance of some kernel patch or a trojan having access to the same memory segment that your program does.

There is of course a question of trust too. If you can't trust your hosting provider, then I won't trust them with my hardware either. Do your trust your operating system? Do you trust all your 3rd party application code and modules?

Lastly and most importantly in my opinion, as almost everything in life, it's a trade-off between your risk and your benefits. Do you want all the many benefits of virtualization and virtual hosting "in the cloud"? (launching virtual servers progrmattically, scaling servers up/down, taking snapshots on the fly, cloning virtual machines, having simple console access to each server without the need for extra hardware, adding virtual network interfaces to create complex network setups to name a few) or you prefer having less flexibility but some more security*? The choice is of course yours. And it depends on your perception of risk vs benefit.

---

*Some would argue that virtualization can help increase your availability, which is of course also an important part of security.