How private is RAM from other users on a VPS?

Question

Can I safely assume that my RAM never can be accessed by another user on e.g. EC2 or Digital Ocean, if we suppose that I trust my provider and we don't consider possible bugs (such as Heartbleed) in my environment.

Answer 1

tl;dr: Virtual machine RAM is reasonably private, given your stated assumptions.

I'm not an expert on virtual machine security (real experts, please come correct any errors in this post), but I found this question interesting enough to do some Googling. Here's what I found.

From a NIST publication titled Security Recommendations for Hypervisor Deployment:

One of the common threats in virtualization platforms is a malicious VM accessing areas of memory belonging to other VMs. This is called a VM Escape attack. Processors with virtualization extensions provide safety against this through features such as Direct Memory Access (DMA) remapping limiting DMA access to what is valid for the VM (i.e., preventing a VM from performing DMA beyond its allocated area).

This paragraph is saying that hardware-accelerated virtualization (which is pretty much all virtualization you will encounter in 2015) can enforce memory access restrictions pretty well--at least better than software-only virtualization.

The NIST paper goes on to suggest the following:

Security Recommendation HY-SR-6: The ratio of the combined configured memory of all VMs to the RAM memory of the virtualized host should not be very high. A typical ratio adopted is 1.5 : 1. In other words if a virtualized host has 64GB of RAM, then the combined configured memory of all VMs running on it should not exceed 96 GB.

Security Recommendation HY-SR-7: The hypervisor should have configuration options available to specify a guaranteed physical RAM for every VM (that requires it) along with a limit to this valu, and to specify a priority value for obtaining the required RAM resource in situations of contention among multiple VMs.

Popular virtual hosts, like EC2 and DigitalOcean, are likely to follow these guidelines for security and performance reasons. Their VMs would run rather slowly if they oversold the RAM. However, for a less popular VM host, I would not know if they followed the guidelines or not. Depending on the workloads of the guests, the host might find it profitable to oversell.

Now remember that the host can definitely store the state of your RAM, and any sort of vulnerability that allows arbitrary code execution on the host can result in the reading of arbitrary VM RAM contents.

The NIST paper also suggests a vulnerability where the images used to create virtual machines are compromised, causing VM users to spin up instances that are already compromised. Amazon and DigitalOcean are likely to follow the NIST recommendations here as well--that VM images must have valid digital signatures and that the images for new instances cannot be modified from a machine running existing instances. Once again, a cloud hypervisor run out of some college dorm room might not have equivalent security on the images.

The whole NIST paper is worth reading--it is not very long--and so are the following Stack Exchange questions:

• Memory Dumping - Cause for concern in Virtualization?
• Does a Virtual Machine stop malware from doing harm?
• Can a virtual machine (VM) “hack” another VM running on the same physical machine?
• How secure are virtual machines really? False sense of security?
• Are there ways to protect the guest kernels at the hypervisor level?

Answer 2

tl:dr; At this time of writing, Docker-style containers are not as secure as hardware-accelerated virtual machine hypervisors.

In response to @grawity on my other answer, this answer covers containers like Docker, which are used to run applications in a way that is similar to virtual machines, but containers are definitely not virtual machines.

What is the difference between a container and a virtual machine? This Smartbear article explains (emphasis mine):

So why should you care about hypervisors vs. containers? Bottomley explains that hypervisors, such as Hyper-V, KVM, and Xen, all have one thing in common: “They’re based on emulating virtual hardware.” That means they’re fat in terms of system requirements.

[...]

Containers, on the other hand, are based on shared operating systems. They are much skinner and more efficient than hypervisors. Instead of virtualizing hardware, containers rest on top of a single Linux instance. This means you can “leave behind the useless 99.9% VM junk, leaving you with a small, neat capsule containing your application,” says Bottomley.

As the NIST quote in my other answer explains, hardware virtualization has security benefits. Container platforms get rid of those benefits in exchange for performance boosts.

Solomon Hykes, the creator of the Docker project, wrote on Hacker News in the summer of 2014 that Docker does not guarantee security against untrusted guests:

Please remember that at this time, we don't claim Docker out-of-the-box is suitable for containing untrusted programs with root privileges. [...] When we feel comfortable saying that Docker out-of-the-box can safely contain untrusted uid0 programs, we will say so clearly.

Various security-boosting settings can be configured, as per the Docker security guidelines, but security is still being enforced by software. As the NIST document in my other answer states, a hardware-accelerated hypervisor uses hardware to prevent VM escape attacks, which dramatically reduces a malicious guest's attack surface. Although Docker may have comprehensive security features, it simply has a larger attack surface.

Gartner's early 2015 analysis of Docker agrees, citing Docker's immaturity compared to traditional virtual machines.

At the time of this writing, I would advise the security-conscious to not share physical hosts with untrusted containers, nor to run untrusted containers on your physical machine. Perhaps this will change in the future.

Answer 3

tl;dr: Cache based attacks to leak RAM are possible. I would be careful of having a VM that spends 100% of its CPU time doing cryptography with secret keys. An attacker can also slow your VM down, which might help a possible RAM-leaking attack. Also, beware of VM snapshots that capture your running RAM (most cloud services don't do that, though).

(I know having three separate answers for the same question is a little ridiculous, but they represent three distinct trains of thought that I am struggling to combine into one answer.)

I just came across Thomas H. Ptacek's review of Cryptography Engineering, and he briefly mentions side channel attacks on virtualized compute clouds like Amazon EC2. Mr. Ptacek did not list any sources or give examples, but here is what I've learned from an (additional) afternoon on Google:

Cache / Memory timing attacks

An April 2014 paper from the China Mobile Research Institute that demonstrates the creation of a covert channel between malware on two VMs. One of the VMs is spun up by the attacker, and the other is yours. This requires your virtual machine to already be compromised, but it enables a very difficult to detect method of data exfiltration. You may be monitoring your VM for malicious outbound network traffic... but you'd be looking in the wrong place as this malware would be using memory bus contention to transmit up to hundreds of kilobytes per second to the attacker's VM.

A 2014 paper from Worcester Polytechnic Institute describes a cross-VM timing attack on AES. The paper authors took great pains to make sure their attack worked in a realistic cloud computing environment, which makes their results rather frightening. The authors note that the vulnerability is made possible by memory deduplication, which is a hypervisor feature that reduces memory usage by detecting duplicate pages. This is great for performance, but deduplication is a clever way to violate the NIST guidelines mentioned in my other answer and oversell your memory.

Matthew Green has an excellent analysis of a 2012 paper describing cross-VM side channel attacks. The attack is more theoretical, more difficult to pull off in practice, and Amazon is likely to have mitigated the vulnerabilities involved since then.

A 2010 paper describes that backing up and restoring virtual machine snapshots can weaken a random number generator and cause data leakage. In this case, it only applies to virtual machine guests with full snapshots, which includes active memory. The paper states that cloud services rely on volume snapshots, which only include disk contents, and do not have weakened random number generators upon a snapshot restore.

This 2009 paper from UCSD and MIT CSAIL is one of the earlier studies of side channel attacks in cloud computing. I would guess (but am not certain) that most of the actual attacks are plugged by now, but the paper is remarkable reading just because it vividly introduces the attacker's mindset.

I/O Slowdown attacks

Regarding your original question, none of these attacks directly leak data. The idea behind these attacks is for the attacker to merely slow the victim's VM down. However, I have a suspicion that an I/O slowdown attack could be used by an advanced adversary as part of a broader, RAM-stealing attack. For that reason, I choose to mention it here. I have a few ideas on how to use an I/O slowdown attack to help, but I don't want to speculate incorrectly.

The two most relevant papers I found were a December 2012 paper that proposes vExplorer, a "distributed VM I/O performance measurement and analysis framework" which can be used to perform I/O based attacks" and a similar paper from George Washington University describing a framework called Swiper.