Intel SGX and MSRs - what do developers need to know?

Question

Intel SGX is an intriguing new technology that will ship as part of upcoming Intel processors. It is designed to enable running software in a secure enclave. Code running in the enclave will be isolated by the hardware from untrusted code running on the same processor, so you can execute security-critical code in the enclave and protect it from attack. While SGX is not supported in currently available processors, Intel suggests that it will be available soon. The hope is that this might provide a strong foundation for certain kinds of isolation: developers will be able to run certain code (e.g., crypto key management code) in a secure enclave, with guarantees that other code won't be able to tamper with it.

A model-specific register (MSR) is a special configuration register that controls the operation of the processor. For instance, MSRs can be used to enable branch tracing, performance counters, hardware watchpoints, and other useful special features. Normally, only privileged code can read and write most MSRs.

My question: How does SGX mode interact with MSRs? What do developers of code that runs in SGX mode need to know about MSRs?

The SGX specification doesn't say how SGX mode interacts with MSRs. However, you could imagine that MSRs could potentially pose a risk to code running in an enclave, depending upon how the two interact. For instance, if untrusted code could enable branch tracing (via a MSR) and then trigger invocation of a secure enclave, and if this setting was retained across the mode switch causing all branches taken by the enclave code to be recorded into memory accessible by untrusted code, then this could allow untrusted code to mount powerful side-channel attacks against enclave code.

So, how do the values of the MSR affect execution of code running in an enclave? Are the MSRs ignored when running in enclave mode? Does code designed to run in an enclave need to clear/reset all the MSRs before doing anything sensitive? What do developers of code that'll run in an enclave need to know about how MSRs work in SGX mode, to write secure code and protect their code from attack?

Answer 1

This is a great question! Never thought about this until you asked. Firstly, code running inside a secure enclave runs in ring 3. So all restrictions that apply to untrusted non enclave ring 3 code apply. So an enclave cannot write to MSR's.

Next, the specs don't explicitly mention anything about MSR's but it does tell you about interaction with IA32 features. For the branch tracing example you gave, look at the section on LBR(last branch record). This tells you exactly how it interacts. From my understanding, EENTER/EXIT instructions are seen as one long branch each to the entry point inside the enclave. Anything that happens after you jump to code inside the enclave is not recorded or visible to untrusted code no matter the privilege because the branch records are popped out of the LBR stack on enclave exit. It is only visible on a debug opt in entry that can be set by the developer of the enclave and as the name clearly states, the enclave is in debug mode.(See chapter 7,section 5 on branch tracing.) So the side channel you are talking about should not be possible.

Branch tracing side channel attacks might not be an issue, but other kinds of performance monitoring could be an issue. Page faults, memory accesses, cache behavior, etc. can still be monitored. Cache-based side channel attacks are still possible as well. So yes side-channel attacks are possible against enclave code. The Haven paper from Microsoft explains the kind of limitations that SGX still has in terms of side-channel attacks.

And to answer your question, what an enclave developer needs to know about MSR's depends on what kind of attacks the developer expects in his code. I assume there will be some sort of threat modeling while designing and developing the enclave and based on what threats you expect, you need to figure out how SGX treats these threats. The branch side channel attach is an example of figuring out how SGX treats it.

For the most part, I expect any MSR/register/processor state that could leak information about the enclave and its execution are hidden/protected from untrusted code, privileged or unprivileged and an enclave writer need not worry about it.