5

The grsecurity folks just released a test patch for the 4.5 Linux kernel which includes Return Address Reuse Attack Protection or RAP, a protection technique against return-oriented programming (ROP).

Their slides are beyond comprehension for me at this point. Can somebody please explain, in layman's terms, how RAP works and how much protection it provides (is it probabilistic)?

Edit: While the slides led me to believe that RAP stands for "Return Address Protection", Brad Spengler from from the grsecurity team confirmed that it actually for "Reuse Attack Protection" (as also confirmed by Polynomial's answer).

Community
  • 1
jotik
  • 221
  • 1
  • 7
  • I'm personally quite curious to know how the public demo and commercial version differ, in terms of prevented attacks (i.e. other than the basics that are described on the grsecurity website). I get that the XOR canary checks are left out, but it'd be neat to know exactly what they protect from and how they work. – forest Apr 30 '16 at 00:52
  • 4
    I've updated our announcement at https://grsecurity.net/rap_announce.php to include a detailed FAQ that answers all your questions and some more that weren't asked. -Brad –  Apr 30 '16 at 16:23

1 Answers1

5

RAP is very similar to a stack canary, except designed to prevent against ROP.

EDIT: After speaking to spender from grsecurity, it turns out that what I'm describing here is a subset of the RAP functionality, specifically the return pointer protection. RAP is actually callled "Reuse Attack Protector", which is designed to protect against all forms of indirect calls. The paper you linked describes some of the functionality within RAP, of which both the return pointer protection and ICFG features are a part of.

We can see the following assembly from the paper:

push %rbx
mov 8(%rsp),%rbx
xor %r12,%rbx
...
xor %r12,%rbx
cmp %rbx,8(%rsp)
jnz .error
pop %rbx
retn
.error:
ud2

The first block there sets up the RAP cookie:

; store the old value of rbx
push %rbx
; read the top value on the stack (the return pointer) and put it in rbx
mov 8(%rsp),%rbx
; xor rbx with a random value stored in r12
; rbx now contains the RAP canary
xor %r12,%rbx

Then, at the end of the function, before the ret:

; xor the RAP canary in rbx with the random value in r12
; this "decrypts" the stack pointer from the RAP canary
xor %r12,%rbx
; compare the top value on the stack (the return pointer)
; with the value decrypted from the RAP canary
cmp %rbx,8(%rsp)
; if the comparison fails, jump to error
jnz .error
; restore the original value of rbx
pop %rbx
; return
retn
.error:
; an error occured, quit the process
ud2

The idea behind ROP is that you would find instructions you want (gadgets) followed by a return instruction. This protection feature re-writes the functions so that the final instruction before each return is always a pop %rbx, and previous instructions are used to validate the RAP canary. This makes it really difficult to find ROP gadgets outside of non-aligned instructions (i.e. gadgets that just happen to exist due to the binary interpretation of instructions outside normal alignment).

The protection can be applied to both usermode and kernel. The cookie value (the value in %r12) is regenerated per task, per syscall, and per iteration in some infinite loops (e.g. event handlers and dispatch loops). The reason for the last one is that infinite loops often use a ret to break out of the loop. (this isn't quite correct).

EDIT: After discussing this more with spender, I'm going to suggest waiting for the explanation to come out. There's a lot of subtlety to how the feature works and I'm not sure I can accurately cover it from the paper and my brief discusson on IRC. Once it's out I'll re-update this answer to give you a better idea of what it does.

Polynomial
  • 132,208
  • 43
  • 298
  • 379