Retrieving an RSA key from a running instance of Apache?

Question

I created an RSA keypair for an SSL certificate and stored the private key in /etc/ssl/private/server.key. Unfortunately this was the only copy of the private key that I had.

Then I accidentally overwrote the file on disk (yes, I know).

Apache is still running and still serving SSL requests, leading me to believe that there may be hope in recovering the private key. (Perhaps there is a symbolic link somewhere in /proc or something?)

This server is running Ubuntu 12.04 LTS.

score 39 · Accepted Answer · edited Oct 29 '13 at 11:35

SUCCESS!

I was able to retrieve the private key. But it wasn't easy. Here's what you need to do:

Make sure you do not restart the server or Apache. The game is over at that point. That also means making sure that no monitoring services restart Apache.
Grab this file - source code for a tool named passe-partout.
Extract the source code and adjust line 9 of Makefile.main to read:
```
$(CC) $(CFLAGS) -o $@ $(OBJS) $(LDFLAGS)
```
(Notice that the $(OBJS) and $(LDFLAGS) are reversed in order.)
Run ./build.sh.
Grab the PID of Apache using:
```
service apache2 status
```
Run the passe-partout command as root:
```
sudo passe-partout [PID]
```
...where [PID] is the value you retrieved in step #5.
If the program succeeds, your current directory will have a bunch of extra keys:
```
you@server:~# ls
id_rsa-0.key  id_rsa-1.key  id_rsa-2.key
```

If all went well (and hopefully it did), one of those keys is the one you need. However, if you had more than one certificate/keyfile in use, then you need to figure out which one it is. Here's how you do that:

First grab a copy of the certificate that matches the signed key. Assuming the file is named server.crt, run the following command:

openssl x509 -noout -modulus -in server.crt | openssl md5

This will output a value that you will need to match against each of the keys. For each key, run the following command:

openssl rsa -noout -modulus -in id_rsa-0.key | openssl md5

If one of them matches, you've found the key.

_{Credit: this article pointed me to passe-partout.}

+1 for the writeup (and a virtual +1 for the article author: being ashamed and still writing it down helps others in that situation). — Joachim Sauer, Oct 29 '13 at 10:06

score 8 · Answer 2 · answered Oct 29 '13 at 01:36

8

Most likely it is storing the key in memory, which it does because it needs to keep a copy after it drops privileges and/or decrypts the key using a supplied passphrase.

In theory, you could get it out of the process image if you attached a debugger, though if they are following best practices it will be encrypted against something in memory.

That said, if it happens that it still has it open, /proc/${PID}/fd/${SOMETHING} may be it. If you overwrote it, your key won't be there because the overwriting data will be. If you copied something else into its place (or deleted or unlinked it, or recursively deleted its parent directory), it will be there.

answered Oct 29 '13 at 01:36

Falcon Momot

24,975
13
61
92

I used `cp` to copy the new key in place of the old one. – Nathan Osman Oct 29 '13 at 01:39
I looked through the open file descriptors in `/proc`... nothing. – Nathan Osman Oct 29 '13 at 01:51
It likely didn't keep a copy open then, which is the usual case for reading private keys; your only options are to figure out how and where it is stored from the source, pull it out using a debugger (assuming you built with symbols), and decrypt it... or replace the certificate. – Falcon Momot Oct 29 '13 at 03:14
I got it! You were right - Apache does keep the keys in memory and I was able to retrieve them. Please see my answer. – Nathan Osman Oct 29 '13 at 04:27

Retrieving an RSA key from a running instance of Apache?

2 Answers2

SUCCESS!

Linked