How can I cache a passphrase for use in a script that uses gpg2 --symmetric with STDIN?

Question

I am working on a computer program that stores and retrieves secrets and needs to run without user interaction on the server.

I have successfully set everything up in my system to use gpg-agent to retrieve the cached passphrase without user interaction to encrypt and decrypt data with gpg2 --symmetric when I specify a file name.

However, this requires me to temporarily store my data on the disk without encryption. I do not want to put unencrypted data on my disk. So now I have changed things to work with STDIN and STDOUT rather than disk files.

The command I am testing with is this: cat test.txt | gpg2 --symmetric -o test.gpg.

The problem is that now it does not seem possible to get the passphrase cached and so I must enter it interactively each time. This will not work for my server program.

I have tried using gpg-preset-passphrase for this. I do not know if it is possible though because it wants a "KEYGRIP". One rough idea I have is this: maybe there is some default keygrip that gpg2 uses with --symmetric and STDIN. But I don't know if that idea will pan out.

I am not planning to specify a passphrase in my code with --passphrase because that means I need to store a secret in the code, which I am not going to do.. and it would allow anyone who could access the code to decrypt the files. It would also expose the passphrase on the command line.

I hope that it is clear enough what I am trying to achieve. Any ideas are appreciated.

Answer 1

I recently struggled with this and (after endless stumbling around gpg docs and qa sites) came across this 2015 thread that mentions the answer:

Maybe what i'm missing is how the "cache_id" is selected for the symmetric passphrase, both at creation time and at re-use time. can you summarize that? I dug around in the code a bit but didn't sort out how it's being done.

If I remember correctly (and the logic hasn't changed), it is a randomly chosen 8-octet salt value: http://tools.ietf.org/html/rfc4880#section-3.7.1.2

...

The S2K salt is used as cache id.

You can get the salt by listing packets for your encrypted file:

$ echo | gpg --list-packets ~/myencryptedfile 2>&1 | grep -o '[0-9A-F]\{16\}'

0123456789ABCDEF

• leading echo forces Inappropriate ioctl for device error, bypassing pinentry prompt
• 2>&1 hide stderr
• grep -o '[0-9A-F]\{16\}' extract salt from command output

Now we have the 8-octet hex-encoded salt value.

Unfortunately that didn't work directly for me. So after more stumbling around the gpg codebase, I came across this snippet:

  memset (s2k_cacheidbuf, 0, sizeof s2k_cacheidbuf);
  *s2k_cacheidbuf = 'S';
  bin2hex (s2k->salt, 8, s2k_cacheidbuf + 1);
  s2k_cacheid = s2k_cacheidbuf;

The crux is the "S" prepended on the second line. I couldn't find this mentioned anywhere else.

So all together now. Encrypt the file:

$ gpg --symmetric --passphrase-fd 3 --batch --output ~/myencryptedfile <<EOF 3<<EOF3
> myencryptedcontent
> EOF
> mypassphrase
> EOF3

• --passphrase-fd 3: supply passphrase over file descriptor 3
• --batch: required with --passphrase-fd
• I prefer to supply passphrase via --passphrase-fd 3 and data to be encrypted over stdin (<<EOF and 3<<EOF3) rather than write it to a file initially for cat, or echoing it into stdin (which makes me fearful of process list snooping). Docs: here-documents (EOF syntax).
  If you don't care about portability outside of bash here-strings are much more palatable:
  gpg --symmetric ... ~/myencryptedfile <<<myencryptedcontent 3<<<mypassphrase

Get the 8-octet hex salt value:

$ echo | gpg --list-packets ~/myencryptedfile 2>&1 | grep -o '[0-9A-F]\{16\}'

0123456789ABCDEF

Make sure passphrase reset is allowed by gpg-agent:

$ cat ~/.gnupg/gpg-agent.conf
allow-preset-passphrase

$ gpg-connect-agent reloadagent /bye  # restart the agent after changing config

Preset the passphrase:

$ /usr/local/Cellar/gnupg/2.2.17/libexec/gpg-preset-passphrase --preset S0123456789ABCDEF <<EOF
> mypassphrase
> EOF

• "S" prefixed onto the salt, so the resulting gpg-agent cacheid is
  --preset S0123456789ABCDEF
• passphrase over stdin via heredoc (EOF) to avoid it showing up in the process list
• If you get ERR 67108924 Not supported <GPG Agent> - no --allow-preset-passphrase, you need to add allow-preset-passphrase to ~/.gnupg/gpg-agent.conf and restart the agent (see above)
• The location of gpg-preset-passphrase may be different on your system.

And finally, decrypt the file!

$ gpg --decrypt --pinentry-mode loopback ./myencryptedfile
gpg: AES encrypted data
gpg: encrypted with 1 passphrase
myencryptedcontent

Notes:

• I've found locating gpg-preset-passphrase in my scripts annoying since it isn't installed anywhere on the system path (at least not in my installation). However, gpg-connect-agent is, and can be used for the same purpose with an additional step. You just need to hex-encode the passphrase before using it (tr -d '\n' because EOF adds a newline):

$ (tr -d '\n' | hexdump -v -e '/1 "%02X"' && echo) <<EOF
> mypassphrase
> EOF
6D7970617373706872617365

$ gpg-connect-agent <<EOF
> preset_passphrase S0123456789ABCDEF -1 6D7970617373706872617365
> EOF
OK

# now decryption should work as above

• I'm running gnupg version 2.2.17 on OSX, installed via homebrew

$ gpg --version
gpg (GnuPG) 2.2.17
libgcrypt 1.8.5