Is it safe to store critical passwords in server environment variables?

23

7

I have a cluster of servers, each with configuration files that currently contain plain-text passwords for sensitive, mission-critical systems (message queues, data stores, and other services).

Some people move critical passwords out of configuration files into an environment variable of the user accounts under which the server processes run. In this way, configuration files can be committed to version control, and the system administrator only needs to create an appropriate environment variable when the server system is set up. Naturally, access to the accounts that run these services is very restricted.

Is this really the best way to avoid passwords in plain-text configuration files, or is there a better way?

Steve HHH

Posted 2014-01-28T19:40:03.790

Reputation: 5 590

3well, keep in mind, if you do store it as a variable, it is in plaintext both at rest and when a user queries it. that means that if you lose a little control of the server tier, you;ve handed the attacker a password. I'd probably use crypto, and decrypt the password info as needed. – Frank Thomas – 2014-01-28T19:45:55.433

Nice thoughts, Frank. If you were to use crypto, what sort of system would you use? Something based upon an RSA/SSH key, a keychain tool, or something else? We currently use Linux >2.6 systems like CentOS and Amazon. – Steve HHH – 2014-01-28T20:36:15.310

Answers

11

If you're on a Linux system, look at /proc/*/environ and decide if environment variables are a good place to store sensitive information or not. /proc/self is the current process:

$ tr '\0' '\n' < /proc/self/environ
USER=me
LOGNAME=me
HOME=/home/me
PATH=/usr/bin:/bin:/usr/sbin:/sbin
MAIL=/var/mail/me
SHELL=/usr/bin/sh
SSH_CLIENT=1.2.3.4 58195 22
SSH_CONNECTION=1.2.3.4 58195 7.8.9.0 22
SSH_TTY=/dev/pts/1
TERM=xterm

Never mind that the thing setting the environment variable is probably reading a file somewhere.

The thing to remember is that using a password means the password is available to the program. If this password is not provided by a user typing it in every time a program needs it, that password must be accessible based on only the program's access. You can encrypt the password locally and have the program decrypt using a key, but all that does is obscure the password against accidental disclosure; someone who has the same access as the program can do the same things the program can do, which includes reading the encryption key.

The right way to do this is to have the application run as a restricted account, and store the password in a file protected with filesystem-level permissions. Hopefully you can "include" a file or similar in order to keep the password out of a version control system (assuming the VCS has no security controls). To protect against inadvertent disclosure, obscure the password however you want - base64 encode it, use pgp to encrypt, whatever makes sense in your server program's set of options. If you're writing a program to do this, about the best you can do is to prompt a user for the password only when needed, and then purge that password from memory as soon as it's used.

dannysauer

Posted 2014-01-28T19:40:03.790

Reputation: 837

3Yes, a file with mode 0600 owned by the user running the program is accessible by the same people who can access the program's environment. But, as I mentioned, the environment is probably configured by reading a file, so copying data into the environment is just increasing the number of places the data is available. And the environment is, by default, duplicated to child processes. And a number of programs have external means to query environment variables, due to bugs or intentional design decisions (think phpinfo()). If a file will be involved either way, why increase the attack surface? – dannysauer – 2015-01-30T16:56:05.107

1The tr command you gave didn't work for me - this did: cat /proc/self/environ | tr '\0' '\n' – robocat – 2016-11-04T00:57:53.150

I'm in my phone so it's hard to tell... That should be a Zero; did you type the letter "o"? – dannysauer – 2016-11-04T01:00:32.413

8

Ultimately, if you have any data that needs to be read as well as written, you're going to end up protecting something with a password (or if you're really paranoid, with a physical hardware smart card and a PIN), no matter how many layers of encryption you have.

This boils down to the basic question of system security vs. convenience. You can add "defense in depth" by having lots and lots of layers of security controls that a malicious actor would have to breach in order to get to the "goods", but then when a legitimate actor wants to read or change some data, they have to go through a bunch of hoops. The alternative is plaintext passwords in text files.

What I'd do if I really wanted to protect some information in a mission-critical system:

  1. Use Full Disk Encryption, so that the contents of the entire persistent storage is encrypted.

  2. Restrict physical access to the machines. Lock the machine's chassis with a secure locking mechanism and control physical access to the keys. Hire muscle (armed guards) to be gatekeepers for access.

  3. Enforce fine-grained Mandatory Access Control (MAC) in the operating system of the device. You can start with something like SELinux on GNU/Linux and set it to Enforcing, and then tailor the policy to the exact needs of the production software, allowing those accounts exactly (and only) the permissions they need to the files they need.

  4. If you're going to have system-specific passwords, and version control for configuration files, you really want to avoid the possible mistake of having a plaintext password mistakenly committed to version control, since it can be hard to dislodge a leaked password from a VCS's cache. Environment variables are one of several viable options for that. The other is a password prompt when the program starts up, but then rebooting the machine and restoring operational status is a manual effort and can't be done autonomously, so there's that convenience vs. security again.

  5. Make sure you have networking specialists on hand to take care of firewall permissions, to minimize your exposure to an attack over the network. Audit (penetration test as well as whitebox test the code) any software that interfaces with external systems, especially the public Internet. "Interfaces" includes not only direct network connections, but also reading or writing "untrusted" data (data whose bytes originated from outside the RAM/disk/CPU of the secure server).

This isn't a complete list, but especially point 4 is probably relevant to you, although if you don't perform at least steps 1 through 3, consideration of point 4 and point 5 is not going to help you very much, because your system is not secure at a fairly fundamental level.

allquixotic

Posted 2014-01-28T19:40:03.790

Reputation: 32 256

I'd skip #1. If the system is running, the filesystem is mounted and unencrypted. If it's not running, your physical access control should be adequate. If it needs rebooted, you either need someone to provide the decryption key on every boot (which is annoying), or you need to have the machine provide that - in which case the credentials are usually also available to anyone with access to the hard drive. Most "server" machines typically don't use disk encryption because of that trade-off cost. :) – dannysauer – 2014-01-28T21:48:39.467

"This boils down to the basic question of system security vs. convenience" -- quoted from my answer -- which applies equally to your comment. You can't maximize security and convenience at the same time. – allquixotic – 2014-01-28T21:57:40.107

1#1 is important in the case that some chunk of the ram hits the disk (swapping) or if it hits a ssd sector which becomes "bad" later on and is moved away from the OS but still holds that piece of ram. even when the disk is currently unlocked that chunk of data ends up scrambled on the platters. – akira – 2014-01-28T22:59:00.240

3

Passing a password in an environment variable is as safe as having the program read it from a file. Only processes running as the same user may read a process's environment, and these processes are allowed to read the same files anyway.

Note that this is different from passing a password on the command line. Command line arguments are readable by all processes running on the same machine (barring hardening measures), not just processes running as the same user.

If you pass a variable through the environment, beware if the program launches other programs. Those other programs will inherit their parent's environment. So don't do this if you fear that the other programs might accidentally leak the contents of their environment.

The flaw in your scenario is “create an appropriate environment variable when the server system is set up”. An environment variable is a dynamic property of a process. You can't create it when setting up a system, not if by setting up you mean something that survives a reboot. What you mean is presumably that the administrator arranged for this variable to be present in the environment when a certain user logs in. This is done through a configuration file (typically ~/.pam_environment or ~/.profile or a file read from ~/.profile). So this solution does not, in fact, move the password out of configuration files.

Setting things up so that passwords are in a user's login-time environment is not a good idea. It means that every process running as that user will have the secret, so it's vulnerable to a leak anywhere.

A password should be put in a file that's aside from the configuration files that are under version control and from the normal deployment mechanisms. It's ok to put the password in the environment at some point if it's convenient, but it should be done for as small a set of programs as possible.

Gilles 'SO- stop being evil'

Posted 2014-01-28T19:40:03.790

Reputation: 58 319