Is it acceptable for security engineers to have admin access to production servers?

Question

In my organization, our server infrastructure team has historically held all admin access to production servers. We're recently trying to strengthen our security posture, so we have hired several new security engineers. We are investing significant funds in new toolsets for them, and we are also trying to identify where we need to change our policies and security models to enable them to do their jobs effectively.

Some of the security engineers believe that they should have admin access to the production servers, in order to investigate suspicious activity, as reported by their new tools.

It has been proposed by the server infrastructure team that they could grant access to individual servers on an as-needed basis for specific incidents. The security engineers argue that such a high-friction process would potentially slow down investigations, which by their nature often take place in situations where minutes count, and where they don't really know which servers they will need access to until they're elbow-deep in the investigation, and pausing every few minutes to request access to a new server would be unnecessarily cumbersome. The server infrastructure team argues that the security engineers' tools should provide visibility into the broad majority of the data the engineers need, and they are reluctant to grant blanket admin access to the engineers.

Are there recommended best-practices for such situations? I understand the server infrastructure team's desire to maintain principle-of-least-privilege for security's sake. But I also understand that security engineers are somewhat implicitly trusted by the nature of their role. In large organizations, how is security engineers' access to servers typically managed?

Answer 1

Good question. This is a difficult topic. The easy answer is, "it depends" and there is no perfect answer. Also, it's not possible to give a prescriptive answer whether they should have this access without considering detailed scope of the teams. But I'll attempt to provide better context.

First, work must get done and the Security Engineers have a good point if they can't accomplish their work. Make sure that the Security Engineers need access. We should break down the problem to identify whether Security Engineers need access or not. These initial steps might be obvious, but I'll list them anyhow:

1. What tasks do Security Engineers need to accomplish? Be specific. The answer to the question heavily depends on the scope of the Security Engineers' work!
2. Of these tasks, what gaps exist in their current access?
3. Is there any means of giving them access to the needed information / controls without providing them direct access?
4. Is it worth doing so? Often it may be too much work or too expensive (i.e. not worth it) to try and implement a solution for them to have the information and controls needed. And in this case direct access is the best/cheapest solution.

If the Security Engineers require access (most of the time they do), then you need to employ defense in depth to try and be as safe as possible. In other words, the company needs to attempt to take steps to prevent an incident and have a proper response capability. By doing so the risk can be reduced to a manageable level.

Some considerations:

• Incident Prevention
  • Least Privilege: Give the fewest number of Security Engineers access as possible. More accounts with access increase the attack surface. For each account you need to worry about:
    1. the account being compromised, and
    2. the employee causing either intentional or unintentional damage.
  • Strong Authentication (multifactor): Enforce strong authentication to prevent the account being compromised and help enforce non-repudiation.
  • Limit accessibility: If possible, reduce the ability to reach the system of the network by requiring access to the intranet, VPN, or from a jump box.
• Incident Response - if an incident occurs these will help you deal with the situation
  • Non-repudiation - Administrator access to systems must be well documented and audited in a means that prevents, as much as possible, an Administrator from denying he/she executed the actions.
  • Separation of duties - For example, Security Engineers shouldn't have the ability to modify logs. This helps with non-repudiation. Also, make sure a Security Engineer can't do irreparable damage by not giving them access to backups of configurations/software/data.

There are many more, but personally I feel as that if the above are well established then the Infrastructure Team would feel more comfortable with giving the Security Engineers access.

In summary: Give the teams the access they need to do their job. Don't limit their access to necessary resources. Yes, this means you're increasing your attack surface, but you can use defense in depth to minimize risk.

Answer 2

I agree that Security Engineers are specially trained to look for certain anomalies and have a different thought process than a SysAdmin. When investigating, they need the freedom to be able to investigate. If they have to continually explain why they are requesting access, it sometimes just comes down to a gut feeling that needs to be eliminated (Threat Hunting). That reason may not be seen as enough by the SysAdmin who doesn't understand and Security doesn't have time to train them on why. I've worked in many Fortune and smaller companies. In the end, my Team always ended up with Admin rights because of the fact it ended up being more work to delegate specific rights than to prescribe a built-in group. Another advantage to not getting to creative with delegated rights is that the overall permissions get lost which causes more security holes that are hard to monitor.

Separation of Duties is misused all the time. The point of Separation of duties is to prevent the ability to hide what is happening and collusion. When any one group has total control over something that is the total opposite of separation of duties. It is a conflict of interest for