There are a lot of complex issues, mainly related to effort and trust dynamics, that undermine security policies in organisations. Whilst individual factors have been uncovered by researchers, there isn't as of yet a single unified theory of what security management styles are preferable or of what policies to implement in every single organisation.
The direct cost of security compliance
First things first, all decent security engineers know that some risks are better ignored than dealt with because the cost of protection would be higher than the loss caused by a breach. Likewise, some policies or mechanisms cost much more to end users / employees than they provide value to organisations, but many organisations have not yet realised this. You can find famous, generic examples of phishing protection or SSL certificates in So long and no thanks for the externalities from Herley.
In organisational security, Beautement et al. explain in The Compliance Budget used qualitative interviews with employees to determine what causes them to comply. Four factors come into play: the costs and benefits of compliance for the employee, and the costs and benefits for the organisation. Essentially, employees make a cost/benefit estimation when they need to decide whether to comply with a security measure or not.
The issue of perception
Evidently, people who have not received training in computer security have lacking mental models of how security works and misestimate the risks they're taking. So rather than actual costs and benefits, employees reason based on their perceptions of costs and benefits, both for themselves and organisations. This means that balancing perceptions is one possible course of action for organisations that are already cost-conscious but meet compliance issues.
Additive costs and the compliance budget
The main idea behind the Compliance Budget model is that employees will tolerate a limited amount of daily effort/cost to themselves in order to fulfill the organisation's benefits. The sources of cost (physical and mental load, embarassment, missed opportunities, and the fact that security competes with more motivating and important tasks) to employees are more numerous and tangible than the individual benefits (avoiding the consequences of a security breach, and the sanctions of being caught bypassing, owing to Beautement et al). If that model is correct, then organisations can act in a number of ways to boost the compliance threshold of employees:
- increase the perceived benefit to the organisation (e.g. by improving employees' mental models of the consequences of breaches)
- increase the perceived benefit of compliance to users (e.g. by punishing employees harsher...)
- decrease the perceived, and actual, costs of compliance to users (which is in itself a very complex topic)
And the hidden costs
Now that the general principles are laid down, there is additional research that explains typical compliance-draining factors, mainly from Bartsch and Sasse in How Users Bypass Access Control and Why. Below is a breakdown of some relevant observations made in the paper.
The cost of correcting and updating policies
CISOs, as all human beings, make mistakes and approximations and can occasionally deploy policies that conflict with the needs of individual workers and prevent work getting done. For instance, a data encryption policy might prevent sales staff from using convenient storage media to go and present their products to remote clients. Furthermore, employee needs change over time as work practices evolve or new projects emerge.
Employees can often need to wait for a long time to get policy changes implemented, which severely undermine their productivity. For instance, teams that welcome interns can end up not being able to provide them with work, unless they bypass access control by sharing credentials or deploying unprotected shared storage medium. The fact that access control is often managed top-down rather than in more decentralised ways can contribute to the existence of delays and productivity deadlocks.
Inter-employee trust vs emotional blackmailing
Ironically, employees who get to manage access control directly for their team (sometimes out of sight of their organisations) can feel an emotional pressure to grant access to their colleagues and subordinates, as they want to avoid grudges in their team. Kirlappos et al extend greatly in the role of employee inter-trust in security decision making in Learning from Shadow Security.
In my opinion, both papers lean towards the idea that security could be collaboratively managed by organisations and employees in positions of making security decisions. In Bartsch and Sasse, the need for organisations to provide high-level security policies that can be implemented in local file-sharing systems has been pointed out, as this can address deficient security mental models and the issue of emotional blackmailing for access.
There are a few suggestions that in-situ, local decision making is more fit in personal security too (Reactive access control, Laissez-faire file sharing).
The above should suffice to explain why rigid security policies are bypassed, and provide options to improve compliance and policy fitness. Albeit there is still a lot of research to be done, there are't any inconsistencies on those topics in the academic literature that would lead me to believe any of these hypotheses, models and results to be incorrect. They're probably the best informed strategy to apply right now.