I am involved with configuring a Hadoop cluster for complete auditability and security. I am new to the Hadoop ecosystem, but I have a decent idea of the basics. I have a few concerns for which I hope someone might be able to point me in the right direction.
- How do I make sure that an authenticated and authorized user cannot take control of the entire cluster’s resources?
- How do I make sure that all the actions taken by an authenticated and authorized user are properly audited so that we can see a log trail of what actions they have taken?
Some people in my organization are very concerned that an authenticated and authorized user might be able to perform “arbitrary execution,” which to them describes a scenario in which someone is able to commandeer the cluster outside of the admin’s controls for their own purposes. Is this something that can actually happen? If so, how can an admin prevent it? I am using Ranger, and have the Yarn and HDFS plugins installed.