1

I am involved with configuring a Hadoop cluster for complete auditability and security. I am new to the Hadoop ecosystem, but I have a decent idea of the basics. I have a few concerns for which I hope someone might be able to point me in the right direction.

  1. How do I make sure that an authenticated and authorized user cannot take control of the entire cluster’s resources?
  2. How do I make sure that all the actions taken by an authenticated and authorized user are properly audited so that we can see a log trail of what actions they have taken?

Some people in my organization are very concerned that an authenticated and authorized user might be able to perform “arbitrary execution,” which to them describes a scenario in which someone is able to commandeer the cluster outside of the admin’s controls for their own purposes. Is this something that can actually happen? If so, how can an admin prevent it? I am using Ranger, and have the Yarn and HDFS plugins installed.

STN
  • 111
  • 1
  • it all kind of depends, authenticated and authorized over what, ssh? Do you have sudo protections enabled? If the user is authorized and authenticated it doesn't sound like arbitrary execution at all. You could try limiting access and restricting the resources requested by users using SLURM or another resource manager/scheduler. https://slurm.schedmd.com/slurm_ug_2012/MapRedSLURM.pdf – Matt Nov 15 '17 at 17:14
  • And that would obviously only be if you need exclusively the MapReduce functionality. – Matt Nov 15 '17 at 17:22
  • @Matt my apologies, I meant authenticated and authorized via kerberos. Our cluster will be kerberized and only service accounts will have SSH access to nodes inside the cluster. I agree that if a human user is authorized and authenticated to a service, then it's hard to call his actions via that service "arbitrary execution". I think what we're really worried about is that somehow a user will be able to literally run"./some-script.sh" at an OS-level on the cluster hardware via the available hadoop services even without having shell access to the cluster machines. Do you think this is possible? – STN Nov 15 '17 at 19:13
  • With enough time and resources anything is possible on a computer. Is it likely that a zero-day hadoop exploit exists that has not been reported? probably, so you should weigh the likelihood of the owner of that attack choosing to target your specific organization. If you are really concerned about it you can use a chroot jail, but a sufficiently dedicated attacker could likely get around that as well with enough time and resources. https://unix.stackexchange.com/questions/105/chroot-jail-what-is-it-and-how-do-i-use-it – Matt Nov 15 '17 at 23:01

0 Answers0