3

We have the Windows Sensu client running on several 2008 R2 systems. One of our checks calls a batch file to gather some info and report back. We are sporadically receiving "Unknown: Unexpected error: Access is denied (5)" messages for these checks - no standard frequency, no correlation... the next check runs correctly without issue.

I've traced the call through the stack and the Sensu Ruby code) spawns the child process of 'cmd.exe /c "/path/to/batch.bat parameters"' without issue. I can recreate the issue by removing all NTFS access to the batch file itself. If you run cmd /c batch.bat with these permissions removed, it will return with that same "Access is denied (5)" error.

Ok, great, it's having issues accessing the file. Why? I've run process monitor traces and can find absolutely nothing wrong with the process: ruby loads up, cmd.exe is called, cmd.exe reads the file with no visible errors, then it returns the check as Access is denied. What the...?

I'm out of ideas. Some additional background info in case it helps: the ruby binaries are running as 32-bit; the Sensu client is the latest from the Sensuapp site; the system is running System Center Endpoint Protection. What am I missing??

Dan
  • 1,278
  • 18
  • 27
  • I'm not too familiar with Sensu, but wanted to suggest a few things to at least check just in case: Any AV or firewall software on the Server, ensure CMD.exe is added as an exception perhaps, check for Sensu Server vs. Sensu Client version in case there are bugs with version differences. Check for bugs with Server version you're running for how you have it configured ensuring the security context that launches the `CMD /C` is always the same with the child process and loop logic, verbose logging/stack trace, Windows Event Viewer check for issues, and Server resources perhaps reboot too. @Dan – Pimp Juice IT Nov 17 '15 at 05:08
  • I know this is mostly common sense stuff I mentioned but I'd be suprised if NTFS ACL permission attributes were changing and more likely think the credential or security context is interpreted differently when the issue occurs such as some kerberos, NTLM, etc. type issue with the process when executed. Check the security of the CMD.exe file on the server too and I'm not sure if that loop has some limit and if that limit is met then it kicks off the CMD process differently. Not sure if you could execute `CMD` without `/C` switch and just add `EXIT /B` to the end of the batch too just to test. – Pimp Juice IT Nov 17 '15 at 05:13
  • Have you tried enabling advanced logging policies and reviewing the event viewer? I'd try the `Account Login, Detailed Tracking, Object Access` and `Privilege Use` fields https://technet.microsoft.com/en-us/library/dn319056.aspx – Mass Nerder Nov 18 '15 at 18:21
  • @PJMahoney, I had AV in the back of my mind (hence my mentioning SCEP running) but was hoping there was something else obvious I was missing before I went mucking with those policies. I modified some of the checks to use Ruby only and no cmd.exe at all - they're still getting Access is Denied, so I guess that rules out cmd.exe as the culprit. Time to make some SCEP exclusions... :-) – Dan Nov 19 '15 at 19:14
  • @MassNerder, I'm going to try the AV exclusions first and then give the extra logging a shot if nothing comes from that. – Dan Nov 19 '15 at 19:14

1 Answers1

-1

This turned out to be a bug within the ChildProcess Ruby library and how it handled assigning the Windows Process to a Job object (race condition). I'll be sharing the fix with the author once I clean up the code.

Dan
  • 1,278
  • 18
  • 27
  • Nearly three years later and I am troubleshooting the same message but your answer has no details about what you did and how you came to the conclusion that you came to... – Robert Kaucher Nov 28 '18 at 18:46