Email notification when automatic service is not started (Windows Server 2012 R2)

Question

We have a few servers (Windows Server 2012 R2) in our environment that install updates automatically and restart after that.

After the restart we would like to check the status of all the (Windows) services which are set to automatic start. If a service did not start, the script would try to start the service 3 times and if all attempts fail, it should send an email notification so that we can check what's wrong with this service on this server.

I was wondering if there already existed an easy solution, maybe a PowerShell script?

score 1 · Answer 1 · answered Oct 14 '15 at 12:42

1

I would advise you to look into getting a NMS. We use PRTG and it works well. In it you can create sensors that monitors windows services using WMI. You can configure the sensor to send mail/start scripts if the service go down.

PRTG is licensed by sensors, but the free version gives you 100, which should be enough for a couple of servers (You need one sensor per service to monitor)

answered Oct 14 '15 at 12:42

Jack Pettersson

236
3
11

Funny you mention PRTG, we actually have that as well, paid version. But we have so many servers and sensors in PRTG we don't want a mail message for every small incident. We want a different approach for this case. Note that we don't monitor every automatic service on every server, but a script could easily get a list of all automatic services and their status. The script is only meant to be run once after a restart. If an important service later stops, we would get a notification on our PRTG TV screen. – Peter Oct 14 '15 at 13:04
@Peter PRTG should be able to do this if you tweak the notifications to how you want. If you want them off at first, set it that way, and for the important ones, set them to notify you after x number of minutes. In any case, any services that you have set to monitor should have some sort of notification which it sounds like you already have. You wouldn't want to monitor unneeded services. – Nixphoe Oct 14 '15 at 13:27
Thanks for your suggestion, but as I just added in my main question, we want as little effort for this as possible. We have over 30 servers (automatic updates) with up to 50 services, some of them are very important and monitored by PRTG, some services are not monitored but we would still like to see if they didn't start after a restart. – Peter Oct 14 '15 at 14:07

score 1 · Answer 2 · answered Oct 15 '15 at 03:58

Powershell would be a great way to monitor these services. Oddly enough, I was just reading a blog about this today. I will try to link the script if I can find it, but the general gist of it would be:

1.) Find services that are set to start automatically.

2.) Check the state of the service.

The trick is that the

get-service

cmdlet does not output any sort of 'StartupType', so you will have to use the Win32_Service WMI object instead.

Foreach($Server in $ServerList){
    Get-WmiObject -ComputerName $Server Win32_Service | 
    Where-Object {$_.StartMode -eq 'Auto' -AND $_.State -eq 'Stopped'}
}

This should get you a list of services fitting your desired parameters.

NOTES There are a couple of notes to this:

1.) I would highly recommend you have this script start a couple of minutes after the servers start up as the $_.StartMode -eq 'Auto' will encompass services set to start immediately after booting AND ones set to start automatically after a delay.

2.) There will be services that will be returned through this one-liner that you probably do not care about monitoring. (I.E. on my laptop, a service called "TrustedInstaller" fit this criteria and was returned as a stopped service :/ ) So you would most likely need to filter out these as well.

Thank you, I found a script with similar commands. However, I need help customizing it to fit my needs. I would appreciate your help: http://serverfault.com/questions/731217/powershell-script-to-restart-automatic-windows-services-and-notify-if-failed — Peter, Oct 23 '15 at 14:57

score 1 · Answer 3 · answered Oct 16 '15 at 17:00

Service > Properties > Recovery

Can select what to do on first 3 failures, including restarting service or running a command.

This is where you would also use Run program to set your script that would restart service, clear a file, send you a notification, etc. I am all for NMS (Zabbix) or Powershell script, but most of this can be solved natively and GPO (if applicable to your environment) can have this set across all necessary services/machines.

Peter · Accepted Answer · 2015-11-11T11:01:33.123

I found a Powershell script that is perfect for me, I just need to customize it a bit. Here is the link http://doitcloudy.blogspot.de/2014/07/scriptgesteuerter-neustart-von-windows.html (German, but the script on the bottom of the page is all English). However, I need some help with this part (lines 130 and following):

foreach ($item in $report){
Write-Host "Starting Service " $item.Name " on server: " $item.Server -Foregroundcolor yellow
$start = Get-Date
$startSVCblock = {param($item) Start-Service -InputObject (Get-Service -ComputerName $item.Server -Name $item.Name)}
$j = Start-Job -ScriptBlock $startSVCblock -Arg $item
do {
    if ($j.State -ne 'Running') { break} 
    $j | Receive-Job
} while (((Get-Date) - $start) -le $timeout)

}

I see that this is the part where the script starts a service, but I'm not used to jobs in Powershell. How should I edit the script so that when starting the service fails, it writes the service name to a log file? Maybe with try/catch?

Edit:

If anyone stumbles upon this and wants to know how I solved it: After a lot of research I found out that it is extremely difficult to write the output of Receive-Job to a log file. So I chose a different approach: I check if the server is running after the job has finished. Here's the code:

foreach ($item in $report){
Write-Output "Starting Service " $item.Name " on server: " $item.Server
$start = Get-Date
$startSVCblock = {param($item) Start-Service -InputObject (Get-Service -ComputerName $item.Server -Name $item.Name)}
$j = Start-Job -ScriptBlock $startSVCblock -Arg $item
do {
    if ($j.State -ne 'Running') { break} 
    $j | Receive-Job
} while (((Get-Date) - $start) -le $timeout)
$newService = Get-Service -ComputerName $item.Server -Name $item.Name
if ($newService.Status -ne "Running") { LogWrite "$item could not start" }

}

I did, the question is here: http://serverfault.com/questions/731217/powershell-script-to-restart-automatic-windows-services-and-notify-if-failed. If I get the answer, I will edit my post here. — Peter, Oct 23 '15 at 14:54

Email notification when automatic service is not started (Windows Server 2012 R2)

4 Answers4