Easiest way to automatically check EC2 disk space and be alerted if it is running low?

Question

Running the Amazon Linux AMI. It seems that CloudWatch does not check for free disk space. I have a number of servers and ideally don't want to have to configure each one with a mail server, script to check disk space etc.

Is there a simpler way to do this?

Answer 1

Amazon provides scripts for this as of march 2012:

Amazon CloudWatch Monitoring Scripts for Linux: http://aws.amazon.com/code/8720044071969977

Answer 2

There is no way for the EC2 control and monitoring tools to give you this data because the file system of your instances is ONLY accessible by the instance itself. Both the basic architecture of the hardware and the security model demand this limitation. Think about how bad it would be if software outside your computer could poke around at the files on your hard drives!

Here is a low key way to make cron (installed on most systems anyway) check this data for your periodically. Your systems should have the minimum requirements to handle root mail notifications anyway. I recommend having at least a materialistic outgoing mail agent and configuring the root or administrator alias to forward to you on all systems you administer. Many programs including cron expect this configuration.

You could add this to your crontab:

0 0 * * * test $(df / | grep ^/ | awk '{print $4}') -lt 1048576 && echo "Warning: Free disk space is less than 1G on /"

To break that down, this

• Creates a job that runes once a day at 00:00.
• Cron automatically handles emailing the system administrator with the output of jobs. This job only produces output if there is an error or if the disk space is low
• The test command sets up a simple shell comparison using the -lt less than operator and a fixed value equivolent to 1Gb free space.
• The df command tests free space on the / file system
• The grep gets you just the line of output you need instead of the headers df includes.
• The awk get's just the fourth column in the output, the free space number.
• The && says to run the next command only if the first one (the test x -lt y) returns true.

Answer 3

I wrote a script as I needed to check several servers within my EC2 group. It needs a file with a list of each server IP/domain name on a single line.

#! /bin/bash

ADMIN="serveralerts@youraddress.com"
ALERT=85

for SERVER in `cat ~/scripts/servers.txt` do
ssh -i ~/.ssh/yourkey.pem $SERVER df -H | grep -vE '^Filesystem|tmpfs|cdrom' | awk '{ print $5 " " $1 }' | while read output;
do
echo $output
usep=$(echo $output | awk '{ print $1}' | cut -d'%' -f1  )
partition=$(echo $output | awk '{ print $2 }' )
if [ $usep -ge $ALERT ]; then
echo "Running out of space \"$partition ($usep%)\" on $SERVER as on $(date)" | 
mail -s "Alert: Almost out of disk space $usep" $ADMIN
fi
done done

Answer 4

Step by step instructions for setting this up on an EC2 instance with CloudWatch:

http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/mon-scripts.html

Answer 5

I use this script: http://nixcraft.com/shell-scripting/3238-shell-script-check-disk-space-remote-systems.html

Answer 6

Cron is your friend. Put this file into your /etc/cron.daily directory and it will run once per day:

#!/bin/sh
# this script is /etc/cron.daily/diskAlert.cron    
ADMIN="jdoe@maildomain.com"
ALERT=90    
df -PkH | grep -vE '^Filesystem|tmpfs|cdrom' | awk '{ print $5 " " $6 }' | while read output;
do
  usep=$(echo $output | awk '{ print $1}' | cut -d'%' -f1 )
  partition=$(echo $output | awk '{print $2}' )
  if [ $usep -ge $ALERT ]; then
    echo "Running out of space \"$partition ($usep%)\" on $(hostname) as on $(date)" |
    mail -s "Alert: Almost out of disk space $usep%" $ADMIN
  fi
done

NOTE: This script will say that mounted CDROMs are full.

Answer 7

This is a quick PowerShell script that I wrote that runs on our DC in AWS and shoots out an e-mail to a group of recipients if a drive is full. It takes a csv with 2 columns - one titled name with a computer name, and one titled drive with a drive letter. We don't have a mail server in our AWS environment, so I configured it to send through SES. You can also slightly modify the script to just send a report every so often of the drive utilization if you like. Just thought that I'd post this here, since all of the solutions that I found were for Linux instances.

$CSVPath = "c:\Scripts\computerNames.csv"
$computerName = new-object System.Data.DataSet
$computerName = Import-CSV $CSVPath
$AwsUn = "" 
$AwsPw = ConvertTo-SecureString "" -AsPlainText -Force

$cred = New-Object -typename System.Management.Automation.PSCredential -ArgumentList $AwsUn, $AwsPw

Foreach($name in $computerName)
{

    $dl = $name.drive

    $Utilization = Get-WmiObject win32_Volume -ComputerName $name.computerName -Filter "DriveLetter = '$($dl)'"|   Foreach{ “{0:N2}” -f ((1-$_.FreeSpace / $_.Capacity)*100) } 

    if($Utilization -gt 90)
    {
        Send-MailMessage -From Sender to Recipients -subject (
    "$($name.computerName) Disk utilization" )-Body "The $dl drive on the AWS instance $($name.computerName) has $utilization% disk utilization.   Please log in and delete log files or contact the Network Operations team to increase the storage allocated to this instance"  -SmtpServer email-smtp.us-west-2.amazonaws.com -Credential $cred -useSSL -port 25
    } 


}