4

Our CEO wants us to check all of our 100+ hosted/partner websites. Not only does she want to check if they are up and running, but she would like a diff analysis on the websites so that we can keep a log of any changes that may have occurred from test to test. Is there an open source script or app that can handle this?

tacos_tacos_tacos
  • 3,220
  • 16
  • 58
  • 97
  • How often would the interval need to be from one test to another? Nagios, Zenoss, etc. typically poll on a 5 minute interval, but that seems to frequent for a 'website diff'. – Stefan Lasiewski Sep 26 '11 at 17:36
  • I believe she wants this daily... We have Nagios running at the moment so I had considered the Nagios approach... I'll see which plugins are out there. – tacos_tacos_tacos Sep 26 '11 at 19:26

2 Answers2

3

The up and running part is easy, you could use a whole range of free tools for that, like Nagios or Cacti or Zenoss, but the diff thing is a lot harder.

You can quite easily write your own plugins to Nagios, so you could implement that strange function yourself and have Nagios do the testing. It would take a bit of work, but it could be done.

Bart B
  • 3,419
  • 6
  • 30
  • 42
1

I think you first need to sit down with your CEO and find out why they want this (and how much they're willing to spend to get it done).

Up and running is reasonable and simple -- any decent monitoring system can do this (See Nagios, InterMapper, OpenNMS). Plan on a dedicated server for this, and parlay it into monitoring internal services too.

Diffs between each run are not really reasonable IMHO - especially with "normal" (5 minute) polling intervals to see if the site is up or down (diffing requires downloading the whole site, which takes lots of space and time -- especially if they care about things like changed images/PDFs).
You can make this happen if there's a good business need for it, but you'd be talking custom code (something like WebWhacker, but command-line/noninteractive to download the website, then a recursive diff), and disk space/network bandwidth.

voretaq7
  • 79,345
  • 17
  • 128
  • 213
  • There's no reason the diff test has to be run as often as the up test. It's trivial in Nagios to set different tests to run at different intervals, you could do a daily diff while poling to see if the service is live every 5 mins. – Bart B Sep 27 '11 at 13:36
  • @BartB - True, but this doesn't make the diff test much more reasonable: We're still downloading the whole site and doing a comparison, so we need 2*sizeof($site) disk space for each site, we're still transferring N sites of data every $interval, and there's going to be CPU time/disk IO involved to do the diff. -- Not something I'd commit to without a good business reason to back up the hardware you're eventually going to need when the inevitable scope creep appears... – voretaq7 Sep 27 '11 at 15:52