Monitoring server room temperature

Question

I have a small server room with its own AC unit. Recently, the AC died, and the temperature increased from 70 F to > 90 F. We rarely go in this room, so I was lucky that someone happened to notice that the fans were running a lot louder than normal as they walked past the door. It looks like I need a way to be notified when the temperature in that room gets too hot.

What tools are you using to monitor the temperature in your server room? How does this tool notify you of a problem (email, SNMP, etc).

Note: I've read this question on server temperature, but I'm interested in the whole room, not just the inside of a server case.

Edit:

Thanks for all the great responses so far! Many of these products measure much more than just temperature. What else should I be looking at and why?

Answer 1

Take a look a ITwatchdogs. Their weather goose line looks very nice. They monitor temp, light, sound, humidity, etc.

Here's a list of vendors (not APC) that have other products.

What I use in my data center appears to be a discontinued model. What ever you end up using, make sure that:

• They use SNMP, not some special protocol.
• Have remote temperature sensors. (I don't know if the netbotz do ...)

Currently the temperature monitors are hung from the middle of the drop ceiling.

Using a simple network monitoring application (zenoss, munin, nagios, etc) just monitor for threshold violations and create alerts for your pager/email.

You should also look into buying a handheld thermometer and walk around to every part of your datacenter/comm closet (including corners) to find any hotspots.

Answer 2

A couple of specific answers and a suggestion ...

1- There are several makers (linked in other answers) of standalone temp and humidity monitoring devices. These are pretty simple .. typically you mount them where you want to monitor, plug them into the network, and configure them to email alerts. They work well.

2- Most servers can raise SNMP alerts when the temperature gets out of spec, and you can use something like WhatsUp from Ipswitch to accept and do something about those alerts.

You should make a point to visit the server room once or twice a day, perhaps on arrival and after lunch, to check on the temp and humidity as well as examine the equipment for lights or status codes that are not normal, unusual noises, etc. If you can't do it then get someone else to do so.

Answer 3

We use the MiniGoos products from http://www.itwatchdogs.com/. I will admit I wasn't involved in the evaluation process for these, but I really like the product. They allow multiple temperature sensors and they have some nifty add ons like air flow monitoring (which we aren't currently using) and things like door open/close detection (which we are). They have an HTTP interface which could be scraped if you wanted, but also support SNMP traps and polls. We plugged it into nagios and it worked like a champ.

I also second and extend a comment josephkern made in his answer. In addition to walking around with a thermometer to get a feel for the temperature pattern in the room, buy some cheap non-connected thermometers and hang them from the ceiling. They're great backup for reporting thermometers, and you'd be surprised how often a problem will be spotted by a human glancing at the therm as they walk in and out of the room before the warning range of the monitoring thermometers is crossed. The earlier you diagnose an AC/HVAC problem, the longer you will have to fix it before you start damaging components, having to shut down machines, or having machines shut down on their own to protect themselves.

Answer 4

If you want a cheap solution, you can buy USB thermometers for about 10USD. I've got a TEMPer device. I've not got around to using it yet, mostly because it's not entirely supported under Linux. Tollef Fog Heen wrote some support, but I haven't checked to see if it's in the kernel yet. It does ship with Windows drivers, so it just a matter of hooking it up to your monitoring. :)

Answer 5

+1 to Joseph Kern's answer. We use one of ITWatchdog's systems,but I've used other systems as well, and the general idea is great.

ITWatchdogs, and a lot of other vendors, make use of the Dallas 1-wire bus to drive their sensors. The actual sensors are available from Farnell etc for very low prices, and they basically attach to a piece of cat3 cable. You can combine an ITWatchdogs (or similar) system with a lot of hand-assembled 1-wire sensors for a relatively cheap solution.

There are also standard RS232 interfaces to the 1-wire bus, (such as this one) with varying levels of complexity, which mean you can directly attach monitoring gear to servers, in case you don't want to run additional gear (remote POP with only one machine, for example)

Answer 6

We have a similar setup (private suite at a data center with its own A/C to cool the room) and use TemPageR from AVTECH to monitor the temperature in the room and get e-mail/SMS notices if the temperature goes over a certain threshold. We use the internal probe to monitor ambient temperature, and a remote probe to monitor the air coming directly out of the A/C blower to make sure it's cold.

We like this because it's a completely standalone device that doesn't need to be connected to software on another server to function. AVTECH makes a full line of environmental monitoring systems including temperature and water sensors.

Answer 7

A more inexpensive route would be to use a temperature monitor that hooks into your buildings security system. If the temperature reaches a set level, people on the security system call list are contacted.

Answer 8

Although we have room-wide environmental alarms not everyone has that luxury, what I'd suggest you do is look at the servers you have, many already have temperature sensors that, in conjunction with free systems management software, allow for SNMP-traps and emails to be sent once these sensors go over a particular level. In particular I know that both HP and IBM servers have these and I'm pretty sure Dell do too. Let me know what make/models you have and I'll try to dig something up.

Answer 9

Low tech: I keep an indoor/outdoor thermostat in my office. The outdoor probe runs through a small channel in the wall to the neighboring server room. At a glance, I can see the temp in there. The thermometer also tracks min/max temp ranges so I check and reset those once a week.

Not in the neighboring office, just run the probe under the door and hang the thermometer in the hall outside the room, maybe?

I also rely on the temp monitors in my HP servers for a more automated approach, but the low tech one has actually been more useful in the past. I glance at the temp everytime I enter my office.

Answer 10

A guy I know has a probe with GSM transmitter on it stuck to one of the walls in the server room. It basically calls him if something is seriously wrong, and sends SMS if it's a minor problem.

I'm going to buy one myself, I'll post back when I have the product name.

Edit: It's something like the SCOM-100 from Infinite Ltd.

Answer 11

We use a Sensatronics device (http://www.sensatronics.com/) and monitor it with SNMP from SolarWinds Orion (http://www.solarwinds.com/products/orion/).

We have had several AC outages in the lab in the past and this has alerted us (through email alerts configured in Orion) in enough time to get repair people on site in time to fix it before the temperature got too hot. One of these instances was after hours, so having the email alert was critical.

Answer 12

There's also the LM Sensors project.

http://www.lm-sensors.org/

Answer 13

We tied a general temperature sensor to our security system. So when a temperature alarm occurs its treated like any other kind of building alarm (door, window break etc.). This way it tied into our general security process for notification.

We chose this route becuase it was simpler and less to go wrong or fail at the critical time.

Answer 14

We use an Enviromux-Mini from NTI to monitor thermals in our wiring closet. Its query-able by snmp so you can graph over time, but you can also have it trigger traps or email alerts if thresholds are crossed. Its small, simple and just plain works. It can be extended for other environmental alerts such as humidity, water detection and contact closures as well.

Also, if you use a switched cdu/pdu in your room, some of them can be extended with thermal monitors such as the Sentry CDU from Servertech.

Answer 15

I found this on YouTube. I've been using ITW's products for years without any problem - I did lots of research when searching for environment monitors and these are the best I found for the price. http://www.youtube.com/watch?v=5Brv0vsHIlA&feature=player_embedded

Answer 16

For monitoring server / data room I am using GSM control panel. It can monitor temperatures and can send information by SMS to several numbers. Furthermore, it can be used as alarm system for security, power failure, fire protection or even control electrical devices through SMS (ex. power on conditioner). System can be configured by SMS or configuration tool via USB. More information about this device can be founded http://www.eldes.lt/en/gsm-remote-control-alarm-systems/esim262.html

Answer 17

Do you need to check only temperature inside the room?

I use Sky Control system to check our server room. We've put temperature sensor, leakage, humidity and smoke sensors and it has already saved us twice. Can be easily set up even without technical knowledge.

Answer 18

The APC NetBotz range of products is pretty good for environmental monitoring, the various products allow you to monitor temperature, humidity, air flow, sound, etc. It can also be used with cameras and door sensors to provide an all round security and monitoring solution.

Answer 19

If you are a bit of a hobbyist you could try this this would be a fun little project.

Pick up a simple PLC

On the an in contact on the PLC have a thermo switch and set it for the temp you want the alarm at.

On the out contact of the PLC have it go to a light or alarm bell outside the server room to alert you.

Answer 20

We make use of the motherboard/CPU temperature on a couple of our servers. If several of these alarm at once, it's pretty clear what the cause is. No extra equipment necessary.

Answer 21

We have a couple things going on in the data center for temperature monitoring.

SNMP temperature monitors. We're using Sensatronics EM1 monitors to be precise. They have both temperature and humidity sensors. I just recently added 3 more sensors to the unit so we're monitoring four different places in our main data center. We're polling SNMP data from them and running it through MRTG for trending and graphing. We also have bigbrother alerting us when the temperature or humidity hits certain thresholds. In theory the physical plant people at the university are also monitoring the room, but they don't seem to notice or care as often as we do.

We also have a bunch of standing thermometer/humidity sensors that are non automated dials. When we suspect we have a hot spot we can just stick unit there and get a visual check on it in a minute or less.

In the future I'd love to pull the data off our AC units as well, but I haven't had time for that yet.

EDITED: You should be looking for humidity between 40% and 55%. If the room is too dry static electricity can be a problem. Too wet and your drives will rust in place.

Here is a link to a pretty decent overview of data center physical standards, including temperature and humidity. Also information on arranging a data center, and cabling standards.

Answer 22

I have an APC Environment Management Unit installed in the server room monitoring mostly temperature and other environment factors. It's a IP-based device that you can manage it from the browser.

http://www.apcc.com/resource/include/techspec_index.cfm?base_sku=ap9340&tab=documentation

Answer 23

In my server room I use two products. one is a backup in case the other fails. The first is AVTECH 11E this unit uses a probe for the front and back of my three racks. I added the monitor for power so it will alert me when my room is down. I have a backup unit Senturion Environmental Monitor which also has the probes but also takes the Light in the room, air flow and temp in the rack, and I can add cameras to the unit. I depend on my Senturion more than my AVTECH as it is a little more accurate than the AVTECH.

Answer 24

I will second kentchen's post. We use the APC EMU unit and it seems to work pretty well in letting us know when something is up.

Answer 25

For less than 100 USD you can get the Lascar "Real-Time USB Temp and Humidity Monitor" from Microdaq.

The software allows you send alerts via email and export the data, though it seems to be a resource hog (at least in is on the PC I run it on - YMMV).

Answer 26

We use the APC EMU Units, and they work well, but for the whole room, we recently switched over to an active-RFID based system from rfcode.com. It uses temp/humidity tags that can be affixed on any flat surface. We have one RF receiver covering a 15K sf room.

Answer 27

We use a number of temperature/humidity sensors that connect through USB. We picked up from raphnet.net. They are inexpensive and work great. (If your a tinkerer, the schematics are available at no cost.) The software for reading the temperature is open source, easy to use and can easily be put into snmp (or probably just about any monitoring system).

Answer 28

We use Ingrasys Insentry. There are 2 ethernet modules we keep one at the rear of the server rack and one at the front and it monitors Temperature and humidity. It uses snmp and you view it through a web console. It also sends alerts to my email when one of the coditions are met (eg. too hot or humidity too high or low or too cold etc). Piece of cake to configure. Set them and forget them till the aircon dies.

Answer 29

Many higher-end servers from vendors like HP will include a plethora of sensors that can be queried via IPMI or SNMP. What we do is take every one of these that we have, our UPS's, and any other device that happens to support temperature monitoring of any kind, and stick them in to our monitoring system (Zenoss).

We then aggregate the data in to a single graph and this gives us a really good picture of what the temperature is like in the room.

What you're looking for in particular is devices that will allow you to monitor the temperature at the inlet fan, and the exhaust fan. In my opinion, that gives you an even more accurate idea of what the temperature is like because then you know exactly what kind of air is coming in and out of your server.

Answer 30

We use dallas-sensors on a serial Port and a Perl-munin Plugin to monitor temperatures. Very lowcost and homemade, but works like a charm.

Answer 31

On the hobbist side of things, you can do with a Raspberry Pi and some temperature sensors - like the TC74 which is available in 3.3v for direct I2C connection to the Raspi. As an added bonus you get a full linux system running there so you can implement other things as well, but it is still a DIY patchwork and I wouldn't ever call it "production ready".

Head on to Joseph Kern's excellent answer for more serious products.

Answer 32

We just had the same thing happen in our small server room - but one of our RAIDs FAILED!! I did some research and found a very inexpensive solution. It emails me when the room reaches a certain temperature or it can email me every few hours with the latest temperature reading. Takes less than 5 minutes to hook up and was around $130. Here is the link:
http://www.temperaturealert.com

Answer 33

1. The security alarm is triggered on temperature (and humidity) as well, the security company will respond like it was an intrusion
   • day-time with a call to reception to check what is wrong
   • outside of office time with on-location visit within 5 minutes with access to the server room and responsibility to try and fix the problem (like a blown breaker) and of course a phone list to start calling to get to someone
2. Monitoring system is also monitoring temperature with a separate sensor (as well as all individual server and network equipment internally reported temperatures) and trigger normal monitoring alarms (mail and battery-backed gsm modem sending out test messages)

For this we just used ActiveExperts Network Monitor, enterprise license was very cheap and it did gsm notifications as well as automatic, scripted response and action to alerts.

Answer 34

Someone in a mailing-list, that I'm member of, recommends ENVIROMUX-MINI.