5

I need to remotely periodically check if a Linux file system is mounted read only on lots of servers. The only tools at my disposal in this instance are the usual CLI scripting languages (BASH/Perl/Python), SSH and SNMP.

My thoughts so far are:

  • Share SSH key and execute the mount command remotely, grep/awk-ing for the RO flag on the given mountpoint.
  • Check via SNMP, although I'm not certain if this information is available or how easy that is.
  • Attempt to SCP a file across, or remotely touch a file.

The SNMP method is the nicest/cleanest I can think of, however I'm at a loss on how to do this as CLI SNMP seems like a nightmare, especially if I need to run through a decision process to find an index based on a given mountpoint.

Ideally I'd like to pull this off using only BASH (inc. sed/awk/grep etc) and SNMP but need some guidance on where to begin! snmpd is already instaled and configured for v2 on all the servers in question.

The final use of this script will be as a Nagios command.

Many thanks in advance for any help!

Note We already have cacti/nagios set up monitoring 100s of servers, I forget to mention this when originally posting!

SimonJGreen
  • 3,195
  • 5
  • 30
  • 55
  • Use the following via ssh: http://serverfault.com/questions/193971/determine-if-filesystem-or-partition-is-mounted-ro-or-rw-via-bash-script – S19N Jan 07 '12 at 00:41
  • Having you considered using something like monit instead? It has tests specifically for checking the state of mounted filesystems. – Zoredache Jan 07 '12 at 01:25
  • @Zoredache Monit is to distributed for this use case unfortunately. We already have cacti/nagios set up monitoring 100s of servers, this is something we'd like to be able to just drop in on top. I should probably have mentioned that as it's quite an important detail! – SimonJGreen Jan 07 '12 at 12:27

4 Answers4

5

In your snmpd.conf check out the exec argument. I use this to remotely poll the usercount of some of my IRC servers, like this:

exec usercount-irc /home/irc/usercount-irc.sh

To poll this you snmpwalk .1.3.6.1.4.1.2021.8, and would get something like this:

# snmpwalk -v 1 -c public localhost .1.3.6.1.4.1.2021.8

UCD-SNMP-MIB::extIndex.1 = INTEGER: 1
UCD-SNMP-MIB::extNames.1 = STRING: usercount-irc
UCD-SNMP-MIB::extCommand.1 = STRING: /home/irc/usercount-irc.sh
UCD-SNMP-MIB::extResult.1 = INTEGER: 0
UCD-SNMP-MIB::extOutput.1 = STRING: 558
UCD-SNMP-MIB::extErrFix.1 = INTEGER: 0
UCD-SNMP-MIB::extErrFixCmd.1 = STRING:

Check the Extending Agent Functionality part of http://linux.die.net/man/5/snmpd.conf for further details!

Edit 1:
UCD-SNMP-MIB:dskTable contains information about mounted volumes and usage. It is located under OID .1.3.6.1.4.1.2021.9 and documented at:
http://www.oidview.com/mibs/2021/UCD-SNMP-MIB.html
http://www.net-snmp.org/docs/mibs/ucdavis.html#dskTable

But I see now it doesn't contain mountflags, so you can't see read only there. Gah. Back to the script above and how to call it! :P

Mattias Ahnberg
  • 4,039
  • 18
  • 19
  • 1.3.6.1.4.1.2021 is the MIB to UCD-SNMP MIB tree, .8 is where external scripts is placed; see here for a view of the MIB tree: http://www.oidview.com/mibs/2021/UCD-SNMP-MIB.html – Mattias Ahnberg Jan 07 '12 at 01:40
  • Thanks @Mattias that's not a bad idea. I've used the exec part of snmpd in the past to monitor specifics from other processes so it's a possibility. Unfortunately it would mean going on to 100s of Linux servers to set this up, but it's one to bare in mind! – SimonJGreen Jan 07 '12 at 12:32
  • I was senile and slow last night, actually there is already a MIB tree for diskspace in UCD-SNMP, its called UCD-SNMP-MIB:dskTable and can be found under OID .1.3.6.1.4.1.2021.9, you can even see it listed under the URL I gave above. D'uh. :P Sorry for the confusion. – Mattias Ahnberg Jan 07 '12 at 18:14
  • D'uh. And AGAIN senile, that MIB tree doesn't contain mount point flags, so I believe you're still forced to use the exec of snmpd or check with, for example, SSH. :/ – Mattias Ahnberg Jan 07 '12 at 18:19
  • I came to the same conclusion sadly :( Hey ho, thanks for the answer! – SimonJGreen Jan 09 '12 at 00:12
2

For most "how do I do X in nagios?", you'll find that someone has already written a plugin that will do all of, or at least some of, what you need.

In this case, you could probably use check_ro_mounts in combination with the 'negate' plugin (part of standard nagios plugins), or just modify it to suit your needs:

http://exchange.nagios.org/directory/Plugins/Uncategorized/Operating-Systems/Linux/check_ro_mounts/details

Keith
  • 4,627
  • 14
  • 25
  • That's true in most instances and would normally be my first stop. In this case however we'd like to avoid deploying a script out to 100s of servers if possible. It does look like though the RW state is not represented through the standard snmp mounts MIB so something like this is going to be the way. – SimonJGreen Jan 11 '12 at 21:56
1

Another way is to use Perl's, Ruby's Net::SSH or Python's paraminko libraries to remotely login and (no need to setup keys beforehand) then parse the contents of the 'mount' command.

dtbnguyen
  • 312
  • 1
  • 5
  • I'm leaning more towards this answer although sadly would mean storing a password in plain text. It's a strong contender! – SimonJGreen Jan 11 '12 at 21:59
0

Management and confirmation of mount points and their flags could be handled through Puppet. See How to manage mounted partitions (fstab + mount points) from puppet

Community
  • 1
Jeff Ferland
  • 20,239
  • 2
  • 61
  • 85
  • Unfortunately checking the state of mounts can't justify a major change like rolling out puppet. Also in our scenario we need the management of these servers decentralised which kinda goes against the puppet way! Thanks anyway though. – SimonJGreen Jan 11 '12 at 21:59
  • @SimonJGreen Puppet with variables and various configuration "blocks" that can be contorted to amazing things. Just feeling the need to evangelize. – Jeff Ferland Jan 11 '12 at 22:48