5

I'm trying to test new network monitoring code for an application that runs on a Debian appliance. I am currently tasked with ensuring that an SNMP trap is thrown when external network shares (such as S3 buckets) are mounted with FUSE and the connection is broken.

The problem is that to make the share inaccessible, I would apparently have to make files unwritable (or block every possible S3-related IP with iptables) and I cannot block write access if the bucket is meant to be read from and written to by the root user, and is monitored by the root user.

Is there some way to make this program, running as root, think the bucket went down?

P.S. I cannot make these buckets using chattr because, afaik, S3 doesn't support the concept of attributes like immutability.

jajavoli
  • 53
  • 3

3 Answers3

3

There are so many ways to simulate something going down. You've already said one which is to use iptables block rules.

Another really simple way - unplug the network router so that your monitoring software can still see the host, but the host can't talk to S3 (or anything else for that matter). To me, this is the simplest but may in some cases not be appropriate.

Another option is to edit the hosts file on the server, and add some dummy IP's for the S3 hostnames so that they resolve to the wrong place. i.e. use an IP that's unassigned to anything. Therefore all S3 requests from host will attempt to go there but won't be able to. When you're done, just delete the IP's out of the hosts file and everything will come back to life without a restart.

hookenz
  • 14,132
  • 22
  • 86
  • 142
  • I have tried most of these, save perhaps for physically unplugging. Oddly, none of that seems to work. I guess that brings us to the question of why the application isn't sending those traps, which I guess is more of a StackOverflow question... – jajavoli Feb 03 '16 at 19:43
  • 1
    I tried all of those methods (save for physically unplugging, which would be hard on AWS) and they all worked. Some of the SNMP traps weren't working within the AWS environment, though. Nice, exhaustive answer. – jajavoli Feb 08 '16 at 20:57
1

Why not just check for the existence of a specific file in the S3 bucket. If it's not there, throw your SNMP trap.

Creating/deleting this file will make it very simple to verify that your script is working properly.

EEAA
  • 108,414
  • 18
  • 172
  • 242
1

I cannot block write access if the bucket is meant to be read from and written to by the root user, and is monitored by the root user.

This story starts and ends with the fact that this bucket needs to be accessed via IAM. Your root account on AWS exists only to setup the IAM roles. Once you've got that setup, you can alter the IAM privileges to test.

Jeff Ferland
  • 20,239
  • 2
  • 61
  • 85
  • It seems I was being unclear. I meant I was trying to deny access to the root user on the Debian appliance. Silly, I know, but that seems to be what is wanted... – jajavoli Feb 03 '16 at 19:47
  • @jajavoli that's a horse of a different color. Why no sudo your script as non-root to test it? – Jeff Ferland Feb 03 '16 at 19:58