1

I am trying to use a puppet master 3.1.1 to manage iptables on various servers. My local puppet agent is 2.7.19, the server OS is CentOS 5.4.

/etc/puppet/modules/mycompany/manifests/config/iptables.pp

class base::iptables (
{
  {
    $identity_environment = $::identity_environment
  }

  if ($identity_environment == "production")
  {
    $start_iptables = "true"
    $run_iptables = "running"
  }

  elsif ($identity_environment == "development")
  {
    $start_iptables = "false"
    $run_iptables = "stopped"
  }

  if ($run_iptables != "stopped")
  {
    file { "/etc/sysconfig/iptables":
      ensure => file,
      owner  => root,
      group  => root,
      mode   => 644,
      source => "puppet://path_to_my_conf",
      require => File["/etc/resolv.conf"],
    }

    service { "iptables":
      subscribe => File["/etc/sysconfig/iptables"],
      enable => "$start_iptables",
      ensure => "$run_iptables",
      status => "[[ `iptables -L -n | egrep -v '(Chain|target)' | grep '^[A-Za-z]' | wc -l` != 0 ]]",
    }
  }

  else
   {
    service { "iptables":
      enable =>"$start_iptables",
      ensure =>"$run_iptables",
    }
  }
}

When I run puppet on a development server:

puppet agent --verbose --no-daemonize --onetime --debug | grep iptable

the following occurs:

debug: Puppet::Type::Service::ProviderRedhat: Executing '/sbin/chkconfig iptables' debug: Serviceiptables: Executing '/sbin/service iptables stop' debug: Puppet::Type::Service::ProviderRedhat: Executing '/sbin/chkconfig iptables' notice: /Stage[main]/Base::Iptables/Service[iptables]/ensure: ensure changed 'running' to 'stopped' debug: /Stage[main]/Base::Iptables/Service[iptables]: The container Class[Base::Iptables] will propagate my refresh event debug: Serviceip6tables: Executing '/sbin/service ip6tables status' debug: Puppet::Type::Service::ProviderRedhat: Executing '/sbin/chkconfig ip6tables' debug: Class[Base::Iptables]: The container Stage[main] will propagate my refresh event

Checking manually with service iptables status shows that the service is indeed running, is puppet stopping the service then somehow starting it due to that 'refresh'?

I attempted to alter my manifest according to this similar question but to no avail, 

service { "iptables":
    ensure => "stopped",
    hasstatus => "true",
    status => "true"
}

results in the same thing - puppet checking, then stopping iptables, but yet the service is turned back on.

I do not know if this could be related to this documented race condition bug that has been fixed in a newer puppet version, I do not have the option of updating or implementing a proper method such as puppetlab's firewall module at this time.

UPDATE [20140724]

After some great feedback here I did some debugging/logging from the /etc/init.d/iptables script. Come to find out When puppet was running, no calls to start the service were being made, however the OS still reported iptables as running even if puppet stopped it.

My inclination was to look at the iptables module itself. Sure enough if I removed the iptables module and then ran puppet again, the module would be re-loaded.

UPDATE [20140729]

My solution is listed below as an answer. I will open a new question for continuing to troubleshoot CentOS 6 with this.

Community
  • 1
Shanedroid
  • 21
  • 6
  • It might be interesting to see an excerpt of all resource_statuses related to iptables from your `last_run_report.yaml`. – Zoredache Jul 23 '14 at 17:08
  • Can you provide the relevant sections without the grep? You're losing relevant output since the lines don't contain 'iptable'. – Shane Madden Jul 23 '14 at 17:13
  • @Zoredache - http://pastebin.com/eeRJdKgg Looks like the service was stopped? I see nothing about it being started again, interesting I should be looking at these more often. – Shanedroid Jul 23 '14 at 17:51
  • @ShaneMadden http://pastebin.com/ErhcH6dh – Shanedroid Jul 23 '14 at 17:57
  • 1
    @Shanedroid Looks like; and it got an exit code of 0. Try running `service iptables stop` as root yourself - see what exit code you get (`echo $?`) and see what `service iptables status` shows then. – Shane Madden Jul 23 '14 at 17:59
  • @ShaneMadden http://pastebin.com/g4WQkZhb If I understand correctly puppet must receive != 0 from an init script? http://docs.puppetlabs.com/references/latest/type.html#service-attribute-hasstatus I have seen similar issues documented about using init and puppet as well, such as http://serverfault.com/questions/389361/how-to-use-a-custom-status-command-for-a-service-in-puppet or http://grokbase.com/t/gg/puppet-users/12bhd8e3da/issue-with-service-disabled-and-stopped#20121120jx2sdw2jr74m7vamsrmofwvdeu – Shanedroid Jul 23 '14 at 18:12
  • `status => true` is a rather mind-boggling hack. "Hey, Puppet, assume this service is always running." – Felix Frank Jul 23 '14 at 20:43
  • @FelixFrank I am confused, even if I purposefully set every service reference command attribute manually to be `=> "/etc/init.d/iptables stop"` the service is still running after puppet runs ?!?! – Shanedroid Jul 23 '14 at 20:58
  • @Shanedroid take the advice from ShaneMadden first - run the appropriate service control command from a shell and closely watch its behavior. – Felix Frank Jul 23 '14 at 21:36
  • @FelixFrank I have done this, I can compare with the actual init scripts to see the appropriate returns, aside from that link I posted in my previous comment I do not see a clear and concise place in puppet documentation that indicates exactly what the return value must be for any given service control so I can better understand if I need to manipulate my own service controls to give puppet the appropriate values to prevent the service from being started after it made sure to stop it. – Shanedroid Jul 23 '14 at 21:47
  • From the top of my head: Puppet expects `0` return codes from all invocations, otherwise it will consider them failed. The only exception is the call for determining the `status` (for a resource with the `init` provider and `hasstatus`, that is `/etc/init.d/iptables status` e.g.), because non-zero is acceptable and interpreted as "service is not running". – Felix Frank Jul 23 '14 at 22:10
  • ok @Felix Frank yes that seems to be true as this is documented in a few places that puppet exoects '0' for returm codes and as far as I know that is considered best practice http://goo.gl/uz7HOm However if I purposely manually give puppet nothing bur `0` forcefully - why does this service still remain running even when according to the output it has been stopped? – Shanedroid Jul 23 '14 at 22:33
  • Your debug output indicated that Puppet issued a `service iptables stop`. You claim this works outside of Puppet, but somehow has no effect or gets reverted when performed by Puppet. This does not look like a Puppet problem though. Next debugging step: Put debug logging *into* the misbehaving initscript, audit when it's called by whom with which parameters etc. – Felix Frank Jul 24 '14 at 08:52
  • Let us [continue this discussion in chat](http://chat.stackexchange.com/rooms/15942/discussion-between-shanedroid-and-felix-frank). – Shanedroid Jul 24 '14 at 21:15

1 Answers1

1

I wanted to leave this here as an answer to my question because it worked for me. If you are running CentOS 5.4 this will work and effectively will ensure that the iptables module is unloaded from the kernel. I will ask a new question specifically relating to my continued work with CentOS 6 and puppet.

exec { "chkconfig_iptables":
          onlyif => "/sbin/chkconfig --level 3 iptables",
          command => "/sbin/chkconfig --level 3 iptables off",
          before => exec["kill_iptables"]
     }
exec { "kill_iptables":
          onlyif => "/sbin/lsmod | grep ip_tables",
          command => "/sbin/service iptables stop;/sbin/modprobe -f -r ip_tables"
      }
Shanedroid
  • 21
  • 6