14

Today my iptables nat on the host system stopped working and I have no clue what happend! (That's very bad, I know)

All commands are executed as root user.

If I run $ iptables -t nat -L I get the following error message:

$ iptables -t nat -L
iptables v1.4.7: can't initialize iptables table `nat': Table does not exist (do you need to insmod?)
Perhaps iptables or your kernel needs to be upgraded.

There are no further updates available. I also restarted the server with several older kernels, but I always get the same error message.

My server is running on CentOS with the official OpenVZ kernel in the latest version.

$ uname -r
2.6.32-042stab088.4

Also tested kernel version: 2.6.32-042stab85.20 and 2.6.32-042stab084.26

First kernel in grub.conf:

title OpenVZ (2.6.32-042stab088.4)
root (hd0,1)
kernel /vmlinuz-2.6.32-042stab088.4 ro root=/dev/md2 rd_NO_LUKS rd_NO_DM nomodeset crashkernel=auto SYSFONT=latarcyrheb-sun16 LANG=en_US.UTF-8 KEYTABLE=de
initrd /initramfs-2.6.32-042stab088.4.img

The following updates were installed:

$ rpm -qa --last
vzctl-4.7.2-1.x86_64                          Mon 05 May 2014 03:25:16 AM CEST
vzctl-core-4.7.2-1.x86_64                     Mon 05 May 2014 03:25:14 AM CEST
util-linux-ng-2.17.2-12.14.el6_5.x86_64       Wed 30 Apr 2014 11:37:19 PM CEST
libuuid-2.17.2-12.14.el6_5.x86_64             Wed 30 Apr 2014 11:37:18 PM CEST
libblkid-2.17.2-12.14.el6_5.x86_64            Wed 30 Apr 2014 11:37:18 PM CEST
vzkernel-2.6.32-042stab088.4.x86_64           Mon 26 Apr 2014 09:01:00 AM CEST
nss-softokn-freebl-3.14.3-10.el6_5.x86_64     Sat 26 Apr 2014 09:01:00 AM CEST
nss-softokn-3.14.3-10.el6_5.x86_64            Sat 26 Apr 2014 09:01:00 AM CEST
bridge-utils-1.2-10.el6.x86_64                Tue 15 Apr 2014 02:22:41 PM CEST
openssl-1.0.1e-16.el6_5.7.x86_64              Wed 09 Apr 2014 10:14:03 AM CEST
...

lsmod gives me the following:

$ lsmod | grep ip
iptable_nat             6302  0
nf_nat                 23213  2 iptable_nat,vzrst
nf_conntrack_ipv4       9946  3 iptable_nat,nf_nat
nf_defrag_ipv4          1531  1 nf_conntrack_ipv4
nf_conntrack           80281  5 iptable_nat,vzrst,nf_nat,nf_conntrack_ipv4,vzcpt
ip6t_REJECT             4711  0
ip6table_mangle         3669  0
ip6table_filter         3033  0
ip6_tables             18988  2 ip6table_mangle,ip6table_filter
iptable_mangle          3493  0
iptable_filter          2937  0
xt_multiport            2716  0
ipt_REJECT              2399  0
ip_tables              18119  3 iptable_nat,iptable_mangle,iptable_filter
ipv6                  322519  35 vzrst,ip6t_REJECT,ip6table_mangle

Running modprobe gives the following error:

$ modprobe /lib/modules/2.6.32-042stab088.4/kernel/net/ipv4/netfilter/iptable_nat.ko
FATAL: Module /lib/modules/2.6.32_042stab088.4/kernel/net/ipv4/netfilter/iptable_nat.ko not found.

But the file does exist:

$ ll /lib/modules/2.6.32-042stab088.4/kernel/net/ipv4/netfilter/iptable_nat.ko
-rwxr--r-- 1 root root 16K Apr  3 16:20 /lib/modules/2.6.32-042stab088.4/kernel/net/ipv4/netfilter/iptable_nat.ko*

Any ideas?

lszrh
  • 683
  • 3
  • 7
  • 15

4 Answers4

10

Due to the update of vzctl from 4.6 to 4.7 they changed the setting of nf_conntrack to be disabled by default. (https://openvz.org/Download/vzctl/4.7/changes)

Corresponding commit message:

...

Disable conntrack for VE0 by default

IP conntrack functionality has some negative impact on venet performance (uo to about 10%), so they better be disabled by default.

...

(Source: http://git.openvz.org/?p=vzctl;a=commit;h=a191a462579ee)

Solution:

In dependence of distribution it is somewhere in /etc directory. Find it:

$ sudo grep -R "options nf_conntrack ip_conntrack_disable_ve0=1" /etc/modprobe.d/

and replace the "1" with "0":

options nf_conntrack ip_conntrack_disable_ve0=0

reboot your system

(Source: https://bugzilla.openvz.org/show_bug.cgi?id=2943#c5)

Alternatively to rebooting:

rmmod iptable_nat
rmmod nf_nat
rmmod nf_conntrack_ipv4
rmmod nf_conntrack

And nat now working

All sources:

Changelog for vzctl 4.7: https://openvz.org/Download/vzctl/4.7/changes

Bug report with solution in comments: https://bugzilla.openvz.org/show_bug.cgi?id=2943

Bug report with shortened solution description: https://bugzilla.openvz.org/show_bug.cgi?id=2755#c4

Corresponding GIT commit: http://git.openvz.org/?p=vzctl;a=commit;h=a191a462579ee

Related Wiki article: https://openvz.org/NAT#IP_conntracks

Tim Brigham
  • 15,465
  • 7
  • 72
  • 113
lszrh
  • 683
  • 3
  • 7
  • 15
4

You should be super user to run iptables. Therefore, run it as root.

$ iptables -t nat -L
iptables v1.4.19.1: can't initialize iptables table `nat': Permission denied (you must be root)
Perhaps iptables or your kernel needs to be upgraded.

# iptables -t nat -L
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination
[...]
Spack
  • 1,594
  • 13
  • 22
1

Seems like something is actually wrong with the file? What if you try to strace modprobe /lib/modules/2.6.32-042stab088.4/kernel/net/ipv4/netfilter/iptable_nat.ko? Does the system really return a ENOENT (No such file or directory)? I can't imagine this would be it, but maybe SELinux is messing with the file? And finally, maybe the file/filesystem is corrupt. md5sum it with a known good version. Or just remove it and copy a new one altogether. Hopefully it's not the filesystem.

Try this: https://stackoverflow.com/questions/3140478/fatal-module-not-found-error-using-modprobe

Community
  • 1
chizou
  • 457
  • 3
  • 8
  • 15
0

I found another solution to this problem! On the node I could not remove 

rmmod iptable_nat
rmmod nf_nat
rmmod nf_conntrack_ipv4
rmmod nf_conntrack

in the Official Guide, I found:

vzctl set VZID --netfilter full --save --setmode restart

Дмаи Летов
  • 1
  • 4