1

I am experiencing a somewhat weird issue with conntrackd. I have created an environment with an active/backup scenario where sessions will be replicated to the backup after a failover and vice versa. I have followed the official manual of the tool and other tutorials that pretty much use the same config.

The issue

I create several tcp sessions using wget or ssh and I can see the sessions being created in conntrack -L and conntrack -E -p tcp

root@master:/home/master# conntrack -L
udp      17 22 src=x.x.x.190 dst=x.x.x. sport=138 dport=138 [UNREPLIED] src=x.x.x. dst=x.x.x.190 sport=138 dport=138 mark=0 use=1
udp      17 4 src=x.x.x.9 dst=255.255.255.255 sport=11235 dport=11232 [UNREPLIED] src=255.255.255.255 dst=x.x.x.9 sport=11232 dport=11235 mark=0 use=1
udp      17 21 src=x.x.x.26 dst=255.255.255.255 sport=17500 dport=17500 [UNREPLIED] src=255.255.255.255 dst=x.x.x.26 sport=17500 dport=17500 mark=0 use=1
udp      17 14 src=x.x.x.212 dst=x.x.x. sport=137 dport=137 [UNREPLIED] src=x.x.x. dst=x.x.x.212 sport=137 dport=137 mark=0 use=1
udp      17 18 src=x.x.x.50 dst=x.x.x. sport=62401 dport=3052 [UNREPLIED] src=x.x.x. dst=x.x.x.50 sport=3052 dport=62401 mark=0 use=1
tcp      6 299 ESTABLISHED src=192.168.0.7 dst=x.x.x.58 sport=46026 dport=80 src=x.x.x.58 dst=192.168.0.7 sport=80 dport=46026 [ASSURED] mark=0 use=1

root@master:/home/master# conntrack -E -p tcp
    [NEW] tcp      6 120 SYN_SENT src=192.168.0.7 dst=x.x.x.58 sport=46030 dport=80 [UNREPLIED] src=x.x.x.58 dst=192.168.0.7 sport=80 dport=46030
 [UPDATE] tcp      6 60 SYN_RECV src=192.168.0.7 dst=x.x.x.58 sport=46030 dport=80 src=x.x.x.58 dst=192.168.0.7 sport=80 dport=46030
 [UPDATE] tcp      6 432000 ESTABLISHED src=192.168.0.7 dst=x.x.x.58 sport=46030 dport=80 src=x.x.x.58 dst=192.168.0.7 sport=80 dport=46030 [ASSURED]

But I cannot see them in internal cache of master or external cache of backup, I can only see udp sessions. (I cannot post the internal and external cache, they are too big. Imagine just like the first block of code but only udp sessions). Which means that tcp sessions are destroyed on failover and not replicated. My gwet pauses and my ssh connection freezes. Even if master takes over again sessions are already lost.

The configuration of conntrackd is:

Sync {
    Mode FTFW {
        DisableExternalCache Off
        CommitTimeout 1800
        PurgeTimeout 5
    }

    UDP{
        IPv4_address 192.168.0.4
        IPv4_Destination_Address 192.168.0.5
        Port 3780
        Interface eth1
        SndSocketBuffer 1249280
        RcvSocketBuffer 1249280
        Checksum on
    }
}

General {
    Nice -20
    HashSize 32768
    HashLimit 131072
    LogFile on
    Syslog on
    LockFile /var/lock/conntrack.lock
    UNIX {
        Path /var/run/conntrackd.ctl
        Backlog 20
    }
    NetlinkBufferSize 2097152
    NetlinkBufferSizeMaxGrowth 8388608
    Filter From Userspace {
        Protocol Accept {
            TCP
            UDP
            ICMP # This requires a Linux kernel >= 2.6.31
        }
        Address Ignore {
            IPv4_address 127.0.0.1 # loopback
            IPv4_address x.x.x.58
            IPv4_address x.x.x.56
            IPv4_address x.x.x.59
            IPv4_address x.x.x.7
            IPv4_address 192.168.0.4
            IPv4_address 192.168.0.5
            IPv4_address 192.168.0.6
            IPv4_address 192.168.0.7
            IPv4_address 192.168.100.100
        }
    }
}

If I use DisableExternalCache On as this question suggests my internal and external caches are both empty (even udp sessions are lost). The same applies if I use Address Accept instead of Address Ignore. DisableExternalCache On is also advised to be used in an active/active scenario instead of an active/backup that I am looking for.

Firewall rules are set to accept and these additional rules are added (taken from netfilter testcase)

[1] iptables -P FORWARD DROP
[2] iptables -A FORWARD -i eth0 -m state --state ESTABLISHED,RELATED -j ACCEPT
[3] iptables -A FORWARD -i eth1 -p tcp --syn -m state --state NEW -j ACCEPT
[4] iptables -A FORWARD -i eth1 -p tcp -m state --state ESTABLISHED -j ACCEPT
[5] iptables -A FORWARD -m state --state INVALID -j LOG
[6] iptables -I POSTROUTING -t nat -s 192.168.0.3 -j SNAT --to 192.168.1.100

I have tried other configurations, other sync modes, scripts that commit changes and flush caches when appropriate. But I cannot seem to find why tcp sessions are not shown in cache. Any thoughts? Am I missing something?

Jimmy_A
  • 81
  • 7

1 Answers1

2

After a lot of reading, re-configuring and external help, the problem was solved (finally the nightmare is over).

The problem was in the Address Ignore part of the configuration, not the rule-set that I thought at first.

In the tutorials that I followed they say:

The "Address Ignore" block should list ALL the IPs the firewall has

But they didn't say that the Address Ignore should list ALL the IPs the firewall has on Local Interfaces.

Placing additional addresses, such as a test host per say, will ignore all the traffic generated from that host. That's why in the expectation table I was able to see the session, but not in cache. Which means that the Address Ignore block should list only it's own (and maybe the Backup's) IP addresses (loopback, ext, int, VIP).

PS: Another thing that should be mentioned is that the firewall should be set to masquerade the Firewall IPs. If not, on failover the VIP changes owner but the session will still be looking for the IP of the machine that it's created.

To overcome that an snat rule should be set.

Jimmy_A
  • 81
  • 7