Load-balancing: response packets not routed - iptables

Question

I have a problem with iptables. I'm trying to setup a load balancer between 2 uplinks. I can mark packets and connections, use ip rule to choose routing table (so changing the gateway). But, when I set the mark in order to route the packets via a different gw than the router default gw, the response packet doesn't get routed properly.

Example while pinging 8.8.8.8:

17:41:48.061404 IP x.x.x.x > google-public-dns-a.google.com: ICMP echo request, id 2622, seq 1, length 64
17:41:48.079664 IP google-public-dns-a.google.com > x.x.x.x: ICMP echo reply, id 2622, seq 1, length 64

I got this using tcpdump on the router. The second packets arrives to the router but it never gets router correctly to the client requesting. This happens with every packet.

Configuration:

• copied the main table to wan_one and wan_two WITHOUT default gw.
• main table has a default gw router via GW 1
• applied different def gws to wan_x tables (GW 1 and GW 2).

• # Match the packets
  ip rule add fwmark 1 lookup wan_one prio 1024
  ip rule add fwmark 2 lookup wan_two prio 1025
  

• # Packets from router 1 or 2 gets routed through correct table
  ip rule add from [ROUTER IP FOR GW 1] table wan_one prio 1026
  ip rule add from [ROUTER IP FOR GW 2] table wan_two prio 1027
  

• iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark
  iptables -t mangle -A PREROUTING --match mark --mark 1 -j ACCEPT
  iptables -t mangle -A PREROUTING --match mark --mark 2 -j ACCEPT
  iptables -t mangle -A PREROUTING -i eth0 -m state --state NEW -m mark --mark 0 -j MARK --set-mark 1
  iptables -t mangle -A PREROUTING -i eth1 -m state --state NEW -m mark --mark 0 -j MARK --set-mark 2
  iptables -t mangle -A PREROUTING -m state --state NEW -m mark --mark 0 -i eth2 -j MARK --set-mark 2
  iptables -t mangle -A PREROUTING -j CONNMARK --save-mark
  

Let's assume link 1 is on eth0, link 2 is on eth1 and LAN is on eth2.

How can I get it working?