0

We have a requirement to reach our Couchbase private IP through from an outside network. We intend to accomplish this through a NAT Gateway which we have created through IP tables.

I can telnet the Nat Gateway's Public IP with the ports that are listening on the private IP of our Couchbase server. But our application which requires a Couchbase connection cannot make that connection. We're not sure why this is not working. Any takers?

# sudo iptables -S
-P INPUT DROP
-P FORWARD DROP
-P OUTPUT ACCEPT
-N ICMP
-N TCP
-N UDP
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -m conntrack --ctstate INVALID -j DROP
-A INPUT -p udp -m conntrack --ctstate NEW -j UDP
-A INPUT -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j TCP
-A INPUT -p icmp -m conntrack --ctstate NEW -j ICMP
-A INPUT -p udp -j REJECT --reject-with icmp-port-unreachable
-A INPUT -p tcp -j REJECT --reject-with tcp-reset
-A INPUT -j REJECT --reject-with icmp-proto-unreachable
-A FORWARD -i eth0 -o eth1 -p tcp -m tcp --dport 80 --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -i eth0 -o eth1 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i eth1 -o eth0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i eth0 -o eth1 -p tcp -m tcp --dport 81 --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -i eth0 -o eth1 -p tcp -m tcp --dport 8091 --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -i eth0 -o eth1 -p tcp -m tcp --dport 8092 --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -i eth0 -o eth1 -p tcp -m tcp --dport 11207 --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -i eth0 -o eth1 -p tcp -m tcp --dport 11211 --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -i eth0 -o eth1 -p tcp -m tcp --dport 11210 --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -i eth0 -o eth1 -p tcp -m tcp --dport 18091 --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -i eth0 -o eth1 -p tcp -m tcp --dport 18092 --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -i eth0 -o eth1 -p tcp -m tcp --dport 11209 --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -i eth0 -o eth1 -p tcp -m tcp --dport 11214 --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -i eth0 -o eth1 -p tcp -m tcp --dport 11215 --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -i eth0 -o eth1 -p tcp -m tcp --dport 4369 --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -i eth0 -o eth1 -p tcp -m tcp --dport 21100:21299 --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -i eth0 -o eth1 -p tcp -m tcp --dport 40996 --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -i eth0 -o eth1 -p tcp -m tcp --dport 21100 --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -i eth0 -o eth1 -p tcp -m tcp --dport 21101 --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j ACCEPT
-A TCP -p tcp -m tcp --dport 22 -j ACCEPT
# iptables -t nat --line-numbers -L
Chain PREROUTING (policy ACCEPT)
num  target     prot opt source               destination
2    DNAT       tcp  --  anywhere             anywhere             tcp dpt:8091 to:couchbase_priv_ip
3    DNAT       tcp  --  anywhere             anywhere             tcp dpt:8092 to:couchbase_priv_ip
4    DNAT       tcp  --  anywhere             anywhere             tcp dpt:11207 to:couchbase_priv_ip
5    DNAT       tcp  --  anywhere             anywhere             tcp dpt:11211 to:couchbase_priv_ip
6    DNAT       tcp  --  anywhere             anywhere             tcp dpt:11210 to:couchbase_priv_ip
7    DNAT       tcp  --  anywhere             anywhere             tcp dpt:18091 to:couchbase_priv_ip
8    DNAT       tcp  --  anywhere             anywhere             tcp dpt:18092 to:couchbase_priv_ip
9    DNAT       tcp  --  anywhere             anywhere             tcp dpt:11209 to:couchbase_priv_ip
10   DNAT       tcp  --  anywhere             anywhere             tcp dpt:11214 to:couchbase_priv_ip
11   DNAT       tcp  --  anywhere             anywhere             tcp dpt:11215 to:couchbase_priv_ip
12   DNAT       tcp  --  anywhere             anywhere             tcp dpt:epmd to:couchbase_priv_ip
13   DNAT       tcp  --  anywhere             anywhere             tcp dpts:21100:21299 to:couchbase_priv_ip
14   DNAT       tcp  --  anywhere             anywhere             tcp dpts:21100:21299 to:couchbase_priv_ip
15   DNAT       tcp  --  anywhere             anywhere             tcp dpt:40996 to:couchbase_priv_ip
16   DNAT       tcp  --  anywhere             anywhere             tcp dpt:21100 to:couchbase_priv_ip
17   DNAT       tcp  --  anywhere             anywhere             tcp dpt:21101 to:couchbase_priv_ip

Chain INPUT (policy ACCEPT)
num  target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
num  target     prot opt source               destination

Chain POSTROUTING (policy ACCEPT)
num  target     prot opt source               destination
2    SNAT       tcp  --  anywhere             couchbase_priv_ip       tcp dpt:8091 to:nat_gateway_priv_ip
3    SNAT       tcp  --  anywhere             couchbase_priv_ip       tcp dpt:8092 to:nat_gateway_priv_ip
4    SNAT       tcp  --  anywhere             couchbase_priv_ip       tcp dpt:11207 to:nat_gateway_priv_ip
5    SNAT       tcp  --  anywhere             couchbase_priv_ip       tcp dpt:11211 to:nat_gateway_priv_ip
6    SNAT       tcp  --  anywhere             couchbase_priv_ip       tcp dpt:11210 to:nat_gateway_priv_ip
7    SNAT       tcp  --  anywhere             couchbase_priv_ip       tcp dpt:18091 to:nat_gateway_priv_ip
8    SNAT       tcp  --  anywhere             couchbase_priv_ip       tcp dpt:18092 to:nat_gateway_priv_ip
9    SNAT       tcp  --  anywhere             couchbase_priv_ip       tcp dpt:11209 to:nat_gateway_priv_ip
10   SNAT       tcp  --  anywhere             couchbase_priv_ip       tcp dpt:11214 to:nat_gateway_priv_ip
11   SNAT       tcp  --  anywhere             couchbase_priv_ip       tcp dpt:11215 to:nat_gateway_priv_ip
12   SNAT       tcp  --  anywhere             couchbase_priv_ip       tcp dpt:epmd to:nat_gateway_priv_ip
13   SNAT       tcp  --  anywhere             couchbase_priv_ip       tcp dpt:40996 to:nat_gateway_priv_ip
14   SNAT       tcp  --  anywhere             couchbase_priv_ip       tcp dpt:21100 to:nat_gateway_priv_ip
15   SNAT       tcp  --  anywhere             couchbase_priv_ip       tcp dpt:21101 to:nat_gateway_priv_ip

updated with packet capture running couchbase connection app from my local machine to Nat-Gateway which forwards packets to our couchbase private IP on a different network.

No. Time    Source  Destination Protocol    Length  Info
36  1.318547    workstation Nat-Gateway TCP 78  49459  >  11210 [SYN, ECN, CWR] Seq=0 Win=65535 Len=0 MSS=1460 WS=32 TSval=397066275 TSecr=0 SACK_PERM=1
37  1.381378    Nat-Gateway workstation TCP 74  11210  >  49459 [SYN, ACK, ECN] Seq=0 Ack=1 Win=28960 Len=0 MSS=1380 SACK_PERM=1 TSval=1398644492 TSecr=397066275 WS=256
38  1.3815  workstation Nat-Gateway TCP 66  49459  >  11210 [ACK] Seq=1 Ack=1 Win=131328 Len=0 TSval=397066338 TSecr=1398644492
40  1.383823    workstation Nat-Gateway Couchbase   137 Hello Request, Opcode: 0x1f, VBucket: 0x0
44  1.454723    Nat-Gateway workstation TCP 66  11210  >  49459 [ACK] Seq=1 Ack=72 Win=29184 Len=0 TSval=1398644508 TSecr=397066340
45  1.454728    Nat-Gateway workstation Couchbase   90  Hello Response, Opcode: 0x1f
46  1.454806    workstation Nat-Gateway TCP 66  49459  >  11210 [ACK] Seq=72 Ack=25 Win=131296 Len=0 TSval=397066410 TSecr=1398644508
47  1.454917    workstation Nat-Gateway Couchbase   90  List SASL Mechanisms Request, Opcode: 0x20, VBucket: 0x0
49  1.511607    Nat-Gateway workstation Couchbase   104 List SASL Mechanisms Response, Opcode: 0x20
50  1.511689    workstation Nat-Gateway TCP 66  49459  >  11210 [ACK] Seq=96 Ack=63 Win=131264 Len=0 TSval=397066466 TSecr=1398644525
51  1.51173 workstation Nat-Gateway Couchbase   98  SASL Authenticate Request, Opcode: 0x21, VBucket: 0x0
52  1.573012    Nat-Gateway workstation Couchbase   106 SASL Authenticate Response, Opcode: 0x21, Authentication continue
53  1.573124    workstation Nat-Gateway TCP 66  49459  >  11210 [ACK] Seq=128 Ack=103 Win=131200 Len=0 TSval=397066527 TSecr=1398644539
54  1.573185    workstation Nat-Gateway Couchbase   137 SASL Step Request, Opcode: 0x22, VBucket: 0x0
55  1.629576    Nat-Gateway workstation Couchbase   103 SASL Step Response, Opcode: 0x22
56  1.629694    workstation Nat-Gateway TCP 66  49459  >  11210 [ACK] Seq=199 Ack=140 Win=131168 Len=0 TSval=397066583 TSecr=1398644555
57  1.629771    workstation Nat-Gateway Couchbase   90  Get Cluster Config Request, Opcode: 0xb5, VBucket: 0x0
60  1.697122    Nat-Gateway workstation TCP 1434    [TCP segment of a reassembled PDU]
61  1.698056    Nat-Gateway workstation TCP 98  [TCP segment of a reassembled PDU]
62  1.698128    workstation Nat-Gateway TCP 66  49459  >  11210 [ACK] Seq=223 Ack=1540 Win=131040 Len=0 TSval=397066649 TSecr=1398644569
63  1.700758    Nat-Gateway workstation TCP 1434    [TCP segment of a reassembled PDU]
64  1.700764    Nat-Gateway workstation TCP 1434    [TCP segment of a reassembled PDU]
65  1.700766    Nat-Gateway workstation Couchbase   931 Get Cluster Config Response, Opcode: 0xb5
66  1.700843    workstation Nat-Gateway TCP 66  49459  >  11210 [ACK] Seq=223 Ack=4276 Win=128320 Len=0 TSval=397066652 TSecr=1398644569
67  1.700843    workstation Nat-Gateway TCP 66  49459  >  11210 [ACK] Seq=223 Ack=5141 Win=127456 Len=0 TSval=397066652 TSecr=1398644569
68  1.700892    workstation Nat-Gateway TCP 66  [TCP Window Update] 49459  >  11210 [ACK] Seq=223 Ack=5141 Win=131072 Len=0 TSval=397066652 TSecr=1398644569
69  1.70236 workstation Private-Couchbase-IP    TCP 78  49460  >  11210 [SYN, ECN, CWR] Seq=0 Win=65535 Len=0 MSS=1460 WS=32 TSval=397066653 TSecr=0 SACK_PERM=1
82  2.70638 workstation Private-Couchbase-IP    TCP 78  [TCP Retransmission] 49460  >  11210 [SYN] Seq=0 Win=65535 Len=0 MSS=1460 WS=32 TSval=397067653 TSecr=0 SACK_PERM=1
91  3.708379    workstation Private-Couchbase-IP    TCP 78  [TCP Retransmission] 49460  >  11210 [SYN] Seq=0 Win=65535 Len=0 MSS=1460 WS=32 TSval=397068654 TSecr=0 SACK_PERM=1
109 4.709508    workstation Private-Couchbase-IP    TCP 78  [TCP Retransmission] 49460  >  11210 [SYN] Seq=0 Win=65535 Len=0 MSS=1460 WS=32 TSval=397069655 TSecr=0 SACK_PERM=1
110 5.710683    workstation Private-Couchbase-IP    TCP 78  [TCP Retransmission] 49460  >  11210 [SYN] Seq=0 Win=65535 Len=0 MSS=1460 WS=32 TSval=397070656 TSecr=0 SACK_PERM=1
116 6.652282    workstation Nat-Gateway TCP 66  49459  >  11210 [FIN, ACK] Seq=223 Ack=5141 Win=131072 Len=0 TSval=397071597 TSecr=1398644569
117 6.732744    Nat-Gateway workstation TCP 66  11210  >  49459 [FIN, ACK] Seq=5141 Ack=224 Win=29184 Len=0 TSval=1398645831 TSecr=397071597
118 6.732835    workstation Nat-Gateway TCP 66  49459  >  11210 [ACK] Seq=224 Ack=5142 Win=131072 Len=0 TSval=397071677 TSecr=1398645831
  • 1
    I would start by grabbing a pcap from both of the machines in question here and comparing them in wireshark. This will tell you if you're getting RST packets, or timeouts, or a protocol error. For example, you've opened TCP, but what if the client is trying to send data via UDP? Telnet would then work (as it's also TCP), but UDP would not. – Mark Henderson May 01 '17 at 19:17
  • I have been capturing packets and the analysis shows me that forwarding is working since we are receiving packet data from private IP couchbase to my local machine (which is on a different network). However you'll notice that at some point (packet #69) the packet changes its destination to our Private-Couchbase-IP instead of our NAT-Gateway which it should be using. The couchbase packet before that receives the correct info of the couchbase server along with it's private IP, so I think that's how the packet changes its destination. If more data would help anyone, please let me know. – Richard Oswald May 05 '17 at 16:27

0 Answers0