shorewall on debian proxy server troubleshooting

Question

The setup is: debian proxy-server -> linsys router -> internal ubuntu server

Problem is: I can access the ubuntu web server (apache2) by entering 192.168.1.128 in a browser on my debian proxy. However, my shorewall forwarding rule does not make it through using either of the following rules:

#Web/DNAT        net             loc:192.168.1.128
#DNAT           net             loc:192.168.1.128:80    tcp     80      -       70.90.XXX.XX

Extra info:

I have the internal ubuntu server accepting all connections right now for debugging purposes (also have shorewall installed)

snipped of output from 'shorewall show log' (on the debian machine):

Feb  1 19:37:15 fw2loc:ACCEPT:IN= OUT=eth1 SRC=192.168.1.137 DST=224.0.0.251 LEN=104     TOS=0x00 PREC=0x00 TTL=255 ID=0 DF PROTO=UDP SPT=5353 DPT=5353 LEN=84 
Feb  1 19:40:31 fw2loc:ACCEPT:IN= OUT=eth1 SRC=192.168.1.137 DST=224.0.0.251 LEN=104 TOS=0x00 PREC=0x00 TTL=255 ID=0 DF PROTO=UDP SPT=5353 DPT=5353 LEN=84

output of 'shorewall show nat' (on the debian machine):

Chain PREROUTING (policy ACCEPT 235 packets, 74689 bytes)
pkts bytes target     prot opt in     out     source               destination         
12   724 net_dnat   all  --  eth0   *       0.0.0.0/0            0.0.0.0/0           

Chain POSTROUTING (policy ACCEPT 8 packets, 670 bytes)
pkts bytes target     prot opt in     out     source               destination         
3   242 eth0_masq  all  --  *      eth0    0.0.0.0/0            0.0.0.0/0           

Chain OUTPUT (policy ACCEPT 7 packets, 611 bytes)
pkts bytes target     prot opt in     out     source               destination         

Chain eth0_masq (1 references)
pkts bytes target     prot opt in     out     source               destination         
1    69 MASQUERADE  all  --  *      *       192.168.1.0/24       0.0.0.0/0           

Chain net_dnat (1 references)
pkts bytes target     prot opt in     out     source               destination         
2   128 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp         dpt:80 to:192.168.1.128 
0     0 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp dpt:443 to:192.168.1.128

EDIT:

I should also mention that after checking the apache logs on the ubuntu machine, none of the requests seem to be making it. When I make a request from the LAN, I get an entry in the access log, but when making a request from outside the lan via the proxy, I get nothing.

Output of 'iptables -vnL' on the proxy:

22  1280 TCPMSS     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp flags:0x06/0x02 TCPMSS clamp to PMTU 
22  1280 eth0_fwd   all  --  eth0   *       0.0.0.0/0            0.0.0.0/0           
22  1280 dynamic    all  --  *      *       0.0.0.0/0            0.0.0.0/0           state INVALID,NEW 
22  1280 smurfs     all  --  *      *       0.0.0.0/0            0.0.0.0/0           state INVALID,NEW 
22  1280 tcpflags   tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           
22  1280 net2loc    all  --  *      eth1    0.0.0.0/0            0.0.0.0/0           
1    60 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           multiport dports 80,443 
22  1280 ACCEPT     tcp  --  *      *       0.0.0.0/0            192.168.1.128       tcp dpt:80

Not really sure what to make of this...

I am starting to wonder if there is a fundamental problem with trying to go through the linksys router and if I should just go debian -> ubuntu with a crossover...

EDIT 2

It looks like packets are being forwarded correctly (I think that is what the output from 'shorewall show nat' did for us)

Output from 'tcpdump -n -i port 80'

public interface:

20:15:40.458118 IP 71.182.212.251.57261 > 70.90.XXX.XXX.80: S 2834662754:2834662754(0) win 65535 <mss 1452,nop,wscale 3,nop,nop,timestamp 25163236 0,sackOK,eol>

private interface:

20:15:45.405347 IP 71.182.212.251.57261 > 192.168.1.128.80: S 2834662754:2834662754(0) win 65535

After a few of the above packets it turns into:

20:16:17.623882 IP 71.182.212.251.57254 > 70.90.XXX.XXX.80: S 1673429824:1673429824(0) win 65535 <mss 1452,sackOK,eol>

and

20:16:17.623917 IP 71.182.212.251.57254 > 192.168.1.128.80: S 1673429824:1673429824(0) win 65535 <mss 1452,sackOK,eol>

it could be usefull to see the filter table (iptables -vnL) instead of the nat table. The logs you provided belong to netbios traffic. You should grep the logs for port 80. On another level, if this a a proxy server, in which way are you proxy-ing the ubuntu server ? Forwarding traffic ? Using Apache, ngnix, etc ? — Torian, Feb 02 '11 at 01:51
right now I am trying to proxy to an apache2 server on either port 80 or 443. I updated with the output of the logs. — Hersheezy, Feb 03 '11 at 00:08

score 1 · Answer 1 · answered Feb 02 '11 at 02:09

1

What host is set as a default gateway on your Ubuntu server? If it is not your proxying Debian server, you should add another rule to /etc/shorewal/masq:

eth1:192.168.1.128         0.0.0.0/0       192.168.1.137       tcp     80

I assume your Debian proxy LAN NIC is eth1 and your Debian server IP address is 192.168.1.137. Basically, the problem here is that DNAT rule does not alter a source IP adress in forwarded packets, so your Ubuntu server tries to answer directly to a request originator, which is outside your LAN. It uses a default gateway to perform the communication, so the answer never gets to the Debian box. This additional rule changes originating IP to the IP of the proxying box itself, so it should help. Anyway, probably it's not what you want because Apache on Ubuntu will record the Debian IP as a client IP to access logs for every request.

answered Feb 02 '11 at 02:09

Alex

7,789
4
36
51

I see. I had tried to set the gateway in my /etc/network/interfaces file on the ubuntu host and that did not seem to help. is this different from what you are proposing? I will get back after I have a chance to try it. Thanks! – Hersheezy Feb 02 '11 at 19:50
Well the changes were made to shorewall and it restarted without complaint. However, forwarding is still unsuccessful with all the logs looking the same. – Hersheezy Feb 03 '11 at 00:07
Probably forwarding is disabled at your Debian host, you should check /etc/shorewall/shorewall.conf and set `IP_FORWARDING=On` there – Alex Feb 03 '11 at 08:48

score 1 · Answer 2 · answered Feb 03 '11 at 02:16

First of all, I assume that the sign # before each rule is not present on the configuration files (it comments that line, making it useless).

Try monitoring you network traffic on the proxy server on both interfaces, public and private. On the public side:

 # tcpdump -n -i <public interface> port 80

On the private side:

 # tcpdump -n -i <private interface> port 80 and host 192.168.1.128

When making requests from the outside world you don see any traffic on the private side, then you should double check:

Kernel allows to forward traffic (should be one):

cat /proc/sys/net/ipv4/ip_forward

If it returns 0:

echo 1 > /proc/sys/net/ipv4/ip_forward

Default policy for the FORWARD chain on the filter table

iptables -nL | grep 'Chain FORWARD'

(according to the info you provided, instead of FORWARD you should see it on net2loc chain)

If default policy is REJECT / DROP, look for a rule on the FORWARD chain that allows to forward traffic onto the HTTP Server:

iptables -vnL FORWARD | grep 192.168.1.128

(again, if nothing meaningfull comes up on FORWARD, check net2loc).

If this doesn't return a line (and policy is REJECT / DROP) then you are missing the rule that allows you to forward traffic ... double check shorewall configuration. Set the loc log to info on the policy configuration file, force a rule:

iptables -A net2loc -i <public iface> -o <private iface> \
--dst 192.168.1.128 -p tcp --dport 80 -j ACCEPT

And see what happens.

The alternative, when making requests from an external location, and you see it on the private side, that would mean that something else is wrong (check the linksys router forwarding policy, routes, etc).

I wouldn't do what @Alex recomends as it is risked. You loose every trace of who is accessing you service therefore you loose statistics. If there is any attack on your site, it is impossible to determine where is it comming from by just looking at your webserver logs, and lots of other reasons. Usually SNAT from public networks to private destinations is highly discouraged.

I had not included the hashes at the beginning of the lines, they were just there when I copied them. It looks like the packets are getting forwarded (see the latest edit). I am to thinking that the linksys router is the problem as I have not configured it for this task at all... I'll look into this. Thanks! — Hersheezy, Feb 04 '11 at 01:35
Oh yeah and Alex mentioned that I would loose the network traffic information, but right now I am just trying to get something to work and after that I'll work on making it correct. — Hersheezy, Feb 04 '11 at 01:53

score 0 · Answer 3 · answered Feb 06 '11 at 19:44

Okay things seem to be behaving now with a slightly different setup:

Now I just go Debian proxy -> Ubuntu server (no more linksys router). I decided not to use the router anyway because that is what we are using for our office wireless. Having what is supposed to be a secure server hooked up to the office's router seems like a not-so-smart move on my part anyway.

Summary of changes:

Gave the private interfaces of both machines static local IP addrs with the following

iface eth0 inet static
     ipaddress 192.168.1.128 #the other one got 192.168.1.137
     netmask 255.255.255.255
     network 192.168.1.0

restarted networking devices

restarted shorewall

Shorewall seems to be doing well and my apache logs are informative (they don't just say that all requests are coming from the debian machine)

Thanks everyone for the help, I used everything that was posted here to arrive at what I hope is an acceptable solution.

Glad to know that. In the future, if you have something to add, just edit your original question or add a comment to it (or to whatever answer did it for you), as answers are intended as not forum posts :) — Torian, Feb 11 '11 at 05:46

shorewall on debian proxy server troubleshooting

3 Answers3