1

I have an apache server that works perfect until I connect to VPN and then all connections to server time out.

Now to my understanding the issue is tun0 becomes the default output interface hence apache gets confused as how to send packets out, so I tried to fix it using control groups by marking packets going out from apache and redirecting them through eth0 as described in this SU answer, but it doesn't work anymore after I upgraded my Ubuntu OS to version 16.04. This is my network diagram:

enter image description here

And here's my network details:

me@mypc:~$ ip route list
0.0.0.0/1 via 10.132.1.5 dev tun0 
default via 192.168.0.1 dev eth0  proto static  metric 100 
10.132.1.1 via 10.132.1.5 dev tun0 
10.132.1.5 dev tun0  proto kernel  scope link  src 10.132.1.6 
123.4.5.6 via 192.168.0.1 dev eth0 
234.5.6.7 via 192.168.0.1 dev eth0 
128.0.0.0/1 via 10.132.1.5 dev tun0 
169.254.0.0/16 dev eth0  scope link  metric 1000 
192.168.0.0/24 dev eth0  proto kernel  scope link  src 192.168.0.6  metric 100 

me@mypc:~$ ifconfig
eth0      Link encap:Ethernet  HWaddr 00:cc:a9:b3:c9:41  
          inet addr:192.168.0.6  Bcast:192.168.0.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:864897 errors:0 dropped:0 overruns:0 frame:0
          TX packets:467142 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:1088053099 (1.0 GB)  TX bytes:220201868 (220.2 MB)
          Interrupt:17 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          ...

tun0      Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  
          inet addr:10.132.1.6  P-t-P:10.132.1.5  Mask:255.255.255.255
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1500  Metric:1
          RX packets:46622 errors:0 dropped:0 overruns:0 frame:0
          TX packets:14950 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100 
          RX bytes:60587170 (60.5 MB)  TX bytes:1396546 (1.3 MB)

I've done further testing and discovered that if I add this routing rule:

sudo route add -host 123.4.5.6 gw 192.168.0.1

I become able to connect to the server from devices connected to my router using the router ip 123.4.5.6 but not from any other ip address.

And after setting up the control groups on apache and trying the following command:

sudo cgexec -g net_cls:novpn wget http://www.whatsmyip.org/

and checking the ip in the downloaded webpage it would be my router ip 123.4.5.6 and not my vpn ip 10.132.1.6.

So I guess the control groups solution works somehow but not with apache and the incoming packets are being received successfully by apache but nothing is going out.

How can I configure apache to use eth0 to output packets instead of tun0?

razzak
  • 33
  • 2
  • 9
  • Please post your apache config file – BobTuckerman Aug 07 '16 at 21:16
  • 1
    In vaugue terms, what is(are) the purpose(s) of the vpn? In line with the 2 answers others have given so far, it seems like you could simplify the setup. For example if you just need web proxy, use 1 or more secure proxies, rather than the PTP VPN; and if you are security-minded, why not then use a better VPN that provides port-forwarding (see @BobTuckerman 's answer below), that is a rather common feature. – bourneN5years Aug 08 '16 at 06:52
  • I'm using my personal pc for this setup, so I need the VPN connection for doing my daily internet stuff securly without exposing my real IP address, plus my VPN ip changes everytime I connect "up to serveral times a day" while my read IP remains the same until I reboot my router "once a month or so", so I want apache to listen to my real ip address. – razzak Aug 08 '16 at 15:55

3 Answers3

3

Managing 2 default routes and getting some traffic to go via one and some to go via the other is a big pain and I would recommend you avoid it if possible. If you don't want to access the internet via the VPN then your setup can be greatly simplified which would solve the issues. Do you really need to access the internet via the VPN? also whats the significance of 10.132.1.5 I don't get why your using it to route to other hosts in 10.132.1.x ? you already have an IP in that subnet 10.132.1.x and should be directly connected.

so IF you don't want to access the internet via the VPN and there is no need to route via 10.32.1.5 you could simplify your routing table down to: 

default via 192.168.0.1 dev eth0  proto static  metric 100 
10.132.1.0/24 dev tun0  proto kernel  scope link  src 10.132.1.6 
169.254.0.0/16 dev eth0  scope link  metric 1000 
192.168.0.0/24 dev eth0  proto kernel  scope link  src 192.168.0.6     metric 100

Which will also solve the apache issue. Why only those routes? firstly:

10.132.1.0/24 dev tun0  proto kernel  scope link  src 10.132.1.6

should be the only route you need to get to all the hosts on the other side of the vpn. As this route matches 10.232.1.0 - 10.232.1.254 if there are other 10.x.x.x address on the vpn network you could broaden the route to 10.0.0.0/8 If you don't understand what the /8 /24 bits mean I highly suggest you read What is the "slash" after the IP? or google for "CIDR notation". will make the /32 /1 bit make more sense.

these old routes put together define your default route via the VPN

0.0.0.0/1 via 10.132.1.5 dev tun0 
128.0.0.0/1 via 10.132.1.5 dev tun0

Breaking it up into 2 routes (one for the first half of the internet and the 2nd covering the rest) means that they have higher priority because they more specific than your default route:

default via 192.168.0.1 dev eth0  proto static  metric 100

The problem then is that you now need to explicitly add routes that are even more specific, to force traffic via your local router(192.16.8.0.1). 

123.4.5.6 via 192.168.0.1 dev eth0 
234.5.6.7 via 192.168.0.1 dev eth0

Without the default route via the VPN those routes aren't needed. If you do need to route traffic via 10.232.1.5 and need to keep to sets of default routes I'd recommend using route rules instead of overriding the default route. Route rules are more flexible but generally how you do it is to match source IP to default gateway, so all traffic sourced from your VPN IP(10.1.232.6) goes vis the VPN default route and all traffic sourced from your local IP(192.168.0.6) uses your ISP as the default route. see https://unix.stackexchange.com/questions/22770/two-interfaces-two-addresses-two-gateways for a guide on how to do that.

Community
  • 1
Nath
  • 1,282
  • 9
  • 10
  • Unfortunately I need internet connection through VPN and still be able to access apache server from my real public ip, `"whats the significance of 10.132.1.5"` my VPN client does this automatically every time i connect, I thought it's kind of weird but it working fine so i didn't bother. – razzak Aug 07 '16 at 22:45
1

So you need both default gateways; then the way to do this then is with route rules:

1) Add a secondary IP to eth0 - ie 192.168.1.7 and restart apache (sounds like your config is listen 0.0.0.0:80 so you'll just need to restart apache to get it to listen on the new IP.

2) Change the Nat rules on your router to foward traffic to this IP:

3) Create a new default route in a secondary route table lets name the table 'apache':

echo "1 apache" >> /etc/iproute2/rt_tables

4) Add a default route to this table via your local router.

ip route add default via 192.168.0.1 dev eth0 table apache

5) Finally you need a rule to define which traffic should use the apache route table.

ip rule add from 192.168.0.7 table apache

192.168.0.7 is a secondary IP and Apache is the only process using it this rule should only match the traffic leaving apache in response to web requests. This will ensure that only that specific traffic will be use the new route table and it won't mess with your VPN traffic or its current routing behaviour.

please note that the ip commands won't persist after reboot. To make them persistent add them to your interface up scripts to make run every time your laptop reboots.

P.S Leaving my old answer and did this as a new answer as this is very different solution.

Nath
  • 1,282
  • 9
  • 10
  • It didn't work, I think maybe it's because there's no traffic coming through **192.168.0.7**! or could be that apache doesn't output trafic to different ip address from the one it receives incoming requests on! – razzak Aug 09 '16 at 19:04
  • That was step 2 you need to change the nat rules to point incoming requests to Apache on 192.168.0.7 – Nath Aug 09 '16 at 19:07
  • Sir you are a genius, it worked. I added static primary and secondary ips and added the routing and ip rules and lo and behold it worked like magic. I would have upvoted this answer if I had enough reputation points. Thanks – razzak Aug 09 '16 at 19:29
0

Just tell apache to listen on multiple IP addresses.

I think you'll want to have these lines in your configuration.

Listen 192.168.0.6:80
Listen 10.132.1.6:80

This tells apache to 'bind' to port 80 on each of those IP addresses. If you're using virtual hosts, you'll also have to add these lines to their configurations as well.

Sidenote

If you're not using one, I recommend using a static IP on eth0 as well as on the VPN server.

If you're not able to configure a static IP on the VPN, there are workarounds you could do. One could be to generate the relevant Listen entries after you connect to the VPN. Or, maybe there's a clean solution that uses hostnames (if you have a DNS server).

Official Documentation:

BobTuckerman
  • 448
  • 3
  • 8
  • My VPN provider doesn't support port forwarding and I don't find it secure, so I need all server traffic to go through `eth0`. – razzak Aug 07 '16 at 22:47
  • 1
    I'm confused, why do you need VPN access if you want all web traffic to go through eth0? Could you just configure Apache to only bind to the IP address on eth0? – BobTuckerman Aug 08 '16 at 01:31
  • 1
    Exactly, wouldn't the clients browsing the web via the 10.x.x.x network, be just as well off accessing the web-server from outside, as well. If not (either way then,) you're opening up lots of other security issues from the browser-side, etc., if you're trying to keep the browsing traffic secure, and not just proxied. – bourneN5years Aug 08 '16 at 06:49
  • See the comment I posted on my question. Basically the VPN is for my personal use, and I want apache to listen to an IP address that doesn't change very often. – razzak Aug 08 '16 at 15:59
  • Given what you're saying, I think it's likely your apache.conf includes the default listen definition which listens on all interfaces. Replace that with a `Listen 192.168.0.6:80` definition, and then it won't listen on the VPN. Will that work? – BobTuckerman Aug 08 '16 at 16:02
  • @BobTuckerman I tried `192.168.0.6:80` it doesn't work, apache tries to output packets through `tun0` and fails. – razzak Aug 08 '16 at 16:06
  • Appears to me that this rule `0.0.0.0/1 via 10.132.1.5 dev tun0 ` is forcing **all** traffic to go through the VPN. For sake of experiement, try removing that rule and see if it works. You'll probably need to disable the option in your VPN that forces all traffic to go over the VPN. http://serverfault.com/a/667294/265400 – BobTuckerman Aug 08 '16 at 19:34