I just tried to insert a PfSense box into my network and I seem to have broken something in the process. Need some outside help to point out any errors I might have missed. The setup was working before inserting the PfSense box.
Network map summary:
Internet <> Edge Router <> PfSense <> Switch <> End Machine
Routes set up as follows:
ER:
- 192.168.2.0/24 -> x.x.x.14 (pfsense WAN ip)
- 192.168.5.0/24 -> x.x.x.14 (pfsense WAN ip)
PfSense:
- 192.168.2.0/24 -> 172.16.1.2 (switch LAN ip)
- 192.168.5.0/24 -> 172.16.1.2 (switch LAN ip)
- Default gateway as x.x.x.1 (gateway of ER, same subnet as pfsense WAN ip)
Switch:
- 192.168.2.0/24 is the default VLAN (interface 2/1) with routing enabled
- 192.168.5.0/24 is a VLAN (interface 2/2) with routing enabled
- Default gateway as 172.16.1.1 (pfsense LAN ip)
Running traceroute to a 192.168.5.x machine from the switch turns up 0.0.0.0 as the first hop. But pinging the same machine from the switch turns up successful.
Pinging from the 192.168.5.x machine is only successful up to 172.16.1.2 (switch LAN ip). Seems like the packet is getting lost between the switch and the pfsense box.
Traceroute works fine from switch to 192.168.2.x machine. Which doesn't really make sense as the only difference is 192.168.2.0/24 is the default VLAN. Why is the switch routing 192.168.5.0/24 through the default gateway when there's a clear route set up as seen in the routing table?
Route table from Switch:
Route Codes: R - RIP Derived, O - OSPF Derived, C - Connected, S - Static
B - BGP Derived, IA - OSPF Inter Area
E1 - OSPF External Type 1, E2 - OSPF External Type 2
N1 - OSPF NSSA External Type 1, N2 - OSPF NSSA External Type 2
S 0.0.0.0/0 [1/0] via 172.16.1.1, 13h:27m:59s, 0/28
C 172.16.1.0/24 [0/1] directly connected, 0/28
C 192.168.2.0/24 [0/1] directly connected, 2/1
C 192.168.5.0/24 [0/1] directly connected, 2/2
Route table from PfSense:
default x.x.x.1 UGS 84864 1500 igb0
x.x.x.0/23 link#1 U 5543 1500 igb0
x.x.x.14 link#1 UHS 0 16384 lo0
127.0.0.1 link#7 UH 52 16384 lo0
172.16.1.0/24 link#3 U 5592 1500 igb2
172.16.1.1 link#3 UHS 11244 16384 lo0
192.168.2.0/24 172.16.1.2 UGS 71953 1500 igb2
192.168.5.0/24 172.16.1.2 UGS 1429 1500 igb2
Finally, I need to point out that I am using OPT1 instead of the default LAN as the LAN interface so I'm not too sure if that's the problem. Are there some hidden rules somewhere that allow passthrough for LAN and not OPT1 that I don't know of?
Thanks!