0

We have a setup of ADSL line -> Cisco 837 ADSL router -> Zyxel ZyWall 35 firewall/NAT -> Switch -> LAN.

It has been fine for years, suddenly DNS resolution stopped working from the LAN to public DNS servers. No changes that I know of, so I can't revert anything.

Current behaviour:

  • DNS requests from the LAN using TCP show up in the oubound firewall log, in the Cisco debug log, in the dns-server-firewall, in tcpdump on the DNS server, the answer comes back, it works fine.
  • DNS requests from the LAN using UDP show up in the outbound firewall log, in the Cisco debug log, but does NOT show in the dns-server-firewall, not in tcpdump on the DNS server, times out.
  • DNS requests from the Cisco using UDP show up in the dns-server-firewall and in tcpdump on the DNS server, answer received, works fine.
  • netcat connections to port 53 or a random port by TCP show up in the dns-server-firewall
  • netcat connections to port 53 or a random port by UDP do not show up in the dns-server-firewall

Summary: TCP seems fine throughought. UDP works from the Cisco over the ADSL, and it works from the LAN to the Cisco, but it doesn't seem to cross the Cisco 837 properly. Update: confirmed with netcat that any UDP traffic from the LAN is affected, not just traffic to port 53.

Update: If I change the firewall's external IP to any other IP in the subnet, this starts working. When I put it back, it stops working. I now suspect it's an ISP issue (does that sound plausible?), and am removing the Cisco config.

TessellatingHeckler
  • 5,676
  • 3
  • 25
  • 44

3 Answers3

1
  • Is it just udp or is tcp also affected?
  • Are there any IPv4/IPv6 issues (sometimes routers get confused, thou I've seen that only for really cheap ones)

A way to test that would be (using Google DNS servers):

$ dig @8.8.8.8 example.com +noall +answer +notcp +ignore # udp only
example.com.        50270   IN  A   192.0.43.10
;; Received 45 bytes from 8.8.8.8#53(8.8.8.8) in 29 ms

$ dig -4 example.com +noall +answer +notcp +ignore # udp only/ipv4 only
example.com.        50270   IN  A   192.0.43.10

$ dig -6 example.com +noall +answer +notcp +ignore # udp only/ipv6 only
example.com.        50270   IN  A   192.0.43.10
;; Received 45 bytes from 8.8.8.8#53(8.8.8.8) in 29 ms

$ dig @8.8.8.8 example.com +noall +answer +tcp # tcp only
example.com.        50256   IN  A   192.0.43.10
;; Received 45 bytes from 8.8.8.8#53(8.8.8.8) in 30 ms

$ dig -4 example.com +noall +answer +tcp # tcp only/ipv4 only
example.com.        50256   IN  A   192.0.43.10
;; Received 45 bytes from 8.8.8.8#53(8.8.8.8) in 30 ms

$ dig -6 example.com +noall +answer +tcp # tcp only/ipv6 only
example.com.        50256   IN  A   192.0.43.10
;; Received 45 bytes from 8.8.8.8#53(8.8.8.8) in 30 ms

Also:

  • Do you have a tcpdump
  • Maybe get some cheap VPS/EC2/Rackspace Instance and open port 53 (netcat is your friend). Test with telnet that you can connect -- just to be sure that your pakets are indeed going beyond your firewall.
Martin M.
  • 6,428
  • 2
  • 24
  • 42
  • I've put Dig on the server, and DNS/TCP works but DNS/UDP does not. IPv6 gives "Can't find IPv6 networking", and that's fine as there should not be any IPv6 configuration there. I have a Wireshark capture, but it shows outgoing traffic and no return traffic for DNS/UDP. I could post it but I'd need to work out how to sanitize it for IPs and filter most of the traffic first. Not sure what your last point means - I'm connected through a VPN over the ADSL, and the router can query DNS - packets are definitely going over the connection. – TessellatingHeckler Jun 20 '11 at 21:17
  • How do you know DNS packets are going over the connection? (What are your means of verification) Since DNS does work over TCP but not UDP - who else has access to the devices that could do firewalling? - Note: I am by no means familiar with Zyxel or Cisco products. So these are just questions along the line of "*Are all the cables connected?*" :) – Martin M. Jun 20 '11 at 21:21
  • I have no certain verification. It's a one-wan network and the firewall is also the default gateway, and "show my IP" websites show the right IP for that connection. I can see DNS requests appear in the firewall logs as allowed traffic. But I can't prove DNS requests go over the Cisco and then the ADSL. I would like to know if there's a way to get a network traffic dump from the Cisco, but so far all I can do is use debug to show me DNS lookups done from the Cisco itself, not to show me all traffic. – TessellatingHeckler Jun 20 '11 at 21:45
  • Good stuff. I'd also like to recommend attempting to telnet to the DNS server on port 53 (DNS) from both places and see the response. – uzzi09 Jun 21 '11 at 05:52
  • I've updated my question to include this - netcat tcp connections to port 53 work, netcat udp connections to port 53 do not, but they do hit the Cisco. – TessellatingHeckler Jun 21 '11 at 14:16
0

1 first start by unplugging both and plugging them back in. Check the gateway setting on the Zyxel and make sure it is pointed to the cisco DSL. Then check to make sure the port in administratively up on the cisco router. Then check to make sure that the ACL on the cisco router will allow the Zyxel to connect to it. Ping the cisco router from the zyxel, any response. Has anything changed recently? Firmware, restore, anything?

IT_Fixr
  • 235
  • 3
  • 12
  • Can't unplug them, they're at a remote site. Have rebooted them though. Firewall gateway is OK and port is showing connected on both - all traffic works except DNS. No idea what's changed, if anything. Zyxel can ping the Cisco. – TessellatingHeckler Jun 20 '11 at 18:41
  • did you clear dns cache on both? – IT_Fixr Jun 20 '11 at 19:55
  • is the zyxel your primary DNS server or do you have an active directory server? – IT_Fixr Jun 20 '11 at 19:56
  • The server is an AD DNS server and is the primary, it answers internal addresses, but forwarding out to public DNS stopped working. I think I have cleared DNS cache on both. – TessellatingHeckler Jun 20 '11 at 19:57
0

There's actually a nice article on Google Code that covers troubleshooting DNS issue with them.

You might want to test another DNS, for example your ISP's DNS or OpenDNS.

I have a feeling your Firewall is unable to reach 8.8.8.8.

Alex
  • 3,079
  • 20
  • 28
  • The firewall can ping 8.8.8.8 OK, just can't do DNS lookups against it - or against the ISP's DNS or OpenDNS. The Cisco can DNS lookup against Google's DNS and the ISP's DNS, so I don't think it's specific to Google DNS. Traceroute from the Cisco gets to 8.8.8.8 really quickly, on the Firewall the first three hops are OK, after that they all show \* \* \* and traceroute doesn't finish. – TessellatingHeckler Jun 20 '11 at 18:52