Mystery firewall? Checklist for diagnosing blocked incoming connections

Question

I'm running a web server for development on a Lenovo Thinkpad T420s, and I cannot connect to it from three separate devices (desktop, iPad, and phone, respectively).

The laptop and other devices are connected to a router running DD-WRT (desktop wired, everything else wireless). I've already determined the following:

Checklist for anyone having trouble connecting to a local server

• The server is running and listening for connections (tested via http to localhost).
• netstat -a -n -b shows that something is listening on the proper port.
• Windows Firewall is completely disabled on public, private, and domain networks.
• The host's IP is static.
• Other servers on the host do not work either (I tried one on port 8000, 1515, and also testing ping).
• On the other (desktop) machine, I was able to host a server and respond to pings from my phone. Interestingly, I was also able to access the desktop server from my laptop and ipad, but only a second after "waking" up the server with a request from my phone first. (This didn't work for the server on the laptop.)

In addition to the usual fixes, I've tried a number of other things:

Checklist for anyone who tried all the above things

• Making sure AP Isolation is disabled (http://www.dd-wrt.com/wiki/index.php/Advanced_wireless_settings#AP_Isolation)
• Repairing Winsock2 (http://answers.microsoft.com/en-us/windows/forum/windows_7-system/resettingrepairingreinstalling-winsock2-testing/aa044f6b-00e2-406b-8edb-cba5c4799cf2)
• VIP Access Connections is not installed, as far as I can tell (see here for someone else finding this blocks incoming connections)
• Sniffing both the incoming connections using Wireshark (this indicated that the webhost heard no incoming request when I made one from my phone.)
• Sniffing the outgoing connections by trying to access the host from my desktop and sniffing from the desktop showed there was no HTTP outgoing request at all, just a series of TCPs:

My question is twofold:

1. Is there something specific on a Thinkpad T420s that could be blocking incoming connections secretly?
2. What's the best way to systematically figure out why incoming connections aren't being responded to? (See above for answer)

Any help on this would be very, very much appreciated. I'm running out of ideas!

Update: It works now, but no idea why. Hopefully this checklist will be useful to anyone else trying to access a web server locally.

Answer 1

1. There are a number of software firewalls and DD-WRT router functions that could block incoming connections. AP isolation and disable firewall would have been my first two recommendations. My next step would be to further examine the router setup. My gut says that if you had a wired connection for the laptop and desktop it would work between the two.

2. A great tool for systematically troubleshooting connection issues would be a packet sniffer like wireshark. At this point you're guessing at what is happening on the network. Then toggling various settings hoping to confirm a guess. Why guess when you can look and see? Packet capture will give you a live readout of what is happening. You may find that the TCP SYN packet never reaches the server, or it does reach the server, but no response is issued. Alternatively it could be something else entirely like a name/ip/arp resolution issue. A sniffer won't fix anything on its own, but it should give you a much better idea about what is happening now.

Trying to interpret a packet capture can be somewhat intimidating at first, try to focus on the basics and ignore the more complex parts.

Wireshark Protocol Analyzer

Edit: Just wanted to follow up now that you've added the packet capture screenshot. It's significantly more clear what is happening. One of the challenges with using a tool like wireshark is understanding what the protocols should look like. Since HTTP rides on top of a normal TCP connection, what you see here is a series of failures to setup a TCP connection on port 1515. The reason you don't see HTTP is that the TCP connection failed before the conversation between client and server could get that far. I'll walk you through it by packet number through the first two attempts:

packet(204) .150 -> .151 [SYN] "Hi I'd like to connect to port 1515"

packet(207) .151 -> .150 [RST, ACK] "Not listening. There's no port 1515 here. Buzz off ok?"

packet(208) .150 -> .151 [SYN] "Hi I'd like to connect to port 1515"

packet(209) .151 -> .150 [RST, ACK] "Not listening. There's no port 1515 here. Buzz off ok?"

etc...

A RST flag in a TCP conversation is a force close. It's more rude than a normal FIN flag that gracefully ends a conversation. This generally means that there is either a firewall blocking port 1515 or nothing is listening on port 1515. I see [RST, ACK] more commonly from servers that don't have a service listening on that port, whereas firewalls tend to just RST or black hole (no response) the traffic.

Now that it's working take another capture just so you can see what a successful connection looks like. It should be something along the lines of:

.150 -> .151 [SYN] "Hi I'd like to connect to port 1515"

.151 -> .150 [SYN, ACK] "Acknowledged, here's my buffer size"

.150 -> .151 [ACK] "Works for me"

.150 -> .151 [ACK][HTTP/GET] "Requesting html via HTTP"

.151 -> .150 [ACK][HTTP/1.1 200 OK] "Here is the html you requested"

series of HTTP conversations riding on top of TCP ACKs

.150 -> .151 [FIN] "Thanks server I'm all done now close connection"

.151 -> .150 [ACK, FIN] "I agree we should close connection"