Assert if a port is open to host, LAN, or the Internet

Question

I can check if a port is open using the following command, though this does not tell me if this port is reachable on the running machine only (host), from the LAN only, or open to the Internet.

netstat -aon | findstr 8877
TCP    [::]:8877    [::]:0    LISTENING    160352

Particularly, for a given port, I am interested to know:

• If the port is open and listening (the above command is sufficient for this purpose);
• Is the port accessible by other computers on the LAN? if not, what is blocking it? (e.g., is it blocked by the firewall of the host?)
• Is the port accessible by other computers on the Internet? if not, what is blocking it? (e.g., is it blocked by the firewall of the router?)

Background

Generally, to find answers to these questions I spend hours checking different firewalls and access policies. With different types of third-party firewalls involved and different types of routers, this becomes a time-consuming task. Ideally, a built-in Linux, Mac, or Windows command, or a third-party tool that can go through the network and identify where a connection to a port was blocked would be very much useful.

Answer 1

From the host itself you can not reliably check if an external firewall or security rule blocks access.

Additionally an external check can’t easily distinguish between the server itself or a network firewall/ACL blocking access.

Generally you need to do some more comprehensive debugging.

But often a systematic approach somewhat avoids "spend hours checking different firewalls and access policies" though.

When you're a bit methodical you should be able to quickly isolate a more exact cause. Admittedly when there are several layers of security and you don't have access to the full security infrastructure it can still be time consuming to resolve though.

First

When your own work station is completely locked down it really helps to have access to an internal test system/server where you can run for example a packet capture and other debugging tools, such as a port scanner.

To debug internet access to services, a similar external test system is essential. Your own corporate network is often, if not an outright trusted source, at least not considered hostile either. I often use my phone (as a hot-spot for a second laptop) and a cheap VPS for that.

Can you replicate the issue or is the connection problem very client specific?

Try to avoid debugging PEBMAC and other issues at the wrong level. Be aware that even otherwise capable users can make error reports that are incomplete and/or reports that feature jargon that is incorrectly applied and steers you in the wrong direction.

You should have a rough idea of your infrastructure and network design and where the problematic client and server/service feature. (For example do you need to consider IPv4 NAT constructs, layered corporate firewalls, DMZ's, VPN interconnects, cloud providers and security groups, the double NAT used to integrate acquired company XYZ that still uses an overlapping RFC 1918 network range, split horizon DNS, load balancers and reverse proxy servers, normal proxy servers etc.)

Start a packet capture on the server

A packet capture on the server will see all incoming packets. Incoming packets are still received when no service is listening on the port. Regardless of the fact that an host based firewall may be active, a packet capture shows packets before they're processed and possibly rejected by the host-based firewall.
_{Linux reference here or Windows here}

• The suitable packet capture tools/commands are OS dependent and vary.
  On a Linux for example start the low level tcpdump packet capture (use additional flags to select the correct interface/ip-address and to reduce the output)

Then attempt to connect to your server from your remote test system or run a port scan.

For example to test the web server make a web request with for example curl -v http://name-or-IP-of-server

When there is no external ACL/firewall that blocks the request you can can expect to see something like:

    tcpdump -n port 80  
    tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
    listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes

    15:33:49.255233 IP 10.131.160.119.56176 > 10.171.2.119.http: Flags [S], seq 805155927, win 65535, options [mss 1460,nop,wscale 6,nop,nop,TS val 251872860 ecr 0,sackOK,eol], length 0
    15:33:49.255390 IP 10.171.2.119.http > 10.131.160.119.56176: Flags [S.], seq 2225282760, ack 805155928, win 28960, options [mss 1460,sackOK,TS val 448523422 ecr 251872860,nop,wscale 7], length 0
    15:33:49.267951 IP 10.131.160.119.56176 > 10.171.2.119.http: Flags [.], ack 1, win 2058, options [nop,nop,TS val 251872873 ecr 448523422], length 0
    15:33:49.269340 IP 10.131.160.119.56176 > 10.171.2.119.http: Flags [P.], seq 1:129, ack 1, win 2058, options [nop,nop,TS val 251872873 ecr 448523422], length 128: HTTP: GET / HTTP/1.1

and the preliminary conclusion is that access is not blocked somewhere between the client and the server and further investigation of the server is needed.

When you don't see any packets arrive: the problem is elsewhere in your network.

When packets are received: check the server health

You can't make a connection from any client when there is nothing listening on the server.

Normally I'd expect monitoring to send alerts for services that are down, but never rely on assumptions alone. Therefore verify that:

1. The service is up and running.

   • Check if the service is running and enabled to start at system boot.

   • The suitable check commands are OS dependent and vary. On modern Linux distributions you could for example use systemctl or the more universal ps.
     To see if my web server is running I would for example use sudo systemctl status httpd and check for output containing things such as "httpd.service; enabled" and "Active: active (running) since ..."

• In some there case there may be dependancies on other services that need to be running. You may need to repeat the same checks for those. For example an RPC service like classical NFS can only be found when the portmap daemon is up and running.

2. The service listens on the correct interface/IP-address i.e. the one(s) that clients will be using.

   • Check if the service is NOT bound to only localhost (IPv4 127.0.0.1 and/or IPv6 [::1] or the name of a loopback interface).

   • Check if the service if bound to the correct IPv4 and/or IPv6 address. That can be any or the specific interfaces/IP-address(es) that client is expected to use.

   • The suitable check commands are OS dependent and vary.
     Many OS's support a form of netstat though. And on modern Linux distributions you could use ss. For example sudo netstat -tnlp & sudo ss -tnlp.

      sudo netstat -tnlp 
     
      Active Internet connections (only servers)
      Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
      tcp6       0      0 :::80                   :::*                    LISTEN      896/httpd
     

3. The service is responsive on the interface/IP-address the client(s) will be using.
   You should to avoid getting caught debugging a "the lights are on but nobody's home" scenario.

   • The suitable check commands are OS dependent and vary according to the service you're debugging. Do not to only debug localhost but especially the external IP-addresses/hostnames of the server.
     To see if my web server is running I would for example use curl -vv http://hostname or curl -kvv https://localhost and but venerable telnet can be useful in many cases, as is openssl s_client -connect ip:port for TLS services.

When packets are received: Check the host-based firewall on the server

When you're logged on anyway: it is quick and easy to check if there is a host based firewall active.

The exact method again depends on the OS, for Linux iptables -L -v -n --line-numbers will display rules if a host based is active and configured.

When the firewall is active, configured and the rules are trivial, you may be able to spot problematic rules immediately and resolve them. Otherwise you may need some more time and effort there.

I described my approach on Linux here: Debugging iptables and common firewall pitfalls?

When no packets are received: The traffic is blocked elsewhere.

The exact error message on seen on your test system can be quite essential:

curl -v http://host.example.com

*   Trying 10.171.2.119:80...
* Failed to connect to host.example.com port 80 after 17485 ms: Operation timed out
* Closing connection 0

versus

curl -v http://host.example.com

*   Trying 10.171.2.119:80...
* Failed to connect to host.example.com port 80 after 17 ms: Connection refused
* Closing connection 0

The first is the typical result of a firewall that (silently) drops traffic that is not allowed. That is a firewall configuration that seems preferred by security advocates. That is a PITA to debug.

The second is the result of firewall blocks and the rejects connections with an appropriate ICMP protocol error message. That is the kind of firewall configuration that helps me as an administrator.

(Note: we already excluded the other common cause of a "connection refused" error, the one that a server generates when the port is open, but no listener is configured and active. That normally triggers a TCP reset RST response.)

Capture the "connection refused" ICMP message

You will need to run a packet capture on the test system along side the attempts you make there to connect to the server.

tcpdump -n icmp

IP 10.171.2.1 > 192.168.178.22: ICMP host 10.171.2.119 unreachable - admin prohibited, length 72

The source 10.171.2.1 sent the connection refused message, in this a very clear "admin prohibited" and that is the firewall that needs investigating first.

Capture the "connection refused" TCP reset message

You will need to run a packet capture on the test system along side the attempts you make there to connect to the server.

tcpdump -n -v 'tcp[tcpflags] & (tcp-rst) != 0'

IP (tos 0x0, ttl 57, id 9273, offset 0, flags [DF], proto TCP (6), length 40)
    10.171.2.1.80 > 192.168.178.22.57231: Flags [R.], cksum 0xe3a0 (correct), seq 0, ack 4064031175, win 0, length 0

When all that failed

You will need to run packet captures at different places in your infrastructure until you spot where packets are getting stopped.

Answer 2

I would use something like wireshark or nmap, as far as internet goes you need to use portforward port 8877 in your router to your server IP address

Answer 3

Use tools like nmap, netstat, and tracert.

Netstat shares the state of open ports local to the machine it is run on. Tracert is traceroute that traverses the network, sharing if an IP and port are accessible. Nmap is a swiss-army tool that allows you to test the state of a port.