How do I kill a process that is dead but listening?

49

15

I'm developing an app that listens on port 3000. Apparently there's an instance of it still listening to the port because whenever I start it, it can't create a listener (C#, TcpListener, but that's irrelevant) because the port is already taken.

Now, the app doesn't exist in the Task Manager, so I tried to find its PID and kill it, which led to this interesting result:

C:\Users\username>netstat -o -n -a | findstr 0.0:3000
   TCP    0.0.0.0:3000           0.0.0.0:0              LISTENING       3116

C:\Users\username>taskkill /F /PID 3116
ERROR: The process "3116" not found.

I haven't seen this behaviour before and figured it was interesting enough to see if anyone has a solution.

UPDATE: I started up Process Explorer and did a search for 3000 and found this:

<Non-existent Process>(3000): 5552

I right clicked on it and chose "Close Handle". It's no longer in Process Explorer, but still shows up in netstat and still stops the app from starting the listener.

UPDATE 2: Found TCPView for Windows which show the process as "<non-existent>". Like with CurrPorts, nothing happens when I try to close the connection in this tool.

Srekel

Posted 2010-11-26T15:24:16.540

Reputation: 611

Add the /T option (taskkill /F /T /PID 17888) to also kill children and the error changes in an interesting way: ERROR The process with PID 17888 (child process of PID 17880) could not be terminated. Reason: There is no running instance of the task. In this case 17888 is running but 17880 is not. So I have an orphaned process locked by a dead parent. – Jesse Chisholm – 2015-06-07T15:07:28.857

1With current version of TCPView 3.05 "Close connection" from context menu of <non-exsitent> process successfully closed the connection in my case and freed the port. – Ventzy Kunev – 2019-04-23T12:12:04.283

a bit of curing the disease by killing the patient, but does it stop if you restart the computer? – Xantec – 2010-11-26T15:30:54.473

2In fact, logging out and back in again was enough, but I've now managed to reproduce it so I'd still like to find a better solution... – Srekel – 2010-11-26T16:09:10.947

Check if any of the programs listed here help

– Sathyajith Bhat – 2010-11-26T16:38:47.340

Sathya, thanks but they didn't. More about files than ports I'm afraid. – Srekel – 2010-11-26T17:30:54.417

@Srekel ok.. I've posted an answer, that might help you. – Sathyajith Bhat – 2010-11-27T01:44:46.407

Perhaps you should look into exactly how you implement your listening on the port. Maybe it's a bug in your implementation or in C#'s handling of it. (Memory management perhaps?) – Moshe – 2010-11-29T15:54:57.613

@Moshe: See my answer. – harrymc – 2010-11-29T20:29:55.703

@srekel any updates ? – Sathyajith Bhat – 2010-12-03T02:04:53.187

I have the exact same problem...damn no answer here! – mmmmmmmm – 2013-04-15T10:07:15.557

Sorry, I don't think I ever found a solution to the problem and have since left the company. :/ – Srekel – 2013-04-29T14:32:03.353

Answers

13

To avoid endless waits on the socket, your program should use the setsockopt function with the SO_REUSEADDR and SO_RCVTIMEO parameters :

SO_REUSEADDR : Allows the socket to be bound to an address that is already in use.
SO_RCVTIMEO : Sets the timeout, in milliseconds, for blocking receive calls. 

harrymc

Posted 2010-11-26T15:24:16.540

Reputation: 306 093

1Did this help anyone with this particular issue (socket listening by a dead process)? – rustyx – 2017-07-17T11:08:14.050

12

We had the same problem and used Process Explorer from Microsoft Sysinternals to look up the process ID that no longer existed.

It turns out that the process was referenced by several DrWatson processes. Killing those processes released the port. DrWatson is used to send memory dumps to Microsoft and this was taking several hours because the process that crashed held several tens of GBs of memory at the time.

David

Posted 2010-11-26T15:24:16.540

Reputation: 121

7

I think you ought to give CurrPorts a try

CurrPorts is network monitoring software that displays the list of all currently opened TCP/IP and UDP ports on your local computer. For each port in the list, information about the process that opened the port is also displayed, including the process name, full path of the process, version information of the process (product name, file description, and so on), the time that the process was created, and the user that created it.

In addition, CurrPorts allows you to close unwanted TCP connections, kill the process that opened the ports, and save the TCP/UDP ports information to HTML file , XML file, or to tab-delimited text file.

CurrPorts also automatically mark with pink color suspicious TCP/UDP ports owned by unidentified applications (Applications without version information and icons)

alt text

Sathyajith Bhat

Posted 2010-11-26T15:24:16.540

Reputation: 58 436

4It shows up in the list under the process name "System". I tried right clicking the item to force close the port but nothing happens. – Srekel – 2010-11-29T11:51:52.550

7

The likely problem is that your process has started another (child) process which inherited the socket handle, and it is still running.

There are various ways to prevent this, for instance: ProcessStartInfo.UseShellExecute = true;

Nick Westgate

Posted 2010-11-26T15:24:16.540

Reputation: 1 677

1Thanks, this was a good one. I was calling subprocess.call(..., cwd=..., shell=True) as an app-launcher in a python web server, and it turned out that I had to kill all those child processes to free the socket. The odd thing is that I'm using shell=True. This had me annoyed for ages. – Daniel F – 2014-07-31T15:22:41.537

I don't think that using a shell results in this behaviour. If you want to terminate all child processes when the parent dies, I think the correct way is to create a Job Object with the JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE flag, add each child process with AssignProcessToJobObject, and allow the handle to close naturally when the parent process exits.

– qris – 2017-07-19T08:30:07.663

1

saw similar issue when root cause was some child process created inheriting the ports, parent process crashed, while child process still holding the port. Resolution was to identify the child process still running and stop it. In the example here the non existing process PID was 7336

> wmic process get processid,parentprocessid | findstr/i 7336
7336             23828

To stop this process:

> taskkill /f /pid 23828

And this resolved the issue.

Michael

Posted 2010-11-26T15:24:16.540

Reputation: 11

1

Can you see the process in Process Explorer? If yes than you can kill it from there, but only after you investigate what it actually is (you can see all the dlls loaded into the process)

Jakub Konecki

Posted 2010-11-26T15:24:16.540

Reputation: 360

1

Try throwing a '-b' flag on your netstat command. It will tell you the name of the executable that is using the port. Then find that proc in task manager and kill it there. If that doesn't work post what the executable is that is holding the port open.

Ge3ng

Posted 2010-11-26T15:24:16.540

Reputation: 187

I can't reproduce it at the moment, but I'm pretty sure that since every other attempt (and tool) at trying to find the name of the process failed, netstat wouldn't have been able to do it either. – Srekel – 2010-12-07T17:20:30.880

1

Need to mention the term linger here.

Details can be found at http://msdn.microsoft.com/en-us/library/ms739165.aspx

In brief: There's an option that tells the socket system to keep a socket open even after it has been closed if unsent data is present.

In your C# app you may specify any related options via Socket.SetSocketOption: http://msdn.microsoft.com/en-us/library/1011kecd.aspx

Since all the stuff mentioned is related to sending and clients, but we had similar issues at work, some further searching reveled that one could specify the linger option for listeners as shown in the example over here: http://msdn.microsoft.com/library/system.net.sockets.tcplistener.server.aspx

Some colleagues talked about ports being held by the OS after application termination/closing for about two minutes. Given that, it would be interesting to hear whether the port is still held after a certain amount of time.

Andreas

Posted 2010-11-26T15:24:16.540

Reputation: 1 373

The ports were held open for much longer than that (don't remember how long but maybe up to an hour before I gave up and rebooted). – Srekel – 2010-12-07T17:17:42.927

I'm not sure if the linger option is relevant to this particular case, as it seems it only specifies what's supposed to happen after a call to CloseSocket. Since I'm killing the application, I doubt that function gets called(?). – Srekel – 2010-12-07T17:18:42.053

1

I also encounter this issue. Finally I found my reason. It's caused by main process invoked a child process by popen in c/c++. But the main process crash/shutdown before invoking pclose. Then the port will treat the main process still live and still listen on it.

lorry.lee

Posted 2010-11-26T15:24:16.540

Reputation: 11

1

I found this answer at ServerFault helpful: http://serverfault.com/a/273727/8856

– Harriv – 2012-12-18T12:10:03.663

1

The problem can be caused if the dead process has started one or more child processes. If

BOOL WINAPI CreateProcess(
_In_opt_    LPCTSTR               lpApplicationName,
_Inout_opt_ LPTSTR                lpCommandLine,
_In_opt_    LPSECURITY_ATTRIBUTES lpProcessAttributes,
_In_opt_    LPSECURITY_ATTRIBUTES lpThreadAttributes,
_In_        BOOL                  bInheritHandles,
_In_        DWORD                 dwCreationFlags,
_In_opt_    LPVOID                lpEnvironment,
_In_opt_    LPCTSTR               lpCurrentDirectory,
_In_        LPSTARTUPINFO         lpStartupInfo,
_Out_       LPPROCESS_INFORMATION lpProcessInformation
);

was used to start a child process, it depends on the value of inherit handles, so with

bInheritHandles = false 

Windows will not block the port if the parent process is stopped and the client process is still running.

V15I0N

Posted 2010-11-26T15:24:16.540

Reputation: 143

0

I don't suppose you have a firewall installed (or tried any Windows networking tweeks)?

Some firewalls exhibit this sort of behaviour and can keep ports open.

If you do have one, try disabling it and then launching your program and closing it and seeing what happens.

William Hilsum

Posted 2010-11-26T15:24:16.540

Reputation: 111 572

Not really an option unfortunately, as I don't have control of the firewall on my computer. – Srekel – 2010-12-07T17:16:23.180

@Srekel - what firewall is it? As you are having problems, I would personally think it is that causing the problems. – William Hilsum – 2010-12-07T17:42:56.537

0

Since your process is listed as System, most likely you can kill it on a "System" command prompt. The System account has more privileges than a regular administrator.

You can a get "System" cmd.exe by scheduling a task for cmd.exe (the scheduler runs as System):

at 15:23 /interactive "cmd.exe" 

Change that time for something in the near future. Make sure as well you're on the machine console (if you are on a regular terminal server session, you won't see the new cmd.exe. Make mstsc login to the console). I guess there other ways to do that, but this worked for me in the past.

David Suarez

Posted 2010-11-26T15:24:16.540

Reputation: 101

0

I had the same problem with xdebug, it left port 9000 open.

I managed to close with cmd using "taskkill /pid xxxx".

The pid of the process using the port can be retrieved with "netstat -o".

My configuration was win 7 home premium.

Christoforos

Posted 2010-11-26T15:24:16.540

Reputation: 111

0

I had this same problem. The process was being debugged while it crashed, and there was still a vsjitdebugger.exe process in a suspended state lingering around, which apparently referred to the process being debugged. Killing that vsjitdebugger.exe process solved the problem.

gpvos

Posted 2010-11-26T15:24:16.540

Reputation: 123

0

I had the same problem and fixed it by :

  • finding the PID with netstat -o
  • Kill it with process xp

Interesting to note that netstat can sometimes return a pid but not the corresponding executable name !

eka808

Posted 2010-11-26T15:24:16.540

Reputation: 126

0

A Hybrid use of TCPView and Process Explorer worked for me ;) I first looked in to the process id which was using the port in TCPView. Isolated the process in Process Explorer and killed the process using Process Explorer.

Sathiya Narayanan

Posted 2010-11-26T15:24:16.540

Reputation: 1