How to kill a process that never dies?

Question

Problem

I have java process which does not die neither with SIGTERM nor SIGKILL.

logstash  2591     1 99 13:22 ?        00:01:46 /usr/bin/java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+DisableExplicitGC -Djava.awt.headless=true -Dfile.encoding=UTF-8 -XX:+HeapDumpOnOutOfMemoryError -Xmx1g -Xms256m -Xss2048k -Djffi.boot.library.path=/usr/share/logstash/vendor/jruby/lib/jni -Xbootclasspath/a:/usr/share/logstash/vendor/jruby/lib/jruby.jar -classpath : -Djruby.home=/usr/share/logstash/vendor/jruby -Djruby.lib=/usr/share/logstash/vendor/jruby/lib -Djruby.script=jruby -Djruby.shell=/bin/sh org.jruby.Main --1.9 /usr/share/logstash/lib/bootstrap/environment.rb logstash/runner.rb --path.settings /etc/logstash

It respawns everytime signal is received.

Sep 15 13:22:17 test init: logstash main process (2546) killed by KILL signal
Sep 15 13:22:17 test init: logstash main process ended, respawning

It sounds strange but even I reboot the server, it still does not die.

Process was executed through init script with below command:

NAME=logstash
LS_USER=logstash
LS_OPTS="--path.settings=/etc/logstash"
LS_PIDFILE=/var/run/$NAME/$NAME.pid
LS_STDERR="/var/log/logstash/logstash.stderr"
DAEMON="/usr/share/logstash/bin/logstash"

runuser -s /bin/sh -c "exec $DAEMON ${LS_OPTS}" ${LS_USER} &>${LS_STDERR} &

Is there any way to force this process to kill other than reinstalling the OS?

Environment

Process :

logstash 5.0.0~alpha5

OS :

Red Hat Enterprise Linux Server release 6.7 (Santiago)

Java version :

openjdk version "1.8.0_101"
OpenJDK Runtime Environment (build 1.8.0_101-b13)
OpenJDK 64-Bit Server VM (build 25.101-b13, mixed mode)

Server is deployed on Microsoft Azure.

Renaming one of the required files for th eprocess before killing it should prevent it from successfully restarting. — Hagen von Eitzen, Sep 15 '16 at 20:19
You got to behead it! That process clearly is an Highlander. — beppe9000, Sep 15 '16 at 22:55
@beppe9000 And while we're in that mood, we might as well execute its parents. — Dmitry Grigoryev, Sep 16 '16 at 07:40
Nuke the entire comp from orbit. It's the only way to be sure. — Rodia, Sep 17 '16 at 04:47
Using `kill` to end a service is never the right way. Have a look at how init systems work. — hashier, Oct 29 '16 at 09:12

score 78 · Accepted Answer · answered Sep 15 '16 at 05:25

init: logstash main process (2546) killed by KILL signal

Actually your process does stop here.

init: logstash main process ended, respawning

A new logstash process is started by init to replace it.

That is also shows which control process is responsible for restarting logstash: init. (On RHEL 6 and CentOS that is Upstart) Your process most likely gets started from either/etc/inittab or a drop-in file in /etc/init/logstash.conf (or similar) and should be controlled with the apropiate tool, initctl and not with kill.

Try initctl list to see if logstash is there.

Then initctl stop logstash will stop it.

Editing or removing the conf file in /etc/init will allow you to disable it persistently.

You might even be able to control the job with the service and chkconfig commands.

+1 for using the right tool for the job. – Mast Sep 16 '16 at 19:52 — Mast, Sep 16 '16 at 19:52

score 0 · Answer 2 · answered Jul 05 '17 at 20:04

0

It is probably because logstash-relay is running ... You should try to stop logstash-relay

after this check if the ps is there then initctl list | sort

I hope this will help you! It fixed the problem for me !

Thanks

VR

answered Jul 05 '17 at 20:04

vrousseu

1
1

The information in the question clearly indicates that it was `init` which was responsible for respawning the process. – kasperd Jul 05 '17 at 20:13

How to kill a process that never dies?

Problem

Environment

2 Answers2