Services remain in failed state after stopped with systemctl

Question

we have a simple systemd script to start a MineCraft server in a service fashion. The SO is CentOS 7. Here the script:

[Unit]
Description=Minecraft Server
After=syslog.target network.target

[Service]
Type=simple
WorkingDirectory=/root/Minecraft
ExecStart=/bin/java -Xmx1024M -Xms1024M -jar minecraft_server.jar nogui
Restart=on-failure

[Install]
WantedBy=multi-user.target

Starting the service works fine but when stopping , the service remains in a failed state. See:

systemctl status minecraftd.service
minecraftd.service - Minecraft Server
   Loaded: loaded (/usr/lib/systemd/system/minecraftd.service; disabled)
   Active: active (running) since Mon 2015-06-01 16:00:12 UTC; 18s ago
 Main PID: 20975 (java)
   CGroup: /system.slice/minecraftd.service
           └─20975 /bin/java -Xmx1024M -Xms1024M -jar minecraft_server.jar nogui
systemctl stop minecraftd.service
systemctl status minecraftd.service
minecraftd.service - Minecraft Server
   Loaded: loaded (/usr/lib/systemd/system/minecraftd.service; disabled)
   Active: failed (Result: exit-code) since Mon 2015-06-01 16:01:37 UTC; 3s ago
  Process: 20975 ExecStart=/bin/java -Xmx1024M -Xms1024M -jar minecraft_server.jar nogui (code=exited, status=143)
 Main PID: 20975 (code=exited, status=143)

Any idea?

Thanks

score 45 · Accepted Answer · edited Oct 11 '20 at 02:17

45

Exit code 143 means that the program received a SIGTERM signal to instruct it to exit. The JVM catches the signal, does a clean shutdown, i.e. it runs all registered shutdown hooks, but still exits with an exit code of 143. That's just how Java works.

You should be able to suppress this by adding the exit code into the unit file as a "success" exit status:

[Service]
SuccessExitStatus=143

edited Oct 11 '20 at 02:17

Evgeniy Berezovsky

851
1
8
26

answered Jun 01 '15 at 16:55

Michael Hampton

237,123
42
477
940

It works. Now the service is in the inactive status, as expected. – kalise Jun 01 '15 at 17:26
4

What would the "proper" way to handle the signal with a Java application? The closest I can find would be shutdown hooks, but nowhere in the documentation does it mention that shutdown hooks alter the exit code of the application. – SPoage Apr 22 '16 at 19:04
@SPoage https://stackoverflow.com/q/2975248/1068283 But, it seems, Java always exits with code 143 in this case, even if a shutdown hook is present. – Michael Hampton Jul 04 '18 at 14:20
Relevant detail: [systemd has code](https://github.com/systemd/systemd/commit/9a57c62944258c750d80bca4fe56de4dbab46d67#diff-c3233f77c007eae6a31dc47bc430a530a3d8ebc1810c52f8168112c9d5eae92cR154-R160) that specifically checks whether the child was killed with `SIGTERM`, and counts it as a clean shutdown. However, the JVM _catching_ the signal and then still exiting with code 143 sets `code == CLD_EXITED` instead of `code == CLD_KILLED`, so that check doesn't trigger. – nh2 Jan 26 '22 at 20:08

Manu · Answer 2 · 2019-01-24T21:10:28.580

17

To complement Michael's answer, the exit code 143 is normal here, it is the way that the java VM received a SIGTERM signal, send by systemd to stop the process. The SIGTERM signal has a numeric value of 15 ( see man signal).

Now according to the Posix specification, "The exit status of a command that terminated because it received a signal shall be reported as greater than 128". (http://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#tag_02_08_02)

Here the Java VM adds 128 + 15 and you get this exit code of 143.

This non zero exit code here make senses, as this allows to see that your java program exited because of an external signal, and you get a chance to find out which signal.

edited Jan 24 '19 at 21:10

answered Dec 30 '17 at 10:34

Manu

320
2
7

The referenced POSIX specification text appears to be specifying how a _shell_ should behave and says "The shell is a command language interpreter." A Java VM doesn't seem to be something covered by that specification. How a shell interprets a Java VM (or any other program) terminating due to SIGTERM - that it should set the exit code to 143 - certainly is covered by the specification, but I'm fairly sure there isn't a shell involved here. – doshea Jul 24 '18 at 08:28
You're right in that this POSIX specification is directed at the UNIX shell, but here it looks like the JVM authors decided to implement their return code the same way. – Manu Jul 24 '18 at 08:47

Services remain in failed state after stopped with systemctl

2 Answers2