Unix/Java process stops running when moved to background

1

We have a Java process that was created using Appassembler. It runs fine as long as it is started and run in the foreground:

[ec2-user@ip-xxx ourapp-0.0.1-SNAPSHOT]$ bin/ourapp
Starting in APP_HOME=/home/ec2-user/app_home
Press Q to quit

We can then access and test the application successfully. However, if we start it in the background, not only does it stop running, but we can't seem to revive it without bringing it to the foreground:

[ec2-user@ip-xxx ourapp-0.0.1-SNAPSHOT]$ bin/ourapp &
[1] 11661
Starting in APP_HOME=/home/ec2-user/app_home
Press Q to quit
                                                     ## Not accessible!
[ec2-user@ip-xxx ourapp-0.0.1-SNAPSHOT]$  jobs
[1]+  Stopped                 bin/ourapp
[ec2-user@ip-xxx ourapp-0.0.1-SNAPSHOT]$  bg %1
[1]+ bin/ourapp &                                    ## Still not accessible!
[ec2-user@ip-xxx ourapp-0.0.1-SNAPSHOT]$  jobs
[1]+  Stopped                 bin/ourapp
[ec2-user@ip-xxx ourapp-0.0.1-SNAPSHOT]$  fg %1
bin/ourapp                                           ## Now, it's accessible.

Am I starting it improperly? Is there a way to keep the job running, even though it's in the background? I need to start this as a daemon process with nohup and then logout, but I can't seem to keep it running successfully unless it remains the foreground process, which is not feasible.

Craig Otis

Posted 2015-01-27T16:31:17.777

Reputation: 776

Press Q to quit... sounds like you're trying to background a keyboard interactive program which of course isn't going to give you desired results when you push it to the background. – Foosh – 2015-01-27T16:36:45.357

On the main thread, it does stall waiting for Q, but that's the only thing it waits for. (The rest of the program is not keyboard-driven, that's just an easy way to kill it if you're actively running the process.) Before querying stdin, it starts up a few background threads that should continue running, no? – Craig Otis – 2015-01-27T16:39:56.117

Yes they "should" keep running, which leads me to believe they are waiting for the main thread for something. Why bother having the "Q" input if you're planning on running this as a daemon? Or better, define your app to accept a "daemon" flag when it's initialized so that it doesn't wait on the keyboard at all but you can still have it for testing purposes. And logging is your friend, especially when trying to find thread synch issues. – Foosh – 2015-01-27T17:03:42.240

Answers

2

It seems that background jobs waiting for input are stopped in most environments.

From the Wikipedia page on Unix Job Control:

A background process that attempts to read from or write to its controlling terminal is sent a SIGTTIN (for input) or SIGTTOU (for output) signal. These signals stop the process by default, but they may also be handled in other ways.

And from a Rutgers page on intermediate use of Unix:

A job being run in the background will stop if it needs input. Input can not be given to a background job so make sure that all input necessary is available to it.

As a solution we simply updated our Java process to accept an optional argument that will cause the main thread to sleep indefinitely, rather than wait for input. We have a shutdown hook that will handle the SIGTERM/SIGINT signals appropriately:

        if (args.length >= 0 && StringUtils.equals(args[0], "daemon")) {
            while (true) {
                try {
                    Thread.sleep(1000);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
        }

Craig Otis

Posted 2015-01-27T16:31:17.777

Reputation: 776