Ubuntu 18.04.2 build of Tomcat 9 not writing to catalina.out, but in systemd status

Question

So I'm having a problem with Tomcat 9 in Ubuntu 18.04.2. Tomcat 8 worked fine, it logged Java stack traces to /var/log/catalina.out fine. Tomcat 9 is unfortunately only logging GET and POST requests with the default Ubuntu configuration via apt.

I looked this up and there appears to be several threads how to fix, the most recent being this one: Where is catalina.out in tomcat 9? -- but the problem is my catalina.sh looks very different than the file shown in this answer.

The parts of my catalina.sh that are similar (but much different) are as below. For some reason in the apt install, it looks like it's instead of looking to launch catalina instead trying to find it's PID?:



if [ -z "$CATALINA_OUT" ] ; then
  CATALINA_OUT="$CATALINA_BASE"/logs/catalina.out
fi

[...]

elif [ "$1" = "start" ] ; then
  if [ ! -z "$CATALINA_PID" ]; then
    if [ -f "$CATALINA_PID" ]; then
      if [ -s "$CATALINA_PID" ]; then
        echo "Existing PID file found during start."
        if [ -r "$CATALINA_PID" ]; then
          PID=`cat "$CATALINA_PID"`
          ps -p $PID >/dev/null 2>&1
          if [ $? -eq 0 ] ; then
            echo "Tomcat appears to still be running with PID $PID. Start aborted."
            echo "If the following process is not a Tomcat process, remove the PID file and try again:"
            ps -f -p $PID
            exit 1
          else
            echo "Removing/clearing stale PID file."
            rm -f "$CATALINA_PID" >/dev/null 2>&1
            if [ $? != 0 ]; then
              if [ -w "$CATALINA_PID" ]; then
                cat /dev/null > "$CATALINA_PID"
              else
                echo "Unable to remove or clear stale PID file. Start aborted."
                exit 1
              fi
            fi
          fi
        else
          echo "Unable to read PID file. Start aborted."
          exit 1
        fi
      else
        rm -f "$CATALINA_PID" >/dev/null 2>&1
        if [ $? != 0 ]; then
          if [ ! -w "$CATALINA_PID" ]; then
            echo "Unable to remove or write to empty PID file. Start aborted."
            exit 1
          fi
        fi
      fi
    fi
  fi

[...]

However, I do see a reference to the Boostrap process under "security". Is this where I should add the line?

Note - catalina.out.2019-05-07.log files ARE being written, but again, only contain REQUESTS not ERRORS.

if [ "$1" = "-security" ] ; then
    if [ $have_tty -eq 1 ]; then
      echo "Using Security Manager"
    fi
    shift
    eval \{ $_NOHUP "\"$_RUNJAVA\"" "\"$LOGGING_CONFIG\"" $LOGGING_MANAGER $JAVA_OPTS $CATALINA_OPTS \
      -D$ENDORSED_PROP="\"$JAVA_ENDORSED_DIRS\"" \
      -classpath "\"$CLASSPATH\"" \
      -Djava.security.manager \
      -Djava.security.policy=="\"$CATALINA_BASE/policy/catalina.policy\"" \
      -Dcatalina.base="\"$CATALINA_BASE\"" \
      -Dcatalina.home="\"$CATALINA_HOME\"" \
      -Djava.io.tmpdir="\"$CATALINA_TMPDIR\"" \
      org.apache.catalina.startup.Bootstrap "$@" start \         
      "$CATALINA_OUT" 2>&1 "&"                                <-- and here?
      2\>\&1 \&\& echo \$! \>\"$catalina_pid_file\" \; \} $catalina_out_command "&"

  else
    eval \{ $_NOHUP "\"$_RUNJAVA\"" "\"$LOGGING_CONFIG\"" $LOGGING_MANAGER $JAVA_OPTS $CATALINA_OPTS \
      -D$ENDORSED_PROP="\"$JAVA_ENDORSED_DIRS\"" \
      -classpath "\"$CLASSPATH\"" \
      -Dcatalina.base="\"$CATALINA_BASE\"" \
      -Dcatalina.home="\"$CATALINA_HOME\"" \
      -Djava.io.tmpdir="\"$CATALINA_TMPDIR\"" \
      org.apache.catalina.startup.Bootstrap "$@" start \
      "$CATALINA_OUT" 2>&1 "&"                                <-- and here?
      2\>\&1 \&\& echo \$! \>\"$catalina_pid_file\" \; \} $catalina_out_command "&"

  fi

My logging properties file DOES have console logging as a handler:

handlers = 1catalina.org.apache.juli.AsyncFileHandler, 2localhost.org.apache.juli.AsyncFileHandler, java.util.logging.ConsoleHandler

Here is what is being logged.

illa/5.0 (X11; Ubuntu; Linux x86_64; rv:66.0) Gecko/20100101 Firefox/66.0   https://localhost:8444/asWeb/Fitness.jsp
07-May-2019 18:17:52.687 INFO [https-openssl-nio-8444-exec-5] org.restlet.engine.log.LogFilter.afterHandle 2019-05-07   18:17:52    10.0.2.2    -   10.0.2.15   8444    POST    /asWeb/r/WebLinks   -   200 502 22  113 https://localhost:8444  Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:66.0) Gecko/20100101 Firefox/66.0    https://localhost:8444/asWeb/Fitness.jsp

<<<<<< this is where it should have caught a System.out.println("DEBUG message to log"); <<<<<<<<


07-May-2019 18:17:56.318 INFO [https-openssl-nio-8444-exec-6] org.restlet.engine.log.LogFilter.afterHandle 2019-05-07   18:17:56    10.0.2.2    -   10.0.2.15   8444    POST    /asWeb/r/Chart  -   200 93  52  3372    https://localhost:8444  Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:66.0) Gecko/20100101 Firefox/66.0    https://localhost:8444/asWeb/Fitness.jsp
07-May-2019 18:17:58.471 INFO [https-openssl-nio-8444-exec-9] org.restlet.engine.log.LogFilter.afterHandle 2019-05-07   18:17:58    10.0.2.2    -   10.0.2.15   8444    POST    /asWeb/r/Fitness    -   200 -   67  5520    https://localhost:8444  Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:66.0) Gecko/20100101 Firefox/66.0    https://localhost:8444/asWeb/Fitness.jsp

I did notice that the logging hander for console output is set to SystemdFormatter not SimpleFormatter. I tried changing the logging handler to Simple but that had no effect. I set it back to SystemdFormatter.

java.util.logging.ConsoleHandler.level = ALL
java.util.logging.ConsoleHandler.formatter = org.apache.juli.SystemdFormatter

After further digging I am able to view the exceptions in systemd status...

root@ass:/var/log/tomcat9# service tomcat9 status
● tomcat9.service - Apache Tomcat 9 Web Application Server
   Loaded: loaded (/lib/systemd/system/tomcat9.service; enabled; vendor preset: enabled)
   Active: active (running) since Tue 2019-05-07 19:28:36 CDT; 1h 21min ago
     Docs: https://tomcat.apache.org/tomcat-9.0-doc/index.html
  Process: 17539 ExecStartPre=/usr/libexec/tomcat9/tomcat-update-policy.sh (code=exited, status=0/SUCCESS)
 Main PID: 17543 (java)
    Tasks: 42 (limit: 4681)
   CGroup: /system.slice/tomcat9.service
           └─17543 /usr/lib/jvm/default-java/bin/java -Djava.util.logging.config.file=/var/lib/tomcat9/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.awt.headless=true -XX:+UseG1GC -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.ha

May 07 20:48:25 ass tomcat9[17543]: DEBUG Fitness All called!
May 07 20:48:25 ass tomcat9[17543]: 2019-05-07        20:48:25        10.0.2.2        -        10.0.2.15        8444        POST        /asWeb/r/Fitness        -        200        -        67        6696        https://localhost:8444        Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv
May 07 20:48:35 ass tomcat9[17543]: 2019-05-07        20:48:35        10.0.2.2        -        10.0.2.15        8444        POST        /asWeb/r/Wx        -        200        -        115        2260        https://localhost:8444        Mozilla/5.0 (Android 4.4.2; Mobile; rv:66.0) G
May 07 20:49:10 ass tomcat9[17543]: DEBUG Fitness All called!
May 07 20:49:10 ass tomcat9[17543]: 2019-05-07        20:49:10        10.0.2.2        -        10.0.2.15        8444        POST        /asWeb/r/Fitness        -        200        -        67        3860        https://localhost:8444        Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv
May 07 20:49:51 ass tomcat9[17543]: DEBUG Fitness All called!
May 07 20:49:51 ass tomcat9[17543]: 2019-05-07        20:49:51        10.0.2.2        -        10.0.2.15        8444        POST        /asWeb/r/Fitness        -        200        -        67        2196        https://localhost:8444        Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv
May 07 20:49:55 ass tomcat9[17543]: 2019-05-07        20:49:55        10.0.2.2        -        10.0.2.15        8444        POST        /asWeb/r/MediaServer        -        200        -        40        722        https://localhost:8444        Mozilla/5.0 (X11; Ubuntu; Linux x86_64;

And using this I can "mock" catalina.out by using

journalctl -u tomcat9 -f

But I want it in catalina.out, not having to view the journal - if possible.

No. The stack traces can be viewed in journalctl -u tomcat9 though. I think this is "the way forward" as Ubuntu 20.04 is the same. — user3260912, May 18 '20 at 01:36

score 1 · Answer 1 · answered Aug 31 '20 at 01:10

1

You need to edit the file: /lib/systemd/system/tomcat9.service

Comment out the line: SyslogIdentifier=tomcat9

and add these two lines:

StandardOutput=append:/var/log/tomcat9/catalina.out
StandardError=append:/var/log/tomcat9/catalina.out

answered Aug 31 '20 at 01:10

Ted Cahall

141
3

I tested this on Ubuntu 20.04 - but I also copied the logging.properties file from a version I downloaded from Apache. It has more logging directives in it. So that may have helped - but was not sufficient. – Ted Cahall Aug 31 '20 at 01:14
Hi Ted, I tried your solution but failed. I did executed "systemctl daemon-reload" and "systemctl restart tomcat9". But there is still no catalina.out in /var/log/tomcat9. Any other suggestion? – wureka Nov 08 '20 at 05:22
@wureka Could be that your systemd version is old. See https://stackoverflow.com/a/48052152/107301 . – Dark Star1 Sep 10 '21 at 12:28
This worked for me in an lxc container based on 20.04 – Dark Star1 Sep 10 '21 at 12:29

Ubuntu 18.04.2 build of Tomcat 9 not writing to catalina.out, but in systemd status

1 Answers1