tomcat6 won't start on SL6

Question

I arrived at work today to find that tomcat6 won't start. When trying to stop and start with the following command(s):

 service tomcat6 stop ; > /var/log/tomcat6/catalina.out && service tomcat6 start && less +F /var/log/tomcat6/catalina.out

It reports back OK for starting tomcat, but using ps, no tomcat process is shown. The catalina.out file doesn't show anything at all either, completely blank. Upon looking at /var/log/tomcat6/tomcat6-initd.log, I see two lines repeated over and over on every attempt:

/usr/sbin/tomcat6: line 60: ${2}: ambiguous redirect
/usr/sbin/tomcat6: line 30: ${2}: ambiguous redirect

Looking at /usr/sbin/tomcat6, both lines read:

${JAVACMD} $JAVA_OPTS $CATALINA_OPTS \
  -classpath "$CLASSPATH" \
  -Dcatalina.base="$CATALINA_BASE" \
  -Dcatalina.home="$CATALINA_HOME" \
  -Djava.endorsed.dirs="$JAVA_ENDORSED_DIRS" \
  -Djava.io.tmpdir="$CATALINA_TMPDIR" \
  -Djava.util.logging.config.file="${CATALINA_BASE}/conf/logging.properties" \
  -Djava.util.logging.manager="org.apache.juli.ClassLoaderLogManager" \
  org.apache.catalina.startup.Bootstrap start \
   >> ${2} 2>&1 &

The first line being the same for both 60 & 30. My $JAVA_OPTS reads:

JAVA_OPTS="-server -Xss128K -Xms3G -Xmx3G -XX:MaxPermSize=256M -Dambiguous.url=https://ambiguous.url.com"

and so far as I can tell (uncertain where to look, honestly), $CATALINA_OPTS is null, but I could be mistaken. One things for certain, I haven't adjusted its value, and am confident that the value has nothing to do with the ambiguous redirect issues.

UPDATE: By updating our tomcat6 package to the latest version, I now do not receive the errors seen above. In fact, the /usr/sbin/tomcat6 file reads entirely different around the areas it was pointing me to before:

  ${JAVACMD} $JAVA_OPTS $CATALINA_OPTS \
    -classpath "$CLASSPATH" \
    -Dcatalina.base="$CATALINA_BASE" \
    -Dcatalina.home="$CATALINA_HOME" \
    -Djava.endorsed.dirs="$JAVA_ENDORSED_DIRS" \
    -Djava.io.tmpdir="$CATALINA_TMPDIR" \
    -Djava.util.logging.config.file="${CATALINA_BASE}/conf/logging.properties" \
    -Djava.util.logging.manager="org.apache.juli.ClassLoaderLogManager" \
    org.apache.catalina.startup.Bootstrap start \
    >> ${CATALINA_BASE}/logs/catalina.out 2>&1 &
    if [ ! -z "$CATALINA_PID" ]; then
      echo $! > $CATALINA_PID
    fi

Now it gets even more complicated, as tomcat6-initd.log does not report anything at all, yet the service still doesn't start up. Running the init script with a -x results in the following output on startup:

# service tomcat6 start
+ '[' -r /lib/lsb/init-functions ']'
+ . /lib/lsb/init-functions
++ lsb_release -i -s
+ DISTRIB_ID=Scientific
++ basename /etc/init.d/tomcat6
+ NAME=tomcat6
+ unset ISBOOT
+ '[' t = S -o t = K ']'
+ '[' -x /sbin/runuser ']'
+ SU='/sbin/runuser -s /bin/sh'
+ TOMCAT_CFG=/etc/tomcat6/tomcat6.conf
+ '[' -r /etc/tomcat6/tomcat6.conf ']'
+ . /etc/tomcat6/tomcat6.conf
++ CATALINA_BASE=/usr/share/tomcat6
++ CATALINA_HOME=/usr/share/tomcat6
++ JASPER_HOME=/usr/share/tomcat6
++ CATALINA_TMPDIR=/var/cache/tomcat6/temp
++ JAVA_OPTS='-Xms3072m -Xmx3072m -Xss512m -XX:MaxPermSize=512m -server -Dambiguous.url=https://ambiguous.url.com'
++ TOMCAT_USER=root
++ SECURITY_MANAGER=false
++ SHUTDOWN_WAIT=30
++ SHUTDOWN_VERBOSE=false
++ CATALINA_PID=/var/run/tomcat6.pid
+ '[' -r /etc/sysconfig/tomcat6 ']'
+ . /etc/sysconfig/tomcat6
+ CONNECTOR_PORT=8080
+ TOMCAT_SCRIPT=/usr/sbin/tomcat6
+ TOMCAT_PROG=tomcat6
+ TOMCAT_USER=root
+ TOMCAT_LOG=/var/log/tomcat6/tomcat6-initd.log
+ RETVAL=0
+ RETVAL=0
+ case "$1" in
+ start
+ echo -n 'Starting tomcat6: '
Starting tomcat6: + '[' 0 '!=' 0 ']'
+ '[' -f /var/lock/subsys/tomcat6 ']'
+ '[' -f /var/run/tomcat6.pid ']'
+ read kpid
+ '[' -d /proc/ ']'
+ log_success_msg
+ /etc/redhat-lsb/lsb_log_message success
                                                           [  OK  ]
+ '[' Scientific = MandrivaLinux ']'
+ RETVAL=0
+ return
+ exit 0

If I were to run the exact same setup on a separate server where tomcat works (there's multiple virtual machines running as a cluster), the output reads:

# service tomcat6 start
+ '[' -r /lib/lsb/init-functions ']'
+ . /lib/lsb/init-functions
++ lsb_release -i -s
+ DISTRIB_ID=Scientific
++ basename /etc/init.d/tomcat6
+ NAME=tomcat6
+ unset ISBOOT
+ '[' t = S -o t = K ']'
+ '[' -x /sbin/runuser ']'
+ SU='/sbin/runuser -s /bin/sh'
+ TOMCAT_CFG=/etc/tomcat6/tomcat6.conf
+ '[' -r /etc/tomcat6/tomcat6.conf ']'
+ . /etc/tomcat6/tomcat6.conf
++ CATALINA_BASE=/usr/share/tomcat6
++ CATALINA_HOME=/usr/share/tomcat6
++ JASPER_HOME=/usr/share/tomcat6
++ CATALINA_TMPDIR=/var/cache/tomcat6/temp
++ JAVA_OPTS='-Xms3072m -Xmx3072m -Xss512m -XX:MaxPermSize=512m -server -Dambiguous.url=https://ambiguous.url.com'
++ TOMCAT_USER=root
++ SECURITY_MANAGER=false
++ SHUTDOWN_WAIT=30
++ SHUTDOWN_VERBOSE=false
++ CATALINA_PID=/var/run/tomcat6.pid
+ '[' -r /etc/sysconfig/tomcat6 ']'
+ . /etc/sysconfig/tomcat6
+ CONNECTOR_PORT=8080
+ TOMCAT_SCRIPT=/usr/sbin/tomcat6
+ TOMCAT_PROG=tomcat6
+ TOMCAT_USER=root
+ TOMCAT_LOG=/var/log/tomcat6/tomcat6-initd.log
+ RETVAL=0
+ RETVAL=0
+ case "$1" in
+ start
+ echo -n 'Starting tomcat6: '
Starting tomcat6:
+ '[' 0 '!=' 0 ']'
+ '[' -f /var/lock/subsys/tomcat6 ']'
+ export CATALINA_PID=/var/run/tomcat6.pid
+ CATALINA_PID=/var/run/tomcat6.pid
+ touch /var/run/tomcat6.pid
+ '[' 0 -eq 0 -a 0 -eq 0 ']'
+ chown root:root /var/run/tomcat6.pid
+ '[' 0 -eq 0 ']'
+ touch /var/log/tomcat6/tomcat6-initd.log
+ '[' 0 -eq 0 -a 0 -eq 0 ']'
+ chown root:root /var/log/tomcat6/tomcat6-initd.log
+ parseOptions
+ options=
++ awk '!/^#/ && !/^$/ { ORS=" "; print "export ", $0, ";" }' /etc/tomcat6/tomcat6.conf
+ options=' export  CATALINA_BASE="/usr/share/tomcat6" ; export  CATALINA_HOME="/usr/share/tomcat6" ; export  JASPER_HOME="/usr/share/tomcat6" ; export  CATALINA_TMPDIR="/var/cache/tomcat6/temp" ; export  JAVA_OPTS="-Xms3072m -Xmx3072m -Xss512m -XX:MaxPermSize=512m -server -Dambiguous.url=https://ambiguous.url.com" ; export  TOMCAT_USER="root" ; export  SECURITY_MANAGER="false" ; export  SHUTDOWN_WAIT="30" ; export  SHUTDOWN_VERBOSE="false" ; export  CATALINA_PID="/var/run/tomcat6.pid" ; '
+ '[' -r /etc/sysconfig/tomcat6 ']'
++ awk '!/^#/ && !/^$/ { ORS=" ";
                                           print "export ", $0, ";" }' /etc/sysconfig/tomcat6
+ options=' export  CATALINA_BASE="/usr/share/tomcat6" ; export  CATALINA_HOME="/usr/share/tomcat6" ; export  JASPER_HOME="/usr/share/tomcat6" ; export  CATALINA_TMPDIR="/var/cache/tomcat6/temp" ; export  JAVA_OPTS="-Xms3072m -Xmx3072m -Xss512m -XX:MaxPermSize=512m -server -Dambiguous.url=https://ambiguous.url.com" ; export  TOMCAT_USER="root" ; export  SECURITY_MANAGER="false" ; export  SHUTDOWN_WAIT="30" ; export  SHUTDOWN_VERBOSE="false" ; export  CATALINA_PID="/var/run/tomcat6.pid" ;  '
+ TOMCAT_SCRIPT=' export  CATALINA_BASE="/usr/share/tomcat6" ; export  CATALINA_HOME="/usr/share/tomcat6" ; export  JASPER_HOME="/usr/share/tomcat6" ; export  CATALINA_TMPDIR="/var/cache/tomcat6/temp" ; export  JAVA_OPTS="-Xms3072m -Xmx3072m -Xss512m -XX:MaxPermSize=512m -server -Dambiguous.url=https://ambiguous.url.com" ; export  TOMCAT_USER="root" ; export  SECURITY_MANAGER="false" ; export  SHUTDOWN_WAIT="30" ; export  SHUTDOWN_VERBOSE="false" ; export  CATALINA_PID="/var/run/tomcat6.pid" ;   /usr/sbin/tomcat6'
+ '[' 0 -eq 0 -a false = true ']'
+ '[' 0 -eq 0 ']'
+ /sbin/runuser -s /bin/sh - root -c ' export  CATALINA_BASE="/usr/share/tomcat6" ; export  CATALINA_HOME="/usr/share/tomcat6" ; export  JASPER_HOME="/usr/share/tomcat6" ; export  CATALINA_TMPDIR="/var/cache/tomcat6/temp" ; export  JAVA_OPTS="-Xms3072m -Xmx3072m -Xss512m -XX:MaxPermSize=512m -server -Dambiguous.url=https://ambiguous.url.com" ; export  TOMCAT_USER="root" ; export  SECURITY_MANAGER="false" ; export  SHUTDOWN_WAIT="30" ; export  SHUTDOWN_VERBOSE="false" ; export  CATALINA_PID="/var/run/tomcat6.pid" ;   /usr/sbin/tomcat6 start'
+ '[' 0 -eq 0 ']'
+ log_success_msg
+ /etc/redhat-lsb/lsb_log_message success
                                                           [  OK  ]
+ touch /var/lock/subsys/tomcat6
+ '[' Scientific = MandrivaLinux ']'
+ exit 0

And just to be thorough, the start() function in /etc/init.d/tomcat6 reads:

function start() {

   echo -n "Starting ${TOMCAT_PROG}: "
   if [ "$RETVAL" != "0" ]; then
     log_failure_msg
     return
   fi
   if [ -f "/var/lock/subsys/${NAME}" ]; then
        if [ -f "/var/run/${NAME}.pid" ]; then
            read kpid < /var/run/${NAME}.pid
#           if checkpid $kpid 2>&1; then
            if [ -d "/proc/${kpid}" ]; then
                log_success_msg
                if [ "$DISTRIB_ID" = "MandrivaLinux" ]; then
                    echo
                fi
                RETVAL="0"
                return
            fi
        fi
    fi
    # fix permissions on the log and pid files
    export CATALINA_PID="/var/run/${NAME}.pid"
    touch $CATALINA_PID 2>&1 || RETVAL="4"
    if [ "$RETVAL" -eq "0" -a "$?" -eq "0" ]; then
      chown ${TOMCAT_USER}:${TOMCAT_USER} $CATALINA_PID
    fi
    [ "$RETVAL" -eq "0" ] && touch $TOMCAT_LOG 2>&1 || RETVAL="4"
    if [ "$RETVAL" -eq "0" -a "$?" -eq "0" ]; then
      chown ${TOMCAT_USER}:${TOMCAT_USER} $TOMCAT_LOG
    fi
    parseOptions
    if [ "$RETVAL" -eq "0" -a "$SECURITY_MANAGER" = "true" ]; then
        $SU - $TOMCAT_USER -c "${TOMCAT_SCRIPT} start-security" \
            >> ${TOMCAT_LOG} 2>&1 || RETVAL="4"
    else

       [ "$RETVAL" -eq "0" ] && $SU - $TOMCAT_USER -c "${TOMCAT_SCRIPT} start" >> ${TOMCAT_LOG} 2>&1 || RETVAL="4"
    fi
    if [ "$RETVAL" -eq "0" ]; then
        log_success_msg
        touch /var/lock/subsys/${NAME}
    else
        log_failure_msg "Error code ${RETVAL}"
    fi
    if [ "$DISTRIB_ID" = "MandrivaLinux" ]; then
        echo
    fi
}

60 and 30 read `${JAVACMD} $JAVA_OPTS $CATALINA_OPTS \`, but I assume it's referencing ${2} at the very end. The ambiguous URL is a variable we set in the JAVA_OPTS to help us reference our single sign on service, and I omitted the URL for security reasons. — Scott, Aug 22 '12 at 19:35
Have you tried starting it only using catalina.sh directly and watching the catalina.out? — Chida, Aug 22 '12 at 19:37
Interestingly enough, in Google'ing, I've seen others do the same, but I am unable to find that file. I presume it's in tomcat's home directory (/usr/share/tomcat for me) under the bin directory, but it's not. Apparently /etc/init.d/tomcat6 ultimately calls to /usr/sbin/tomcat6. Would that be the equivalent? — Scott, Aug 22 '12 at 19:45
A bit puzzling. Do you see any pid file such as this or anything else -- /var/run/tomcat6.pid — Chida, Aug 22 '12 at 19:53
... Rebooting fixed it... Out goes the whole "You never need to reboot linux" saying. A lesser man might be embarrassed. — Scott, Aug 22 '12 at 20:02

score 2 · Accepted Answer · answered Aug 22 '12 at 20:04

2

Typed reboot into the console, mashed the enter key with considerable force using my forehead, and the problem went away.

answered Aug 22 '12 at 20:04

Scott

397
1
4
14

Same thing happened to me. I don't know why either. – Nic May 07 '14 at 18:22
I encountered this again, and this time installing `tomcat-webapps` seemed to help. Still not sure why. – Nic May 21 '14 at 23:06

tomcat6 won't start on SL6

1 Answers1