12

I've compiled nginx with custom paths for my production server and when I try to start/restart the service using:

service nginx start

or

service nginx restart

It enters a new line without returning the shell: Picture of terminal when running the command

So the problem is that I can't control nginx using service command. The service actually runs but it won't return a shell to me so I always have to press ctrl+c to get it back.

I also have to mention that nginx runs just fine when calling it by its own nginx command and stops or reloads easily using nginx -s stop/reload.

This problem persists using systemctl start nginx as well, but systemctl stop nginx works just fine.

Info:

$ lsb_release -a
    Distributor ID: Ubuntu
    Description:    Ubuntu 15.10
    Release:    15.10
    Codename:   wily

$ uname -r
    4.2.0-27-generic

$ nginx -V
    nginx version: nginx/1.9.11
    built by gcc 5.2.1 20151010 (Ubuntu 5.2.1-22ubuntu2) 
    built with OpenSSL 1.0.2d 9 Jul 2015
    TLS SNI support enabled
    configure arguments: --sbin-path=/usr/bin/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --with-debug --with-pcre --with-http_ssl_module

$ cat /etc/default/nginx
    NGINX_CONF_FILE=/etc/nginx/nginx.conf
    DAEMON=/usr/bin/nginx

$ cat /etc/init.d/nginx
    NGINX_BIN=/usr/bin/nginx
    test -x $NGINX_BIN || { echo "$NGINX_BIN not installed"; 
        if [ "$1" = "stop" ]; then exit 0;
        else exit 5; fi; }
    NGINX_PID=/var/run/nginx.pid

    # Check for existence of needed config file and read it
    #NGINX_CONFIG=/etc/sysconfig/nginx
    #test -r $NGINX_CONFIG || { echo "$NGINX_CONFIG not existing";
    #   if [ "$1" = "stop" ]; then exit 0;
    #   else exit 6; fi; }
    #
    # Read config   
    #. $NGINX_CONFIG

    # Source LSB init functions
    # providing start_daemon, killproc, pidofproc, 
    # log_success_msg, log_failure_msg and log_warning_msg.
    # This is currently not used by UnitedLinux based distributions and
    # not needed for init scripts for UnitedLinux only. If it is used,
    # the functions from rc.status should not be sourced or used.
    #. /lib/lsb/init-functions

    # Shell functions sourced from /etc/rc.status:
    #      rc_check         check and set local and overall rc status
    #      rc_status        check and set local and overall rc status
    #      rc_status -v     be verbose in local rc status and clear it afterwards
    #      rc_status -v -r  ditto and clear both the local and overall rc status
    #      rc_status -s     display "skipped" and exit with status 3
    #      rc_status -u     display "unused" and exit with status 3
    #      rc_failed        set local and overall rc status to failed
    #      rc_failed <num>  set local and overall rc status to <num>
    #      rc_reset         clear both the local and overall rc status
    #      rc_exit          exit appropriate to overall rc status
    #      rc_active        checks whether a service is activated by symlinks
    . /etc/rc.status

    # Reset status of this service
    rc_reset

    # Return values acc. to LSB for all commands but status:
    # 0   - success
    # 1       - generic or unspecified error
    # 2       - invalid or excess argument(s)
    # 3       - unimplemented feature (e.g. "reload")
    # 4       - user had insufficient privileges
    # 5       - program is not installed
    # 6       - program is not configured
    # 7       - program is not running
    # 8--199  - reserved (8--99 LSB, 100--149 distrib, 150--199 appl)
    # 
    # Note that starting an already running service, stopping
    # or restarting a not-running service as well as the restart
    # with force-reload (in case signaling is not supported) are
    # considered a success.

    case "$1" in
        start)
        echo -n "Starting nginx "
        ## Start daemon with startproc(8). If this fails
        ## the return value is set appropriately by startproc.
        /sbin/startproc -p $NGINX_PID $NGINX_BIN

        # Remember status and be verbose
        rc_status -v
        ;;
        stop)
        echo -n "Shutting down nginx "
        ## Stop daemon with killproc(8) and if this fails
        ## killproc sets the return value according to LSB.

        /sbin/killproc -p $NGINX_PID -TERM $NGINX_BIN

        # Remember status and be verbose
        rc_status -v
        ;;
        try-restart|condrestart)
        ## Do a restart only if the service was active before.
        ## Note: try-restart is now part of LSB (as of 1.9).
        ## RH has a similar command named condrestart.
        if test "$1" = "condrestart"; then
            echo "${attn} Use try-restart ${done}(LSB)${attn} rather than condrestart ${warn}(RH)${norm}"
        fi
        $0 status
        if test $? = 0; then
            $0 restart
        else
            rc_reset    # Not running is not a failure.
        fi
        # Remember status and be quiet
        rc_status
        ;;
        restart)
        ## Stop the service and regardless of whether it was
        ## running or not, start it again.
        $0 stop
        $0 start

        # Remember status and be quiet
        rc_status
        ;;
        force-reload)
        ## Signal the daemon to reload its config. Most daemons
        ## do this on signal 1 (SIGHUP).
        ## If it does not support it, restart the service if it
        ## is running.

        echo -n "Reload service nginx "
        ## if it supports it:
        /sbin/killproc -p $NGINX_PID -HUP $NGINX_BIN
        #touch /run/nginx.pid
        rc_status -v

        ## Otherwise:
        #$0 try-restart
        #rc_status
        ;;
        reload)
        ## Like force-reload, but if daemon does not support
        ## signaling, do nothing (!)

        # If it supports signaling:
        echo -n "Reload service nginx "
        /sbin/killproc -p $NGINX_PID -HUP $NGINX_BIN
        #touch /run/nginx.pid
        rc_status -v

        ## Otherwise if it does not support reload:
        #rc_failed 3
        #rc_status -v
        ;;
        reopen)
            echo -n "Reopen the logfiles "
            /sbin/killproc -p $NGINX_PID -USR1 $NGINX_BIN
            rc_status -v
            ;;

        status)
        echo -n "Checking for service nginx "
        ## Check status with checkproc(8), if process is running
        ## checkproc will return with exit status 0.

        # Return value is slightly different for the status command:
        # 0 - service up and running
        # 1 - service dead, but /run/  pid  file exists
        # 2 - service dead, but /var/lock/ lock file exists
        # 3 - service not running (unused)
        # 4 - service status unknown :-(
        # 5--199 reserved (5--99 LSB, 100--149 distro, 150--199 appl.)

        # NOTE: checkproc returns LSB compliant status values.
        /sbin/checkproc -p $NGINX_PID $NGINX_BIN
        # NOTE: rc_status knows that we called this init script with
        # "status" option and adapts its messages accordingly.
        rc_status -v
        ;;
        probe)
        ## Optional: Probe for the necessity of a reload, print out the
        ## argument to this init script which is required for a reload.
        ## Note: probe is not (yet) part of LSB (as of 1.9)

        test /etc/nginx/nginx.conf -nt /run/nginx.pid && echo reload
        ;;
        *)
        echo "Usage: $0 {start|stop|status|try-restart|restart|force-reload|reload|probe}"
        exit 1
        ;;
    esac
    rc_exit

UPDATE: The problem also persists while using Docker container on CoreOS alpha.

UPDATE 2: Here are the outputs for strace -o log -f service nginx start and journalctl -xe:

strace -o log -f service nginx start Log output [too long to post here]


    journalctl -xe
    Feb 26 07:25:38 lucifer polkitd(authority=local)[870]: Registered Authentication Agent for unix-process:8181:8813595 (system bus name :1.77 [/usr/bin/pkttyagent --notify-fd 5 --fallback], o
Feb 26 07:25:38 lucifer systemd[1]: Starting The NGINX HTTP and reverse proxy server...
-- Subject: Unit nginx.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit nginx.service has begun starting up.
Feb 26 07:25:38 lucifer nginx[8211]: nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
Feb 26 07:25:38 lucifer nginx[8211]: nginx: configuration file /etc/nginx/nginx.conf test is successful
Feb 26 07:25:38 lucifer systemd[1]: nginx.service: PID file /var/run/nginx.pid not readable (yet?) after start: No such file or directory
Feb 26 07:25:43 lucifer polkitd(authority=local)[870]: Unregistered Authentication Agent for unix-process:8181:8813595 (system bus name :1.77, object path /org/freedesktop/PolicyKit1/Authen
T0M XeOn LuCiFeR
  • 121
  • 1
  • 1
  • 6
  • 1
    Since `nginx` is working fine when starting from a cli on his own, you need to debug it's *service* script. – drookie Feb 23 '16 at 15:08
  • Try to strace the problem. It will show you what system call it can't finish. Just install strace, then strace -ff service nginx start. – anx Feb 24 '16 at 10:34
  • @drookie @anx I have posted the results of `journalctl` and `strace` here but honestly i have no clue how to understand the problem from strace's output. i would appreciate a little bit of help, if possible. – T0M XeOn LuCiFeR Feb 26 '16 at 12:24
  • @T0M XeOn LuCiFeR have you fixed this issue ? – Bhargav Patel Dec 22 '17 at 06:47
  • Maybe related to this bug: https://bugs.launchpad.net/ubuntu/+source/nginx/+bug/1581864 – Elliott B Jun 25 '18 at 04:20

4 Answers4

9

I'm having the same issue with Ubuntu 16.04, systemd and NginX 1.10.1 from source.

I was using the default nginx.service file: https://www.nginx.com/resources/wiki/start/topics/examples/systemd/

The issue was the nginx.pid location. To fix it, I:

  1. Fired up Nginx without the service

    sudo nginx start
    
  2. Updated the locate db

    sudo updatedb
    
  3. Found the location of the pid file

    locate "nginx.pid"
    
  4. Updated the nginx.service file to the location where I found it

    PIDFile=/usr/local/nginx/logs/nginx.pid
    

    (No idea why it was stored in my logs dir...)

  5. Then run daemon-reload to reload the nginx.service file

    systemctl daemon-reload
    

Afterwards, systemctl start nginx works like a charm. Hope this helps.

X X
  • 5
  • 3
Maxim
  • 191
  • 1
  • 2
  • Given `/usr/local/nginx/logs/nginx.pid`as path for `PIDFile` worked for me (using ubuntu server 16.04.2 and nginx 1.12.1). Thx – youssman Jul 18 '17 at 15:21
  • It worked for me. When building nginx from source I had configured it with " --pid-path=/usr/local/nginx/nginx.pid". When I set up nginx.service, I had used wrong path and cause the same problem. When corrected, it did not caused hanging and subsequent termination. Thanks! – retromuz Oct 21 '18 at 03:26
3

It hangs because of this error:

PID file /var/run/nginx.pid not readable (yet?) after start

Newer Linux distros comes with systemd. If you use a service bundled with your distro you'll get it already configured for systemd.

Since you are compiling nginx from the sources and you are using a SysV init file (/etc/init.d/nginx), systemd will use a generator to parse it (systemd-sysv-generator).

In your SysV script, you define the pid file and starts the process with:

NGINX_PID=/var/run/nginx.pid
...
/sbin/startproc -p $NGINX_PID $NGINX_BIN

If I'm not wrong you are using a SUSE Linux init script on Ubuntu (because of the startproc command), that startproc command only reads the pid file (specified by the -p parameter), it doesn't create it, thus systemd can't find a pid file and it hangs.

In your case, the solution is either to create the pid file in your SysV init script (on the /var/run/nginx.pid location), use an Ubuntu SysV init script, or a systemd one.

This may also happen (not what's happening to you) when you have a correct SysV init script which creates a pid file, but it's different from the one commented at the top of the file. The systemd generator reads the comments, for example this one:

# pidfile: /var/run/nginxd.pid

And uses the pidfile defined there.

More information:

DiegoG
  • 235
  • 2
  • 4
2

It sounds like /usr/lib/systemd/system/nginx.service is malformed. Make sure this line is set:

PIDFile=/var/run/nginx.pid

Alternatively you can copy the [Service] section to a new file called /etc/systemd/system/nginx.service. Unit files placed in /etc/systemd/system allow you to override sections without duplicating the entire file installed by the package manager.

eradman
  • 150
  • 4
0

Inspiration https://serverfault.com/a/788576 and https://askubuntu.com/a/733604

Ubuntu 20.04

I changed two things

First: I runned service NetworkManager-wait

sudo systemctl enable NetworkManager-wait-online.service
sudo systemctl start NetworkManager-wait-online.service

Second: I added to path PIDFile (and pidfile) prefix var

[Service]
...
PIDFile=/var/run/nginx.pid
...
ExecStop=-/sbin/start-stop-daemon --quiet --stop --retry QUIT/5 --pidfile /var/run/nginx.pid
..