Delay init from starting a service for a period of time?

Question

I am trying to get a rudimentary NFS server up and running. Right now the server is configured as an NFS server due to a workaround for a vendor issue not supporting direct attached clustered storage, which we are trying to get them to resolve. The vendor software is Splunk. The splunk feature we are using requires files be located on shared storage (which for us is /mnt/nfs until they support a real clustered filesystem).

Currently the server has a GFS2 filesystem mounted at bootup (it is the only server with the filesystem actively mounted so there should be no problems with locking). We went with GFS2 so switching over to a clustered filesystem is easy should the vendor begin supporting it. NFS is configured to mount that filesystem at /mnt/nfs, which the splunk installation than sees.

Splunk is configured to find it's configuration files in /mnt/nfs. However, I am running into a problem where the splunk daemon starts before nfs is finished loading, and because it sees nothing at /mnt/nfs it starts creating files there, and then when the files disappear (nfs finishes mounting the share), splunk craps out.

Splunk is set to run at runlevel 3, S90. NFS is set at runlevels 2-5, S60. Is there any way to delay the startup of the splunk process further?

score 1 · Answer 1 · answered Sep 13 '12 at 07:43

head /etc/init.d/nfs

#!/bin/sh
#
# nfs           This shell script takes care of starting and stopping
#               the NFS services.
#
# chkconfig: - 60 20
# description: NFS is a popular protocol for file sharing across TCP/IP \
#              networks. This service provides NFS server functionality, \
#              which is configured via the /etc/exports file.
# probe: true

head /etc/init.d/splunk

#!/bin/sh
#
# /etc/init.d/splunk
# init script for Splunk.
# generated by 'splunk enable boot-start'.
#
# chkconfig: 2345 90 60
# description: Splunk indexer service
#
RETVAL=0

Make sure that the Splunk's start priority level (90) is greater than the NFS's start priority level (60).

I am running into a problem where the splunk daemon starts before nfs is finished loading,

If for some reasons, NFS doesn't start successfully, I would do it by using a trick:

Turn off the splunk:
```
chkconfig splunk off
```

Edit the start() function in the NFS init script to start Splunk after NFS is started successfully, something like this:

Before:

echo -n $"Starting NFS daemon: "
daemon rpc.nfsd $RPCNFSDARGS $RPCNFSDCOUNT
RETVAL=$?
echo
[ $RETVAL -ne 0 ] && exit $RETVAL

After:

echo -n $"Starting NFS daemon: "
daemon rpc.nfsd $RPCNFSDARGS $RPCNFSDCOUNT
RETVAL=$?
echo
[ $RETVAL -ne 0 ] && exit $RETVAL || /etc/init.d/splunk start

Give it a try.

Delay init from starting a service for a period of time?

1 Answers1