You can safely restart celery worker without losing any tasks with the TERM signal. So the celery will end the current tasks and die.
If you want your tasks to be retried if something goes wrong you can also use the acks_late option (Task.acks_late / CELERY_ACKS_LATE).
Q: How can I safely shut down the worker?
(from: http://celery.readthedocs.org/en/latest/faq.html#how-can-i-safely-shut-down-the-worker)
Answer: Use the TERM signal, and the worker will finish all currently executing jobs and shut down as soon as possible. No tasks
should be lost.
You should never stop worker with the KILL signal (-9), unless you’ve tried TERM a few times and waited a few minutes to let it get a
chance to shut down. As if you do tasks may be terminated
mid-execution, and they will not be re-run unless you have the
acks_late option set (Task.acks_late / CELERY_ACKS_LATE).
To stop the worker
(from: http://celery.readthedocs.org/en/latest/userguide/workers.html#worker-stopping)
Q: Stopping the worker
Shutdown should be accomplished using the TERM signal.
To send a TERM signal, use the linux command KILL
(from: http://linux.about.com/od/commands/l/blcmdl1_kill.htm)
kill - terminate a process SYNOPSIS
kill [ -s signal | -p ] [ -a ] [ -- ] pid ... kill -l [ signal ]
DESCRIPTION
The command kill sends the specified signal to the specified process
or process group. If no signal is specified, the TERM signal is sent.
The TERM signal will kill processes which do not catch this signal.
For other processes, it may be necessary to use the KILL (9) signal,
since this signal cannot be caught.