The Issue (Modified: with REPRO STEPS!)
To repro, spin up a new VPS with Ubuntu 16.04 (probably works with any Debian distro, but not tested) and follow these instructions:
Through experimentation I boiled down the instructions to the absolute minimum necessary to repro
apt-get install php7.0 php7.0-dev php-pear
apt-get install libsodium-dev
pecl install libsodium
echo "extension=libsodium.so" >> /etc/php/7.0/fpm/php.ini
shutdown -r 0
When I reboot the server (shutdown -r 0
) PHP-FPM is dead (confirm via service php7.0-fpm status
)
If I run service php7.0-fpm start
, it starts up in a matter of milliseconds with no issue.
I've created an issue on the libsodium-php tracker in hopes someone can find a solution there: https://github.com/jedisct1/libsodium-php/issues/94
The Question
Now that I've narrowed down the issue, what I'm really looking for is either:
A way to figure out why it's timing out (is there a good way to profile what resources are being accessed or what a boot-time process is doing?)
A way to fix it (is there some way to alter the boot order so that libsodium boots last, in case it's a dependency issue?)
New Updates
I've continued to look into this and figured out that all of PHP is stalling during the initialization of the libsodium library (the call to sodium_init()
). This call takes about 3-4 minutes to complete, and until this time has passed even a call to php -v
fails.
As a workaround I have added the following to /lib/systemd/system/php7.0-fpm.service
:
TimeoutStartSec=600
This causes my web server to return HTTP status code 502 for a couple of minutes after a reboot but eventually it starts working again (better than manually having to log in and fix it)
I will try to get in contact with the developers of libsodium and libsodium-php to find a better solution, however for now I believe I have solved this myself:
The Solution
When something isn't running at startup, begin investigation with
journalctl -u <service>
If it's timing out, try a fresh install -- disable all extensions, modules, plugins, remove custom config, etc. If the fresh install works, enable things one at a time until you find the cause
If you absolutely need the plugin / extension / whatever then you can increase the timeout by modifying the
*.service
file and addingTimeoutStartSec=...