Our current configuration:
- Debian 8.6
- Nginx web server
- PHP5-FPM v5.6.27-0+deb8u1
- Zend Engine v2.6.0 with Zend OPcache v7.0.6-dev
We use Phalcon framework v2.0.11 (and v2.0.13 on test environment, where we experience the same errors). In order to work, Phalcon needs to be compiled and then it becomes a PHP pre-loaded module (phalcon.so). During compilation, it invokes Zend libraries among the others.
We also use Memcached (as service and as PHP module).
The application runs normally except that Nginx randomly throws:
502 Bad Gateway
errors during navigation. Reloading the page (F5) or pressing "Back" browser button, the page gets loaded without any error.
Sometimes, 502 errors are more frequent than other moments, apparently regardless of the load or traffic on the server.
The only errors we can read from the logs are not eloquent at all:
php5-fpm.log:
WARNING: [pool www] child 2183 exited on signal 7 (SIGBUS) after 0.120012 seconds from start
WARNING: [pool www] child 1391 exited on signal 7 (SIGBUS) after 59.871442 seconds from start
WARNING: [pool www] child 12836 exited on signal 7 (SIGBUS - core dumped) after 560.364868 seconds from start
WARNING: [pool www] child 10874 exited on signal 7 (SIGBUS - core dumped) after 38.964131 seconds from start
...
...
nginx/error.log:
[error] 8428#0: *368771 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: xxx.xxx.xxx.xxx, server: xxxxxx.xxxxxxxxx.xxx, request: "POST /abc/def_ghi HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "xxxxxx.xxxxxxxxx.xxx", referrer: "https://xxxxxx.xxxxxxxxx.xxx/abc"
...
...
After days of reasearch, we tried applying all the suggestions we found on internet. The parameters we modified, tested and checked on Nginx and php-fpm are, among the others:
(on php.ini)
output_buffering
max_execution_time
memory_limit
(on www.conf)
listen = /var/run/php5-fpm.sock or listen = 127.0.0.1:9000
pm = ondemand/static/etc.....
pm.max_children 500/30/1/100/etc....
pm.start_servers = 30/50/1/etc......
pm.min_spare_servers
pm.max_spare_servers
pm.max_requests
(on nginx virtual server conf file)
fastcgi_pass
fastcgi_buffers
fastcgi_buffer_size
fastcgi_connect_timeout
fastcgi_send_timeout
fastcgi_read_timeout
Every combination with any value on the above parameters didn't change anything on the "502" errors behaviour. They keep appearing sometimes.
So, we tried dumping the core of PHP on process crash with GDB. We found out that every time a 502 error is thrown, we receive always the same identical dump (I mean, with the same error). This is an example of a dump:
GNU gdb (Debian 7.7.1+dfsg-5) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/sbin/php5-fpm...Reading symbols from /usr/lib/debug/.build-id/d4/62618919aec6e5b126ad219b9d08046ef6b875.debug...done.
done.
[New LWP 17814]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `php-fpm: pool www '.
Program terminated with signal SIGBUS, Bus error.
#0 lex_scan (zendlval=zendlval@entry=0x7fff14a7b0b8) at Zend/zend_language_scanner.c:1082
1082 Zend/zend_language_scanner.c: no such file or directory.
The error is this one:
#0 lex_scan (zendlval=zendlval@entry=0x7fff14a7b0b8) at Zend/zend_language_scanner.c:1082
1082 Zend/zend_language_scanner.c: no such file or directory.
If we search this error on the internet, we find little or nothing.
We tried recompiling Phalcon, after an upgrade of PHP (from PHP 5.6.24 to 5.6.27), but the error keeps appearing.
We honestly can't understand what else we should do to have an explanation on this error and understand how to solve it definitively.
Thank you for your help.
