Using Internet Explorer to invoke PHP/CURL for long running data API causes Apache 2 server to freeze and require restart

Question

I'm running a PHP program that works fine as long as it's not invoked by a Microsoft Internet Explorer browser, after which it spawns the below processes, locks up Apache 2 and requires a restart of the web server (on Ubuntu 12.04 LTS).

bob@drools:/etc/php5/apache2# ps auxwww | grep apache2
root      8737  0.1  2.5 369164 25800 ?        Ssl  12:41   0:00 /usr/sbin/apache2 -k start
www-data  8743  0.0  3.2 393748 33268 ?        Sl   12:41   0:00 /usr/sbin/apache2 -k start
www-data  8755  0.1  3.3 393856 33904 ?        Sl   12:41   0:00 /usr/sbin/apache2 -k start
www-data  8779  0.1  3.2 393724 33252 ?        Sl   12:45   0:00 /usr/sbin/apache2 -k start
www-data  8782  0.1  3.2 393716 33236 ?        Sl   12:45   0:00 /usr/sbin/apache2 -k start
www-data  8785  0.1  3.2 393684 33204 ?        Sl   12:45   0:00 /usr/sbin/apache2 -k start
www-data  8812  1.1  3.2 393684 33264 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
www-data  8815  1.3  3.2 393684 33264 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
www-data  8818  1.3  3.2 393684 33264 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
www-data  8821  1.5  3.2 393684 33264 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
www-data  8824  1.4  3.2 393684 33264 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
www-data  8827  1.4  3.2 393684 33264 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
www-data  8830  1.4  3.2 393684 33264 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
www-data  8835  2.5  3.2 393684 33256 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
www-data  8838  2.8  3.2 393684 33264 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
www-data  8841  2.5  3.2 393684 33264 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
www-data  8844  2.5  3.2 393684 33264 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
www-data  8847  3.2  3.2 393684 33264 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
www-data  8850  3.0  3.2 393684 33264 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
www-data  8853  3.2  3.2 393684 33264 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
www-data  8856  3.2  3.2 393684 33264 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
www-data  8861  3.3  3.2 393684 33264 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
www-data  8864  3.6  3.2 393684 33264 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
www-data  8867  3.5  3.2 393684 33264 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
www-data  8870  3.6  3.2 393684 33264 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
www-data  8873  3.6  3.2 393684 33264 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
www-data  8876  3.5  3.2 393684 33264 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
www-data  8879  3.3  3.2 393684 33264 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
www-data  8881  3.5  3.2 393684 33264 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
www-data  8883  3.6  3.2 393684 33264 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
www-data  8886  3.5  3.2 393684 33264 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
www-data  8891  3.5  3.2 393684 33264 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
www-data  8894  3.5  3.2 393684 33264 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
www-data  8896  3.5  3.2 393684 33264 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
www-data  8900  3.5  3.2 393684 33264 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
www-data  8901  3.5  3.2 393684 33264 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
www-data  8904  3.5  3.2 393684 33264 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
www-data  8909  3.8  3.2 393684 33264 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
www-data  8912  3.8  3.2 393684 33264 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
www-data  8915  3.8  3.2 393684 33264 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
www-data  8918  3.6  3.2 393684 33260 ?        Sl   12:47   0:00 /usr/sbin/apache2 -k start
root      8922  0.0  0.1   9396  2000 pts/0    S+   12:47   0:00 grep --color=auto apache2

It used to lock up the entire server until I changed some of the "mpm_" module parameters to something more reasonable in /etc/spache2/apache2.conf.

Given the issues with Internet Explorer, I've even added this line:

**" SetEnvIf User-Agent ".*MSIE.*"   nokeepalive "**

in the virtual hosts file located here: /etc/apache2/sites-available.

There are a number of articles written on the issue, but I've not had any success implementing any of them:

Apache Server 2 hangs after receiving requests from IE 10/11:

More R&D: Internet Explorer 10 (Windows 8) crashing Apache

The PHP program uses cURL to take a list of 25 items and perform a (GET) API call for each to an external server that returns JSON data for further processing. It's a classical long running data program.

What bakes my noodle is that it runs fine in every other browser except Internet Explorer - which causes the web server to misbehave.

I've interrogated the listed R&D and then some, implemented the suggested fixes, yet I still get the same predictable, recreatable, problematic server behavior.

I need to figure out how to protect the server from behaving badly when it encounters and the Internet Explorer browser making these particular requests of it. I'd like to understand why it happens in the first place.

Any guidance, perspectives, direction or solutions would be greatly appreciated...

Here's a snapshot of my cURL code:

<?php

// *** CURL Init, SetOps, and Execution Statements ****
$ch = curl_init();


// *** Execute the  API call for each part number and store in the Associative Array ****
$index=0;
foreach ($partNumbersArray as $partNum) {

    $MyValue = $partNum;

    $MyUrl = $MyNiinjaBaseURL."/".$APICmd1."/".$MyDataSet."/".$MyValue."?key=".$MyKey."&$"."filter=substringof('".$MyValue."',PartNumbers)";


    // *** cURL SetOpts, and Execution Statements ****
    curl_setopt($ch, CURLOPT_URL, $MyUrl);
    curl_setopt($ch, CURLOPT_HEADER, 0);
    curl_setopt($ch, CURLOPT_FRESH_CONNECT, true);
    // curl_setopt($ch, CURLOPT_TIMEOUT, 15);       // <= THIS *never* worked with any reliability ....
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

    $server_output = curl_exec ($ch);   // <= THIS executes the cURL call and stores the resulting JSON object in the variable '$server_output'

    $niinjaResultsJsonArray[$MyValue] = $server_output;        // Add the JSON object to the Array and index to PartNumber
    $index++;                                                // Increment the index

} // End Execution of NIINJA API Calls

// ** Close the CURL Object and release resources
curl_close ($ch);

?>

Here is the PHP information page: http://www.versaggi.net/phptest.phtml

I think you need to somehow log the full incoming HTTP requests from both IE and some other browser that does not have a problem so we can compare them and look for differences. Please have a look at [this question](http://serverfault.com/questions/51409/how-to-dump-entire-http-requests-with-apache) for how you can do that. There must be something IE does with the HTTP request (some extra or missing header etc?) that leads Apache to treat it differently. Either that, or it's at a lower (IP packets) level, which I sure don't hope. — Stijn de Witt, Jul 27 '14 at 21:45
I put a bounty on it to hopefully help you get some attention for your question. — Stijn de Witt, Jul 27 '14 at 21:48
Thinking of it some more, you could probably report this to Apache as a bug... Because there really is no way I can explain this away as not an Apache bug. That might also help you get some very experienced Apache gurus to look at the issue (and hopefully fix it). If you want to go that route it might help if you shrink down the page experiencing the issue to the smallest possible scenario that still has the issue. This can be helpful in and of itself anyways. — Stijn de Witt, Jul 27 '14 at 21:58
Does the client have any influence on which URLs are being accessed by the php script? Are the cURL requests still ongoing when you find the server in the above state? Could it be that IE is retrying the requests when they are responding too slowly? If each HTTP request to your webserver can cause it to initiate another 25 HTTP requests to a backend, that can escalate quite quickly. Could you reuse the responses you get with cURL for more than one client? — kasperd, Jul 29 '14 at 11:26
I've added my CURL code for perusal in the main description ...it's been edited to protect the innocent ... — ProfVersaggi, Jul 29 '14 at 11:58
Referenced in [a comment to *Apache server 2.4.6 hangs after serving requests from Internet Explorer 10/11*](https://stijndewitt.com/2014/01/10/apache-hangs-ie11/#comment-406). — Peter Mortensen, Oct 25 '19 at 16:51

mc0e · Answer 1 · 2019-10-27T17:57:51.767

A long time ago, I saw Apache lock-ups resulting from an Apache process making a call via HTTP to another URL serviced by an Apache process on the same server. I sometimes wound up with a bunch of processes waiting on such calls with no available Apache processes to service them. In my case I had a translation layer in front of some web pages, but calling an API on your own site is much the same thing.

The characteristics of the browser making the original call could make this more likely to occur. For example, keep-alive, timeout behaviour and so forth, but it's not fundamentally the browser at fault.

If it's anything like what I saw, then you want to be looking at the timeout behaviour in your use of curl. The code you've included suggests you're onto that, but you might need to be more fine-grained in your understanding of exactly what point in the request it's getting to. It might be interesting to look at it with tcpdump (or ngrep, Wireshark, or whatever). Also it'd be good to know what system call is in progress when the calling process hangs. That is, look at it with strace -p [PID].

You should probably also be thinking about whether you can remove the HTTP call from your use of the API. Can you keep things within the same Apache process by making a direct call to the appropriate code that handles the API request?

It's probably relevant to tell people how you're running PHP (E.g. mod_php, fpm, etc.). That may be part of understanding the mechanism by which the code locks up.

This might help with the interrogation of the PHP internals ... http://www.versaggi.net/phptest.phtml — ProfVersaggi, Jul 30 '14 at 11:40
As an experiment I did disable the API calls from the CURL loop and ran a bunch of tests. That isolated the problem as during those tests, Apache2 deamon remained healthy. — ProfVersaggi, Jul 30 '14 at 16:46

Using Internet Explorer to invoke PHP/CURL for long running data API causes Apache 2 server to freeze and require restart

1 Answers1