Apache - Custom error page returning error AH01071 only for file requests

Question

I have a site hosted with a shared hosting provider. It is an Apache with FPM/FastCGI and PHP 7.2

Being shared hosting, the only configuration I have access to is htaccess, but obviously not any of the Apache conf files.

I have a custom error page configured in my htaccess like this: ErrorDocument 404 /error404.php. Today I noticed that my custom error 404 page was not being displayed. Instead plain text File not found. is returned to the browser with the status code 404 in the header. Further investigation revealed that this only happens when the request is for a file. If you request a non-existent directory then you get the custom error page! For example, requesting mydomain.info/dummy.htm gives the error but requesting mydomain.info/dummy/ returns the custom error page.

The server is logging error AH01071 which is Primary script unknown for every File not found. error.

It appears that ModSecurity is enabled on the server because the logs record malicious requests being rejected, e.g. [client xxx.xxx.xxx.xxx] ModSecurity: Access denied with code 403 (phase 2). ... etc

Also, I recently changed to PHP 7.2 as recommended by the hosting provider. Changing back to 5.6 doesn't change the symptoms though.

Any ideas what is causing this? I have seen info that suggests perhaps a ProxyPass or ProxyErrorOverride might fix the problem, but I don't know where to set that up.

For the record, here is the complete htaccess, warts and all:

RewriteEngine on

# AddType TYPE/SUBTYPE EXTENSION
AddType audio/mpeg mp3
AddType video/mp4 mp4 m4v

# Add WWW
RewriteCond %{HTTP_HOST} ^mydomain\.info [NC]
RewriteRule ^(.*) https://www.mydomain.info/$1 [R=301,L,NE]

# Redirect for .COM
RewriteCond %{HTTP_HOST} mydomain\.com$ [NC]
RewriteRule ^/?(.*) https://www.mydomain.info/$1 [R=301,L,NE]

# Force HTTPS
RewriteEngine On 
RewriteCond %{HTTPS} off 
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L,NE]

# Home page canonicalization
RewriteCond %{THE_REQUEST} ^.*\/index\.htm\ HTTP/
RewriteRule ^(.*)index\.htm$ /$1 [R=301,L,NE]

# Removed page_missing.htm
Redirect 301 /page_missing.htm /new_page.htm#section_b

# Some content moved to sub-folder
Redirect 301 /extra_content.htm /extra/extra_content.htm

# Internally redirect all HTM & HTML URLs to PHP
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*)\.(htm|html)$ /$1\.php

# Error 404 page
ErrorDocument 404 /error404.php

<IfModule mod_expires.c>
    # Activate mod_expires for this directory
    ExpiresActive on

    # Default
    ExpiresDefault "access plus 7 days"

    # Default for actual documents
    ExpiresByType text/html "access plus 15 minutes"

    # cache CSS files for 7 days
    ExpiresByType text/css "access plus 7 days"

    # locally cache common resource types for 7 days
    ExpiresByType image/jpg "access plus 7 days"
    ExpiresByType image/jpeg "access plus 7 days"
    ExpiresByType image/gif "access plus 7 days"
    ExpiresByType image/png "access plus 7 days"
    ExpiresByType application/pdf "access plus 7 days"    
    ExpiresByType audio/mpeg "access plus 7 days"    
</IfModule> 

Answer 1

# Internally redirect all HTM & HTML URLs to PHP
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*)\.(htm|html)$ /$1\.php

I wouldn't necessarily expect this to cause the problem you are experiencing, however, you have directives that "blindly" rewrites any non-existent .htm (or .html) request to the equivalent .php file, whether that .php file exists or not. (The error document should then be catching the missing .php file, not the missing .htm file that was initially requested.)

This could also explain the difference in behaviour you are seeing with requesting a non-existent "directory" (ie. a request of the form /dummy/), which would not be rewritten by the above directive and appears to "work" as intended (ie. the custom error document is called).

You could modify the above rule to only rewrite to .php if the file exists. For example:

# Internally redirect all HTM & HTML URLs to PHP
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{DOCUMENT_ROOT}/$1.php -f
RewriteRule ^(.*)\.(htm|html)$ /$1.php [L]

No need to escape the literal dot in the RewriteRule substitution. You should include the L flag (although it is currently the last mod_rewrite directive, so doesn't strictly matter).

---

UPDATE: If you request a non-existent php page then you still get the "File not found" response.

This sounds like a server-config issue. You might be able to "workaround" this problem by manually rewriting to the error document:

RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule \.php$ /error404.php [L]

Although you will probably need to modify error404.php to account for this.

OR... I would also be curious whether triggering the 404 from Apache (which, on the face of it, doesn't appear to make too much sense) would change this behaviour. For example, instead of the above rewrite:

RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule \.php$ - [R=404]

The idea behind this approach is that it will hopefully trigger an internal subrequest for the error document before the request is passed to the PHP handler (which appears to be what is conflicting with the error document). The PHP handler is then only called to serve the error document.