403 forbidden on Apache after trying to install nginx reverse proxy

Question

My setup:

Digital Ocean droplet running Debian 8

2 websites with each their domain running on Apache2

Tried installing nginx and configure it as a reverse proxy following these instructions: https://www.digitalocean.com/community/tutorials/how-to-configure-nginx-as-a-web-server-and-reverse-proxy-for-apache-on-one-ubuntu-14-04-droplet

It instantly broke my sites, giving Forbidden 403 error when trying to access them.

I spent so many hours trying to make it work and now decided to leave it and just use Apache2 like i did before.

But now the sites are still showing Forbidden 403 even after nginx is stopped. Briefly installed lighttpd + lighttpd php5-cgi and then i could access the sites, however, it was showing just 1 site on both domains.

I have chown -R www-data:www-data /var/www

Also did a chmod -R 755 /var/www

Please, if anyone could provide some input, I would be so happy. I am going crazy trying to fix this mess. :(

Apache ports.conf:

Listen 80

<IfModule ssl_module>
        Listen 443
</IfModule>

<IfModule mod_gnutls.c>
        Listen 443
</IfModule>

Sample from Apache error log:

[Thu Mar 03 13:56:36.965194 2016] [authz_core:error] [pid 31517] [client 185.106.92.253:55470] AH01630: client denied by server configuration: /var/www/html/xmlrpc.php
[Thu Mar 03 13:56:43.316074 2016] [authz_core:error] [pid 31518] [client 185.106.92.253:52484] AH01630: client denied by server configuration: /var/www/html/xmlrpc.php
[Thu Mar 03 13:56:47.635774 2016] [authz_core:error] [pid 31496] [client 185.106.92.253:53967] AH01630: client denied by server configuration: /var/www/html/xmlrpc.php
[Thu Mar 03 13:57:00.853631 2016] [authz_core:error] [pid 31670] [client 185.106.92.253:50494] AH01630: client denied by server configuration: /var/www/html/xmlrpc.php
[Thu Mar 03 13:57:08.455024 2016] [authz_core:error] [pid 31668] [client 185.106.92.253:45464] AH01630: client denied by server configuration: /var/www/html/xmlrpc.php
[Thu Mar 03 13:57:21.641599 2016] [authz_core:error] [pid 31517] [client 185.106.92.253:38106] AH01630: client denied by server configuration: /var/www/html/xmlrpc.php
[Thu Mar 03 13:57:28.132631 2016] [authz_core:error] [pid 31518] [client 185.106.92.253:48468] AH01630: client denied by server configuration: /var/www/html/xmlrpc.php

apache2.conf:

Mutex file:${APACHE_LOCK_DIR} default

PidFile ${APACHE_PID_FILE}

Timeout 300

KeepAlive On

MaxKeepAliveRequests 100

KeepAliveTimeout 100

User ${APACHE_RUN_USER}
Group ${APACHE_RUN_GROUP}

HostnameLookups Off

ErrorLog ${APACHE_LOG_DIR}/error.log

LogLevel warn

IncludeOptional mods-enabled/*.load
IncludeOptional mods-enabled/*.conf

Include ports.conf

<Directory />
    Options FollowSymLinks
    AllowOverride None
    Require all denied
</Directory>

<Directory /usr/share>
    AllowOverride None
    Require all granted
</Directory>

<Directory /var/www/site1>
    Options Indexes FollowSymLinks
    AllowOverride All
    Require all granted
</Directory>

<Directory /var/www/site2>
        Options Indexes FollowSymLinks
        AllowOverride All
        Require all granted
</Directory>


<Directory /srv/>
    Options Indexes FollowSymLinks
    AllowOverride None
    Require all granted
</Directory>

AccessFileName .htaccess

<FilesMatch "^\.ht">
    Require all denied
</FilesMatch>

LogFormat "%v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" vhost_combined
LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %O" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent

site1.conf:

<VirtualHost *:80>
    ServerName www.site1.com
    ServerAlias site1.com

    ServerAdmin webmaster@localhost
    DocumentRoot /var/www/site1

    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined

    <Directory /var/www/site1/>
        Options FollowSymlinks
        AllowOverride none
        Require all granted
    </Directory>

        <IfModule mod_fastcgi.c>
                AddHandler php5-fcgi .php
                Action php5-fcgi /php5-fcgi
                Alias /php5-fcgi /usr/lib/cgi-bin/php5-fcgi
                FastCgiExternalServer /usr/lib/cgi-bin/php5-fcgi -socket /tmp/php5-fpm.sock -pass-header Authorization
        </IfModule>

</VirtualHost>

SetOutputFilter DEFLATE
SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|ico|png)$ \ no-gzip dont-vary
SetEnvIfNoCase Request_URI \.(?:exe|t?gz|zip|bz2|sit|rar)$ \no-gzip dont-vary
SetEnvIfNoCase Request_URI \.pdf$ no-gzip dont-vary

BrowserMatch ^Mozilla/4 gzip-only-text/html
BrowserMatch ^Mozilla/4\.0[678] no-gzip
BrowserMatch \bMSIE !no-gzip !gzip-only-text/html

site2.conf:

<VirtualHost *:80>
    ServerName www.site2.com
    ServerAlias site2.com

    ServerAdmin webmaster@localhost
    DocumentRoot /var/www/site2

    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined

    <Directory /var/www/site2/>
        Options FollowSymlinks
        AllowOverride none
        Require all granted
    </Directory>

        <IfModule mod_fastcgi.c>
                AddHandler php5-fcgi .php
                Action php5-fcgi /php5-fcgi
                Alias /php5-fcgi /usr/lib/cgi-bin/php5-fcgi
                FastCgiExternalServer /usr/lib/cgi-bin/php5-fcgi -socket /tmp/php5-fpm.sock -pass-header Authorization
        </IfModule>

</VirtualHost>

SetOutputFilter DEFLATE
SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|ico|png)$ \ no-gzip dont-vary
SetEnvIfNoCase Request_URI \.(?:exe|t?gz|zip|bz2|sit|rar)$ \no-gzip dont-vary
SetEnvIfNoCase Request_URI \.pdf$ no-gzip dont-vary

BrowserMatch ^Mozilla/4 gzip-only-text/html
BrowserMatch ^Mozilla/4\.0[678] no-gzip
BrowserMatch \bMSIE !no-gzip !gzip-only-text/html

Answer 1

Too long for a comment but probably not a complete answer:

It is odd that you get errors on the file-system path /var/www/html/ which is outside of the DocumentRoot paths /var/www/site1 and /var/www/site2.

That implies that there may be another configuration file in play. You may want to check what these Include directives are loading:

 IncludeOptional mods-enabled/*.load
 IncludeOptional mods-enabled/*.conf

and if there are any .htaccess files to want to source content from /var/www/html.

In the error message below:

[Thu Mar 03 13:57:28.132631 2016] [authz_core:error] [pid 31518] [client 185.106.92.253:48468] AH01630: client denied by server configuration: /var/www/html/xmlrpc.php

you see that it gets generated by the module: authz_core which indicates that it has to do with the access controls that are governed by that module; the manual lists quite a few.

It is this server configuration that denies access by the way:

<Directory />
    Options FollowSymLinks
    AllowOverride None
    Require all denied
</Directory>

a sensible default access policy which denies access unless specifically allowed, but you don't allow anything for the path /var/www/html ; you have only specifically allow access to: /usr/share /var/www/site1 /var/www/site2 & /srv.

---

EDIT Hmmm /var/www/html may be the compile time default DocumentRoot.

And from your hapache2.conf it seems that an IncludeOptional sites-enabled/*.conf or similar is missing.

That would explain why you get directed to /var/www/html and fail

Answer 2

Thanks to @HBruijn for the input. Means a lot!

Now my sites are somewhat running again. One is showing 500 Internal error, but the other is good. :) That I can manage!

I changed /etc/apache2/mods-available/fastcgi.conf to the following which solved the issues:

<IfModule mod_fastcgi.c> 
  AddHandler php5-fcgi .php 
  Action php5-fcgi /php5-fcgi 
  Alias /php5-fcgi /usr/lib/cgi-bin/php5-fcgi_example.com 
  FastCgiExternalServer /usr/lib/cgi-bin/php5-fcgi_example.com -socket /var/run/php5-fpm_example.com.sock -pass-header Authorization 
  <Directory /usr/lib/cgi-bin> 
      Require all granted 
  </Directory> 
</IfModule>