How do I separate Apache server log entries for a domain that is different than my server's?

Question

I'm using the following version of Apache ...

httpd -v
Server version: Apache/2.4.6 (CentOS)
Server built:   Apr 24 2019 13:45:48

I have a lot of entries in my /var/log/httpd/access_log file that are for domains that are not my domain ...

79.143.191.42 - - [17/Dec/2019:10:36:30 -0500] "GET http://www.guenstiger.de/Produkt/Prada/Infusion_d_Homme_Deodorant_100_ml.html HTTP/1.1" 400 64146 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/536.26.17 (KHTML like Gecko) Version/6.0.2 Safari/536.26.17"

I'm not sure how these entries are ending up in my log file, since the domain does not resolve to my server, but is there are any way to funnel these requests into a separate log file?

Edit: Here is my VirtualHost configuration (done after suggestion from answer) ...

<VirtualHost *:80>
    CustomLog /var/log/httpd/garbage_access_log common
</VirtualHost>
<VirtualHost *:80>
    ServerName mydomein.com

    Alias /static /var/www/html/myproject/static
    <Directory /var/www/html/myproject/static>
        Require all granted
    </Directory>

    # Next, add the following directory block
    <Directory /var/www/html/myproject>
        <Files wsgi.py>
            Require all granted
        </Files>
    </Directory>

    WSGIDaemonProcess myproject python-home=/var/www/html/venv python-path=/var/www/html
    WSGIProcessGroup myproject
    WSGIScriptAlias / /var/www/html/myproject_project/wsgi.py

</VirtualHost>

Edit: In response to the suggestion given, I tried this

<If "%{HTTP_HOST} == 'mydomein.com'">
    CustomLog /var/log/httpd/access_log common
<Else>
    CustomLog /var/log/httpd/other_access_log common
    ErrorDocument 421 "%{HTTP_HOST} is not available here"
</If>

which resulted in the error ...

Expected </Else> but saw </If>

and so I tried this ...

<If "%{HTTP_HOST} == 'mydomein.com'">
    CustomLog /var/log/httpd/access_log common    # Error on this line
</If>
<Else>
    CustomLog /var/log/httpd/other_access_log common
    ErrorDocument 421 "%{HTTP_HOST} is not available here"
</Else>

but this resulted in the error

CustomLog not allowed here

Mathias Weidner · Answer 1 · 2019-12-18T17:33:33.693

1

These requests look like someone's trying to use your webserver as a proxy. The error code 400 means that they didn't succeed.

If these requests don't come with a Host: ... header, you could use one default VirtualHost to catch such requests and other VirtualHost environments using ServerName for your real website.

Then you can use different log files for the different environments by putting the Log statement inside the VirtualHost directive.

It should look similar to this:

<VirtualHost *:80>
  ...
  CustomLog /var/www/domains/default/apache.access.log common
</VirtualHost>

<VirtualHost *:80>
  ServerName mydomein.com
  ...
  CustomLog /var/www/domains/mydomein.com/apache.access.log common
</VirtualHost>

edited Dec 18 '19 at 17:33

answered Dec 17 '19 at 18:37

Mathias Weidner

417
3
10

I added my VirtualHost configuration as an edit to my question. Your comment, "Then you can use different log files for the different environments," is a good restatement of my question. THat's what I'm asking -- how do I do that? – Dave Dec 18 '19 at 17:07
You want to use more than one VirtualHost statement, the first one to be the default. Then you place the Log statement inside the VirtualHost to use different files for different VirtualHost. I've added some links that hopefully explaint it better. – Mathias Weidner Dec 18 '19 at 17:26
Thanks! I'll give that a whirl – Dave Dec 18 '19 at 17:51
so I created my configuration per your suggestion (included in my question), and although the unneeded requests go to the garbage log file, which I want, the requests that I do want never actually hit my domain. Now I get a 404 for every page I request from my domain. – Dave Dec 18 '19 at 20:40
I don't see a CustomLog statement for the second VirtualHost in your posted configuration. What are your error log files say? HTTP status code 404 means file not found. You probably need DocumentRoot statements for all VirtualHost environments. – Mathias Weidner Dec 19 '19 at 12:22
For the second VirtualHost block, I just want it to go to the normal access_log -- do I need to specify a CustomLog statement for that? – Dave Dec 19 '19 at 18:23
At the moment I don't know if you need to specify a CostomLog statement for the second VirtualHost block. It was just a suggestion from me. Did you try it? I also don't know what's in your error log. Did you look into it? – Mathias Weidner Dec 20 '19 at 11:45

score 0 · Answer 2 · edited Jun 11 '20 at 10:02

0

Regarding CustomLog directive's Syntax and Custom Logging:

Conditional logging is provided so that individual requests may be included or excluded from the logs based on characteristics of the request.

"characteristics of the requests" includes the Host header

Syntax: CustomLog file|pipe format|nickname [env=[!]environment-variable| expr=expression])

The third argument is optional and controls whether or not to log a particular request... The condition can be expressed as arbitrary boolean expression. If the condition is not satisfied, the request will not be logged.

It seems that the simplest solution would be to add an expression to the end of your CustomLog, like so:

<VirtualHost *:80>
    ServerName mydomein.com
    
    # Use the Host header to determine where to log
    # Use the Host header to determine how to respond. 
    CustomLog /var/log/httpd/access_log common "expr=%{HTTP_HOST} == 'mydomein.com'"
    CustomLog /var/log/httpd/garbage_access_log common "expr=%{HTTP_HOST} != 'mydomein.com'"

    Alias /static /var/www/html/myproject/static
    <Directory /var/www/html/myproject/static>
        Require all granted
    </Directory>

    # Next, add the following directory block
    <Directory /var/www/html/myproject>
        <Files wsgi.py>
            Require all granted
        </Files>
    </Directory>

    WSGIDaemonProcess myproject python-home=/var/www/html/venv python-path=/var/www/html
    WSGIProcessGroup myproject
    WSGIScriptAlias / /var/www/html/myproject_project/wsgi.py

</VirtualHost>

To sum up, this maintains your current functionality and will log to the appropriate file

Good luck and I hope this helps.

edited Jun 11 '20 at 10:02

Community

1

answered Jan 03 '20 at 16:19

mikey

51
1
8

1

Is there any way to configure things such that that I don't have to hard-code "ServerName bogus.domain.de" -- I just have a generic catch all for anything that's not my primary domain? I'd like to have two access logs -- one for my primary domain and one for everything else. – Dave Jan 03 '20 at 17:56
Hi Dave, that's a great question; I agree that using a bogus domain is unintuitive and less than desirable. As such I've edited my answer to give some reasoning on my initial suggestion, a few more approaches, more links and what I think is an even simpler and more reasonable solution – mikey Jan 04 '20 at 04:14
Thanks for your updated response. I tried a couple of variations on your expanded answer (included as an edit to my question) and included the errors that result. Ultimately, it seems like the config doesn't like the placement of the CustomLog statement. – Dave Jan 04 '20 at 18:49
Hey Dave, thanks for giving it a go and I'm sorry that it didn't work. I was sure that it was a solid solution. Try out the "Conditional Logging" example a go: https://httpd.apache.org/docs/2.4/expr.html#examples – mikey Jan 05 '20 at 00:40
Hey @Dave, I've updated the answer show how the conditional logging example would look with a single virtual host. I removed the ErrorDocument directive; it's probably not required, as your server already responds with a 400. – mikey Jan 05 '20 at 08:03
1

Thx @mikey. A quick update as I just tried the updated solution right now. Got a "AH00526: Syntax error" complaining about "error in condition clause" for the line 'CustomLog /var/log/httpd/access_log common "expr %{HTTP_HOST} == 'mydomein.com'"'. – Dave Jan 06 '20 at 15:44
Hey @Dave, no worries! Sorry about that. I looked back over the documentation and there was a missing '=' after 'expr'. That means the expression should read: "expr=%{HTTP_HOST} == 'mydomein.com'". I've updated the answer to reflect this. – mikey Jan 12 '20 at 22:48
@Dave how did it go? – mikey Jan 27 '20 at 23:34

How do I separate Apache server log entries for a domain that is different than my server's?

2 Answers2