Separate Nginx access log file for certain requests only

Question

As far as I can see Nginx supports by default 2 log files: error_log (tracks issues related to the Nginx server itself) and access_log (tracks requests processed by Nginx). Whilst it is possible to control the format of access_log with the log_format directive, I have been unsuccessful at finding a way of logging only certain requests to a separate file, and therefore would like to ask the question on SF as a reference for future readers:

Is there a way to log certain requests to a different log file than the one defined by access_log?

FYI the reason behind this question is that I have a rule that denies access to unwanted crawlers with a 200 (because 403 would give them a hint that they're being blocked), and filtering those requests out of the access_log becomes more difficult.

nginx 1.7.0+ allows using an if condition in access_log directive itself. Check http://stackoverflow.com/a/25852578/2208271 — Sithsu, Sep 15 '14 at 16:30

Max · Accepted Answer · 2012-06-01T13:44:14.593

9

cjc put me on the right track. Using access_log in an if statement by itself is not possible (You get a nginx: [emerg] "access_log" directive is not allowed here error). So the workaround is as follows:

if ($http_user_agent ~* (crawler) ) {
  set $crawler 'yes';
}
location ~ .* {
  if ($crawler = 'yes') {
    access_log /var/log/nginx/blockedbots.log;
    return 200;
    }
}

edited Jun 01 '12 at 13:44

answered Jun 01 '12 at 13:23

Max

3,373
15
51
71

Um, isn't this an access log inside an if statement..? – Xeoncross Jan 23 '15 at 18:28
Yep, the [doc says that](http://nginx.org/en/docs/http/ngx_http_log_module.html) you can use `Context: http, server, location, **if in location**, limit_except` – Xeoncross Jan 23 '15 at 18:29

score 6 · Answer 2 · answered Jul 15 '15 at 14:58

6

access_log supports if:

(access_log path [format [buffer=size [flush=time]] [if=condition]];)

access_log /var/.... if $crawler;

Source:

http://nginx.org/en/docs/http/ngx_http_log_module.html

answered Jul 15 '15 at 14:58

Thomas Decaux

1,239
11
13

This is the cleaner solution as of `NGINX 1.7.0`. – Oliver Sep 25 '17 at 07:37

score 4 · Answer 3 · answered Jun 01 '12 at 12:29

4

You should be able to put an access_log directive inside an if block, according to the documentation:

http://wiki.nginx.org/HttpLogModule

So, you should be able to do something like:

if ($http_user_agent ~* (crawler) ) {
   access_log /path/to/other/log/file ;
}

answered Jun 01 '12 at 12:29

cjc

24,533
2
49
69

`access_log` inside `if` statement by itself isn't possible, see my answer below. – Max Jun 01 '12 at 13:43

score 0 · Answer 4 · answered Mar 30 '22 at 14:19

When I came upon this post I was looking for a way to send AWS ELB healthcheck logs to a separate file as they were clogging up access.log. I was never able to get access_log to work with if inside a location block, and this does not work either:

map $http_user_agent $healthcheck {
    "ELB-HealthChecker/2.0" 1;
    default 0;
  }

access_log /var/log/nginx/elb-healthcheck.log main if=$healthcheck;
access_log /var/log/nginx/access.log main if=!$healthcheck;

In a sea of braces it is easy to forget that the NGINX configuration file is not a programming language.

What I ended up with, albeit clunky looking, is this:

map $http_user_agent $not_elb {
    "ELB-HealthChecker/2.0" 0;
    default 1;
}
map $http_user_agent $elb {
    "ELB-HealthChecker/2.0" 1;
    default 0;
}

access_log /var/log/nginx/access.log main if=$not_elb;
access_log /var/log/nginx/elb-healthcheck.log main if=$elb;

Regular traffic goes to access.log and ELB healthcheck logs go to elb-healthcheck.log.

Separate Nginx access log file for certain requests only

4 Answers4

Linked