What is the best tool to aggregate traffic stats from multiple nginx servers?

Question

The setup:

2 or more nginx machines
each machine has the same virtual hosts
traffic is load balanced via DNS to each machine

I need to figure out what are the best tools to use to get some traffic stats, mostly interested in amount of hits and total traffic in gigabytes. Obviously, the log information will come from nginx, formatted like this:

  log_format  main  '$remote_addr $host $remote_user [$time_local] "$request" '
                    '$status $body_bytes_sent "$http_referer" "$http_user_agent" "$gzip_ratio"';

score 1 · Answer 1 · answered May 04 '10 at 01:13

1

awstats is definitely the tool for the job.

answered May 04 '10 at 01:13

Marco Ramos

3,100
22
25

1

awstats includes logmerge.pl which will merge and timesort the log before handing it to awstats. LogFile="/usr/share/doc/awstats/examples/logresolvemerge.pl /gfs/weblogs/c1ws*-vcommon-access.log.1|" – user6738237482 Jun 17 '10 at 07:59

score 0 · Answer 2 · answered Mar 12 '10 at 14:26

0

rsync logs (or share nfs if you're feeling brave) to one location then use awstats to analyse.

answered Mar 12 '10 at 14:26

rytis

2,324
1
18
13

score 0 · Answer 3 · answered Jun 26 '10 at 03:44

Regardless of which analytics program you intent to use, logresolvemerge.pl is almost certainly going to be your winner for merging the logs themselves. In addition to Marco's listed syntax for awstats, it works like a champ as a standalone program along the lines of:

/path/to/logresolvemerge.pl /path/to/one/big/logfile /path/to/many/files/ending/in/*.log > /path/to/merged/log.file

What is the best tool to aggregate traffic stats from multiple nginx servers?

3 Answers3