1

We are running an Apache/Tomcat/Railo (Coldfusion) web server stack. We are looking at ways to monitor the average page load times for all web servers as well as look at the page load times for specific pages in our application. If possible I would like to do this via the Apache server logs.

Any thoughts on software or approaches I can use? (I am fairly new to the Web Server Monitoring World).

Thanks!

kilmore
  • 11
  • 1

2 Answers2

2

in your apache httpd.conf, track down the <IfModule log_config_module> section.

Add %D to the common logformat:

LogFormat "%h %l %u %t \"%r\" %>s %b %D" common

%D is time taken to serve the page in Microseconds

beware that a number of tools cannot parse the apache logs when they have extra columns. It's probably safer to add 

LogFormat "%h,%l,%u,%t,\"%r\",%>s,%b,%D" performance
CustomLog "logs/performance.log" performance

to create a second log in CSV format. From there, I normally use LogParser (very old but very useful) to analyse the data. You can also add %{CFID}C,%{CFTOKEN}C to your LogFormat to track individual sessions around your site, which can be very handy

barnyr
  • 226
  • 1
  • 2
  • 5
  • 2
    Note that `%D` is essentially "time to last byte". That means that various page sizes, end-users internet connection speed and potentially aborted connections will all skew results. – Mxx Jan 22 '14 at 17:08
  • Very good point @Mxx ! – barnyr Jan 22 '14 at 17:29
  • Thank you for the suggestion on the additions to the LogFile for Apache, that will be very helpful. I am striking out on finding software that will consolidate the logs from various servers and let me pull reports or do analysis for the entire environment. I am not necessarily looking for an opensource or free option, just something that will let me look at the web servers as a whole. If this isn't the best way to look at the overall health of the web servers and the website that they publish I am totally open minded. This is my first time managing something like this. – kilmore Jan 22 '14 at 17:52
1

Perhaps what you should look into is something called RUM(Real User Monitoring) and APM(Application Performance Management/Monitoring). When it comes to the web, the concept behind such a solution is javascript code is loaded/injected into your pages and it then reports back real user's experience of your site/application. Better yet, if your users are using decently up-to-date browsers then you can also get even more detailed performance metrics from their browsers' navigation timing api.

There are numerous 3rd party services and commercial and open-source products that provide RUM/APM.

Some of the better solutions can even combine RUM and APM and allow you to correlate end-user's experience with what's been happening on your servers at that point in time.

Mxx
  • 2,312
  • 2
  • 26
  • 40
  • Thank you so much for the suggestions. I am going to try out LogParser and see if that will work. On the RUM/APM front I was hoping for something that didn't require code changes. The two RUM solutions I looked at required minor changes to the existing website code. My hope is to find a solution where I can compile/consolidate logs to a single location and run reports. This would bypass the need to add this code to our current (and backed up) development life cycle. Hopefully this is plan is not a pipe dream. I am okay with solutions that require licensing. – kilmore Jan 23 '14 at 01:04
  • @kilmore if NewRelic is compatible with your platform stack, it can work without change to application code..at least its PHP agent works that way. Also take a look at Splunk and logstash. – Mxx Jan 23 '14 at 04:38