1

I currently analyze varnish hit rate like this

enter image description here

While this is good, a major part of the request the server services is non-cacheable. This brings down the hit rate drastically.

enter image description here

How can I modify this such that I count only requests that are candidates for caching and calculate the rate? Which varnishstat metric should I be looking?

Quintin Par
  • 4,293
  • 10
  • 46
  • 72

1 Answers1

0

It seems like the stats are accurate, but you want some URLs not to count as misses.

It depends what you mean by requests that are not not cacheable. If there is a URL pattern where you know every URL request will be unique, then why even lookup that URL or enter the backend's response in the cache?

So exclude the URLs you want to discount in a Varnish configuration file like sitename.vcl:

sub vcl_recv {
    # Do not cache following pages
    if (req.url ~ "^/monitor.*\?heartbeat=") { 
        return (pass); 
    }
}

For more information read man vcl or have a look at the few examples on the Varnish wiki.

Otherwise are they POSTs (automatically not cached), or are they excluded in your .vcl file?

This is not a stats problem. If your vcl_recv() returns a pass or an error, then it shouldn't count as a lookup or a miss (see flowchart here), and your existing calculation is already representative.

From varnishstat -l, the varnish docs, and from what I see that is able to be graphed in Munin, there is no relevant counter that does anything significantly different from cache_hit or cache_miss. 

  s_req = client_req = (cache_hit + s_fetch) 
  s_fetch = cache_miss + s_pass

So 1 - (s_fetch / s_req) would look like a worse hit rate than you already have, counting passed or 'uncacheable' pages as misses if it doesn't already.

(Incidentally, if you change the backend for the 'uncacheable' page, then I don't think Varnish has counts for requests per backend.)

Silly idea: you could run logtail varnishncsa.log | grep -c "\(POST\|whateverURLisnoncacheable\)" and subtract the result from your total of misses, but it's a lot of work to probably produce less accurate results.

Cedric Knight
  • 1,098
  • 6
  • 20
  • There are a lot of heartbeat requests that update status. – Quintin Par Aug 30 '17 at 15:38
  • So these are unique URLs that go through default Varnish processing, are looked up and count as misses? Assuming the heartbeat URLs can be distinguished, why not configure a .vcf file to pass them? That way they don't count as misses, are returned a little more quickly and don't use up cache space either. – Cedric Knight Aug 30 '17 at 18:35
  • Aah, how do I do that? – Quintin Par Aug 30 '17 at 18:49
  • Something like `if (req.url ~ "^/monitor.*\?heartbeat=") { return (pass); }` Here's an example https://stackoverflow.com/questions/13964351/excluding-a-specific-url-from-varnish-cache What are typical URLs you don't want to count or cache? – Cedric Knight Aug 30 '17 at 19:35