Ignoring GET parameters in Varnish VCL

Question

Okay: I've got a site set up which has some APIs we expose to developers, which are in the format

/api/item.xml?type_ids=34,35,37&region_ids=1000002,1000003&key=SOMERANDOMALPHANUM

In this URI, type_ids is always set, region_ids and key are optional.

The important thing to note is that the key variable does not affect the content of the response. It is used for internal tracking of requests so we can identify people who make slow or otherwise unwanted requests.

In Varnish, we have a VCL like this:

if (req.http.host ~ "the-site-in-question.com") {
  if (req.url ~ "^/api/.+\.xml") {
    unset req.http.cookie;
  }
}

We just strip cookies out and let the backend do the rest as far as times are concerned (this is a hackaround since Rails/authlogic sends session cookies with API responses).

At present though, any distinct developers are basically hitting different caches since &key=SOMEALPHANUM is considered as part of the Varnish hash for storage. This is obviously not a great solution and I'm trying to work out how to tell Varnish to ignore that part of the URI.

Answer 1

If you need to leave the key=SOMEALPHANUM intact to the backend in case not delivered from cache, then it might be better to regsub within the vcl_hash function instead, as this won't really alter the url but instead it just alters the hash of the key.

sub vcl_hash {
  if(req.http.host ~ "the-site-in-question.com" & req.url ~ "^/api/") {
    set req.http.X-Sanitized-URL = req.url;
    set req.http.X-Sanitized-URL = regsub(req.http.X-Sanitized-URL, "&key=[A-Za-z0-9]+", "");
    set req.hash += req.http.X-Sanitized-URL;
  } else {
    set req.hash += req.url;
  }
  set req.hash += req.http.host;
  hash;
}

Answer 2

in vcl_recv

set req.url = regsub(req.url, "&key=.*$", "");

Answer 3

Similar to the answer by cd34, but taking into account the possibility of different orders for the query parameters and relying on the fact that the problem defines the value as alphanumeric:

set req.url = regsub(req.url, "&key=[A-Za-z0-9]*", ""); 

(I can't comment yet, otherwise this would be a comment on cd34's answer)