Varnish not working with last-modified

Question

I'm new to Varnish and to reverse proxies in general. I'm testing Varnish in an application that works like a blog where I have a page with an "updated_at" that changes once the page is updated. As a consequence, I decided that my cache strategy should be based on the last-modified header. The reason for that is the user wants to see the change immediately once the page is modified in the backoffice.

In addition to that, I understood that I could add a s-maxage with must-revalidate to cache-control so if the last-modified didn't change Varnish would not even try to make a new request.

I also added an expire for the browser so there will be a client cache. As a result, the browser will not even ask varnish to do a request to Apache again.

However, I noticed that the cache-control s-maxage has priority over the last-modified even with must-revalidate. Moreover, even if I don't add a cache-control, 'last-modified' doesn't work. Varnish use its default ttl and nothing happens. What am I doing wrong?

I'm using Symfony and these are the headers:

$response = new Response();
$response->setPublic();

// expiration model for the browser cache (EXPIRE)
// the browser will only make a new request to Varnish again after ten minutes (600 seconds)
$date = new \DateTime();
$date->modify('+600 seconds');
$response->setExpires($date);

// expiration model for Varnish (CACHE-CONTROL)
// Varnish will only make a new request to Apache again after one hour (3600 seconds)
// The must-revalidate tells Varnish to do this request before serving the files after one hour (it's already de default behaviour)
// cache-control has priority over expire
//$response->setSharedMaxAge(3600);
$response->headers->addCacheControlDirective('must-revalidate', true);

// validation model for varnish (LAST-MODIFIED)
// Varnish will only make a new request to Apache again if the updated_at of the page changed even if the expiration time is ended
$response->setLastModified($page->getUpdatedAt());

// if the response didn't change, stop here
if ($response->isNotModified($request)) {
    return $response;
}

Here are my headers the first time when I have a MISS and the second time when I have a HIT. Then, it doesn't matter if the last-modified changes the headers are always the same as this second one. (My browser cacher is disabled so I have 200 instead of 304)

first request

second request

Answer 1

There is not such thing as "Last Modified doesn't work". This header simply indicates the date resource was last modified. The clients may issue "if not modified since" requests afterwards, but this is not important to your situation.

It is natural that subsequent requests will return exactly the same headers. Objects in Varnish are cached including the headers information. So when Varnish extracts a cached entry, it will return exact copy of it.

Understanding it

You are dealing with two caches at the same time: Varnish and browser cache. You have only one set of HTTP headers per request. You can't specify one HTTP Expires for browser and then another just for Varnish.

So if you want to cache things differently (and you do), one solution is adjusting beresp.ttl in Varnish to amount of time during which you want to relax the PHP backend, i.e.: 24 hrs.

Somewhere in VCL:

sub vcl_backend_response {
    # A TTL of 24h
    # set beresp.ttl = 24h;
}

Then, considering that you keep your existing PHP code, Varnish caches the first request to a page for 24 hours and makes it cacheable by browsers for 10 minutes.

Once the article is updated, you have to purge Varnish cache, aka Cache Invalidation. See some insight on this page (scroll to the bottom).

The reason for that is the user wants to see the change immediately once the page is modified in the backoffice.

And yet you want them to have 10 minutes browser cache :)