Caching all files in varnish

Question

I want my varnish servers to cache all files. At backend there is lighttpd hosting only static files, and there is an md5 in the url in case of file change, ex. /gfx/Bird.b6e0bc2d6cbb7dfe1a52bc45dd2b05c4.swf). However my hit ratio is very poorly (about 0.18)

My config:

sub vcl_recv {
    set req.backend=default;

    ### passing health to backend
    if (req.url ~ "^/health.html$") {
        return (pass);
    }

    remove req.http.If-None-Match;
    remove req.http.cookie;
    remove req.http.authenticate;

    if (req.request == "GET") {
        return (lookup);
    }
}

sub vcl_fetch {
    ### do not cache wrong codes
    if (beresp.status == 404 || beresp.status >= 500) {
        set beresp.ttl = 0s;
    }
    remove beresp.http.Etag;
    remove beresp.http.Last-Modified;
}

sub vcl_deliver {
    set resp.http.expires = "Thu, 31 Dec 2037 23:55:55 GMT";
}

I have made an performance tuning:

DAEMON_OPTS="${DAEMON_OPTS} -p thread_pool_min=200 -p thread_pool_max=4000 -p thread_pool_add_delay=2 -p session_linger=100"

1. The main url which is missed is... /health.html. Is that forward to backend correctly configured?
2. Disabling health checking hit ratio increases to 0.45. Now mostly "/crossdomain.xml" is missed (from many domains, as it is wildcard). How can I avoid that?
3. Should I carry on other headers like User-Agent or Accept-Encoding? I thing that default hashing mechanism is using url + host/IP. Compression is used at the backend.
4. What else can improve performance?

Answer 1

More information is required to be sure (lighty response headers), but I would assume your backend is not setting an Expires nor Cache-control header.

This is required so Varnish knows how long to store the data in cache for, or you can also specify a default ttl in the DAEMON_OPTS via -t default_ttl_in_seconds.

Or indeed within the VCL itself, as answered here: Varnish Cache - default TTL?

HOWEVER: this assumes again that your backend light server is not returning an expiry in the past, and not returning a Cache-control header that prevnets caching (i.e. no-cache,must-revalidate iirc), if this is the case setting a default ttl will do nothing to alleviate the issue.