0

There's plenty of options to cache requests on web servers/proxys, but i'm looking for a way to made incoming requests to "wait" until the cache is ready and then response with the cache to all the queued requests.

I'm almost sure i read about it in some Medium post but i cannot find it, so maybe some guru here can guide me on this path.

UPDATE

Thanks to @martynas-saint comment i found the post i was looking for and by consecuence a lot of resources talking about this topic, so thank you so much !

FxckDead
  • 103
  • 2

1 Answers1

0

Varnish web cache does this by default - it is called requests coalescing or collapsing. To find more about it check this - https://info.varnish-software.com/blog/request-coalescing-and-other-reasons-to-use-varnish-as-an-origin-shield

We collapse requests so your setup doesn't The big bad wolf of content distribution is something called the "thundering herd" effect triggered by a massive amount of requests arriving at the same time. Think about something like Black Friday, but with HTTP requests, the systems get flooded and just choke on the incoming traffic. It either slows down to a crawl or crashes entirely. In practice, that usually happens for big online events, such as the Olympic Games, football matches, or, well, Black Friday.

However, that kind of traffic is usually highly cacheable, and Varnish implements request coalescing (or collapsing) to save the day. The idea is dead simple: if 20 users request the same content at the same time, fetch the data only once from the origin. There are quite a few interesting technical details to it, but I won't bore you with that today, don't worry. The important point is that Varnish does all the hard work for you, transparently, so that the thundering herd doesn't trample your origin.

This describes possible code solution (Using Netflix/Hystrix which is not actively developed anymore):

https://medium.com/@jacqtrain/the-hystrix-collapser-and-how-it-might-save-your-webservices-85fb183db620

Martynas Saint
  • 1,211
  • 7
  • 15
  • 1
    This does not provide an answer to the question. To critique or request clarification from an author, leave a comment below their post. - [From Review](/review/low-quality-posts/468687) – Dave M Nov 17 '20 at 21:48
  • Thanks, improved the answer. – Martynas Saint Nov 17 '20 at 21:51
  • 1
    Adding info from the link would improve your response. Link only answers are usually closed as the links can become inactive over time – Dave M Nov 17 '20 at 22:00
  • thanks! "requests coalescing" was the term i was looking for, with that i found lots of alternatives and docs about the topic – FxckDead Nov 18 '20 at 14:38