wget without keeping the cache

Question

I'm trying to run a remote bash script hosted on github:

wget -O - https://raw.githubusercontent.com/xxxxx/script.sh

All works good, no problems

But when I update the script above, and I run the same command, it processes the old script

I tried with --no-cache, --delete-after, without success

How to get a fresh copy of the wget everytime?

Answer 1

raw.githubusercontent.com is using a CDN, as you can inspect by doing:

$ dig raw.githubusercontent.com +short
github.map.fastly.net.
1.2.3.4

so, if that CDN is enforcing a TTL-based caching (and it seems so, by checking the Cache-Control header of 300 seconds), you are stuck with it. As you saw asking with --no-cache doesn't do anything because that's the way these are designed to prevent abuses.

You may bypass it in many hacky ways, which I won't list, but you are basically trying to circumvent a policy set forth by github and I would advise against. Moreover a method which works today might not work programmatically tomorrow or in subsequent calls.

So.. Just wait for the TTL to expire and get the new copy.

You may write a script which polls every x-seconds (don't make it too fast though) and wait until you see X-Cache: MISS - that will tell you that you got a new copy. Or that checks the ETag for changes.

Or you may check if github provides you with an API to invalidate your own code, or a way to call files with the latest commit, like: https://raw.githubusercontent.com/xxxxx/script.sh?commit=12345