Nginx caching reverse proxy with URL rewrite

Question

I am serving large static files from a backend storage server with slow spinning disks. This server is hidden behind fast nginx reverse proxy with local cache on SSD. It works great and fast.

Now I want to change storage backend, and as a result I cannot maintain same location of stored files on a backend. Instead of root of the server, they will have to be served from a subdirectory. How can I modify nginx reverse proxy config so that it proxies all non-cached requests to backend to a subdirectory, and clients are not aware that anything has changed?

I cannot do anything on storage server to maintain old URL scheme, so I have to do it on a frontend. No 301/302 headers are supposed to be passed anywhere.

So currently I have:

Client see: https://frontend.com/file.txt
Nginx fetches non-cached files from: https://backend.com/file.txt

What I want to achieve:

Client see: https://frontend.com/file.txt (no observable changes to a client)
Nginx fetches non-cached files from: https://backend.com/directory/file.txt

I have tried many dozens of configurations, without luck. When I am trying this configuration - instead of silently fetching data from different URL, it ends up in an infinite loop of adding test via regexp.

    location / {
        rewrite /(.*) /test/$1  break;
        proxy_pass http://f002.backblazeb2.com;
        proxy_redirect     off;
        proxy_set_header   Host $host;

        proxy_cache_use_stale error timeout http_500 http_502 http_503 http_504;
        proxy_cache_revalidate on;
        proxy_read_timeout     2;
        proxy_connect_timeout  3;

        proxy_http_version 1.1;
        proxy_set_header Connection "";

        proxy_cache_valid 200 302 60s;
        proxy_cache_valid 404      1m;

        limit_conn perip 23;
        limit_req zone=dynamic burst=60;

        expires 24h;
    }

score 1 · Accepted Answer · answered Nov 10 '19 at 20:00

The following worked:

server {
listen 80;
listen      [::]:80;

server_name  blablabla.com;
proxy_cache one;

listen 443 ssl http2;
listen [::]:443 ssl http2;

ssl_certificate ....;
ssl_certificate_key ....;

location / {
   proxy_ignore_headers X-Accel-Expires Expires Cache-Control Set-Cookie;

   proxy_pass https://f002.backblazeb2.com/file/some-directory/;##$request_uri appended automatically

   [...]
}

score 0 · Answer 2 · answered Nov 03 '18 at 14:58

The following should work:

location / {
    proxy_pass http://f002.backblazeb2.com/test$request_uri;
    proxy_redirect     off;
    proxy_set_header   Host $host;

    proxy_cache_use_stale error timeout http_500 http_502 http_503 http_504;
    proxy_cache_revalidate on;
    proxy_read_timeout     2;
    proxy_connect_timeout  3;

    proxy_http_version 1.1;
    proxy_set_header Connection "";

    proxy_cache_valid 200 302 60s;
    proxy_cache_valid 404      1m;

    limit_conn perip 23;
    limit_req zone=dynamic burst=60;

    expires 24h;
}

This gives me unlimited redirect to the same server with 301 code... And it really puzzles me. it keeps going adding test againg and again with 301: HTTP/2.0 301 Moved Permanently location: https://s3.zeptobars.com/test/test/test/test/test/Asdfsdfsdffsfd.jpg — BarsMonster, Nov 03 '18 at 16:13

Nginx caching reverse proxy with URL rewrite

2 Answers2