Cant connect to S3 with nginx

Question

im trying to serve static and media files from amazon S3 bucket but nginx cant connect to it

Here is the error i get

<Error>
<Code>AccessDenied</Code>
<Message>
AWS authentication requires a valid Date or x-amz-date header
</Message>
<RequestId></RequestId>
<HostId></HostId>
</Error>

and my Nginx configuration

server {
listen          80;
server_name     my_elastic_ip;

location = /favicon.ico { access_log off; log_not_found off; }

location / {
    try_files $uri @s3;
}

location @s3 {
    set $s3_bucket          'my_bucket.s3.amazonaws.com';
    set $url_full           '$1';
    set $aws_access_key     'my_access_key';
    set $aws_secret_key     'my_secret_key';

    proxy_http_version      1.1;
    proxy_set_header        Host $s3_bucket;
    proxy_set_header        x-amz-date $date_gmt;
    proxy_set_header        Authorization 'AWS $aws_access_key:$aws_secret_key';
    proxy_hide_header       x-amz-id-2;
    proxy_hide_header       x-amz-request-id;
    proxy_hide_header       Set-Cookie;
    proxy_ignore_headers    "Set-Cookie";
    proxy_buffering         off;
    proxy_intercept_errors  on;

    resolver                8.8.4.4 8.8.8.8 valid=300s;
    resolver_timeout        10s;
    proxy_pass              http://$s3_bucket/$url_full;
}
}

Edit 1 :

I have solved the probleme replacing the x-amz-date by :

set_by_lua $now         "return ngx.cookie_time(ngx.time())";
proxy_set_header        x-amz-date $now;

You need extra nginx packages, install it with :

sudo apt-get install nginx-extras

now i get this error :

<Error>
<Code>SignatureDoesNotMatch</Code>
<Message>
The request signature we calculated does not match the signature you provided. Check your key and signing method.
</Message>

Edit 2:

To create signature i have added set-misc-nginx-module (https://github.com/openresty/set-misc-nginx-module#installation) to nginx (install nginx optional module)

Then update my nginx configuration to :

server {
listen          80;
server_name     my_ip;

location = /favicon.ico { access_log off; log_not_found off; }

location / {
    try_files $uri @s3;
}

location @s3 {
    set $s3_bucket          'my_bucket';
    set $key                'my_file';
    set $aws_access_key     'my_access_key';
    set $aws_secret_key     'my_secret_key';

    set_by_lua $now         "return ngx.cookie_time(ngx.time())";
    set $aws_signature      '';
    set $string_to_sign     "$request_method\n\n\n\nx-amz-date:$now\n/$s3_bucket/$key";
    set_hmac_sha1           $aws_signature $aws_secret_key $string_to_sign;
    set_encode_base64       $aws_signature $aws_signature;

    proxy_http_version      1.1;
    proxy_set_header        x-amz-date $now;
    proxy_set_header        Authorization 'AWS $aws_access_key:$aws_signature';
    proxy_set_header        Host $s3_bucket.s3.amazonaws.com;
    proxy_hide_header       x-amz-id-2;
    proxy_hide_header       x-amz-request-id;
    proxy_hide_header       Set-Cookie;
    proxy_ignore_headers    "Set-Cookie";
    proxy_buffering         off;
    proxy_intercept_errors  on;

    resolver                8.8.4.4 8.8.8.8 valid=300s;
    resolver_timeout        10s;
    proxy_pass              http://s3.amazonaws.com;
}

}

getting this error :

Status: HTTP/1.1 403 Forbidden

<Code>AccessDenied</Code>
<Message>Access Denied</Message>

So what's `$date_gmt` set to? Is it Amazon's required format? — ceejayoz, Oct 30 '15 at 17:38
Unrelated, but `Authorization 'AWS $aws_access_key:$aws_secret_key'` is also incorrect. You need a signature here -- you never send your secret key *anywhere*. — Michael - sqlbot, Oct 30 '15 at 21:03
What is the header x-amz-date for? Nginx S3 proxy works well without extra headers. How do you generate pre signed URL? — Anatoly, Oct 31 '15 at 10:22
x-amz-date is a header parameter for the date http://docs.aws.amazon.com/fr_fr/AmazonS3/latest/API/RESTCommonRequestHeaders.html — bboumend, Nov 02 '15 at 09:16
still struggling on how to serve static files from S3, nobody have clue ? — bboumend, Nov 09 '15 at 18:15

score 1 · Answer 1 · answered Jan 29 '16 at 16:30

Consider temporarily setting the proxy_pass backend service to either a local service or an HTTP echo service so you can review the full HTTP request being sent to Amazon. (If you use an HTTP echo web service, remove your sensitive bits from the request first!).

Then you debug what's wrong with the Amazon request directly. Once you figure that out, you can make the appropriate changes to Nginx so it sends a valid request header for you.

score 0 · Answer 2 · answered Sep 29 '20 at 17:05

NGINX S3 Gateway

My team at NGINX has published a fully functional (within Docker) example of how to proxy S3 API backends. It supports v2 and v4 authentication signatures. The meat of the implementation is done in njs and should be reasonably performant. Using that project as a reference, you may find a more comprehensive implementation that avoids some of the problems that you are having.

Cant connect to S3 with nginx

2 Answers2