curl can not download file but browser can

URL example: https://xxxxxxxxxx.s3.amazonaws.com/xxxx-xxxx-xxxx-xxxx/xxxxxxxxxxxxx/x?AWSAccessKeyId=AKIAIFKEBR4CLVTDYSLA&Signature=Kv%2B859dJ2eUmLgHdkT%2F9nUJEeRM%3D&Expires=1922517562

I tried to decode %XX but no success.

Also I tried aws s3 cp s3://xxxxxxxxxx/xxxx-xxxx-xxxx-xxxx/xxxxxxxx... - --recursive but Streaming currently is only compatible with non-recursive cp commands - I need to stream multiple files from S3.

Using wget I am getting the same 403.

URL from s3_client.generate_presigned_url.

Vitaly Zdanevich

Posted 2018-04-06T17:56:55.647

Reputation: 93

Answers

Use single quotes to make sure that any potentially special characters are taken literally.

[user@localhost ~]# curl 'https://xxxxxxxxxx.s3.amazonaws.com/xxxx-xxxx-xxxx-xxxx/xxxxxxxxxxxxx/x?AWSAccessKeyId=AKIAIFKEBR4CLVTDYSLA&Signature=Kv%2B859dJ2eUmLgHdkT%2F9nUJEeRM%3D&Expires=1922517562'

The key is single quotes.

apocalysque

Posted 2018-04-06T17:56:55.647

Reputation: 583

1As it turns out - it does not work for all cases... Any other trick? – Vitaly Zdanevich – 2018-04-07T20:21:40.800

What sort of cases is it not working for? If you're doing any variable substitution in the URL it won't work between the single quotes. – apocalysque – 2018-04-09T04:42:17.193

Works great for me. Was getting an Access Denied without the single quotes. Thanks! – Joshua Pinter – 2019-09-08T02:29:36.587

-1

Aha, I found that curl can download unquoted url (/ instead of %2F) when such url in "double quotes".

Vitaly Zdanevich

Posted 2018-04-06T17:56:55.647

Reputation: 93