0
Uploading a file over 10 GB to s3 fails with:
aws s3 cp 'foo.log.gz' s3://mybucket/2018/12/ --storage-class REDUCED_REDUNDANCY
upload failed: ./foo.log.gz to s3://mybucket/2018/12/foo.log.gz
An error occurred (BadDigest) when calling the UploadPart operation (reached max retries: 4): The Content-MD5 you specified did not match what we received.
How to work around that error?
user51070
Posted 2018-12-10T10:34:04.850
Reputation: 105
0
This worked in part: md5 is wrong is should use sha256, setting this helps:
aws configure set default payload_signing_enabled = true
Second Solution
Issue more in depth is related to:
http://tracker.ceph.com/issues/20447
Even deeper: https://github.com/ceph/ceph/pull/15965/files
TLDR
Some regions use ONLY "v4 signatures" (so does eu-central-1) there is a bug in the sha2 signing in awscli.
The issue only strikes on multi part uploads. If the minimal junk size ia greater than the file, the bug in part singing is not fatal:
aws configure set multipart_chunksize = 20GB
user51070
Posted 2018-12-10T10:34:04.850
Reputation: 105