AWS S3 Upload Broken Up

0

1

perhaps this is my own misunderstanding of how AWS CLI works, but I uploaded a converted qcow2 VM image converted to a .raw using the AWS CLI. It took about 6 hours to move the 128GB file. I let it run overnight.

When I tried to access the file to import it as a snapshot the following day (this morning) I looked at the S3 bucket on the AWS management console and the .raw file has been broken into like a thousand parts. The Image.raw that I uploaded is not there as a whole image. Instead, I get 2019-08-02-234RTIH2F9832F92 type filenames.

I am trying the upload again in the management GUI/web portal for AWS instead of CLI but I'm afraid it's going to happen again.

I have looked at many AWS documentations for this and frankly, they suck. They will hyperlink a command like "then use the import vm command" for instance and it freaking links over to Glacier storage! (seemingly unrelated to the topic). I find the documentation very lacking and the best information I've found has been on SO/SE community and random people's blogs, which is unfortunate.

Why is my file breaking into 1000s of smaller files. What is a better way to do this?

Thanks in advance.

stfuaboutmoonmen

Posted 2019-08-29T13:33:34.717

Reputation: 1

Download one of those files and open it in a text editor. It sounds like you may have configured this bucket to store its own logs inside itself, and those are your log files. A documented example log filename is 2013-11-01-21-32-16-E568B2907131C0C0 – Michael - sqlbot – 2019-08-30T09:58:41.090

Please provide the aws-cli commands you used, redacting as appropriate. – Michael - sqlbot – 2019-08-30T09:59:38.823

No answers