I've been looking into using AWS cli for data integrity checks to verify a backup has been transferred from a Linux file server correctly to AWS s3. Likewise, I would like to verify when restoring a file from backup to the Linux file server it also transferred correctly.
I examined the etag stored with the object on S3, because it appears to be a md5sum. However, if the file is transferred as multipart in the case of large files, the etag is no longer valid.
But before I go further in trying to do a MD5sum to what has just been sync'ed to S3 each time, is this really necessary to do this? When using rsync between Linux file systems over the internet, it isn't common practice to do an md5sum on the files transferred to verify them. Because it is assumed I think that rsync has already taken care of this?
So I'm wondering does AWS cli sync already take care of the data integrity check for us?