Is there a way to retrieve all contents from a versioned Amazon S3 bucket as of a particular datetime

Question

Is it possible via the AWS CLI or via some third-party tool to easily download all contents of a versioned bucket as it was on a particular datetime?

I.E. I would like to retrieve the version of each object in the bucket which would have been regarded as the 'latest' version on the given date.

Possible duplicate of [Restore a versioned S3 bucket to a particular point in time](https://serverfault.com/questions/589713/restore-a-versioned-s3-bucket-to-a-particular-point-in-time) — squarecandy, May 09 '17 at 16:35

Håkan Lindqvist · Answer 1 · 2014-06-14T13:27:17.110

Looking at the aws cli, I only see the versioning functionality exposed in the s3api section, not in the s3 section.

From what I've understood it's only possible to fetch previous versions of objects based on their version id.

By default, the GET operation returns the current version of an object. To return a different version, use the versionId subresource.

See http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectGET.html for examples and additional details

To solve what you're asking for my understanding is that, unless you have been tracking the version ids yourself, you will need to fetch a list of the versions first to determine which ones to get.

You can use the versions subresource to list metadata about all of the versions of objects in a bucket. You can also use request parameters as selection criteria to return metadata about a subset of all the object versions.

See http://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketGETVersion.html for details.

That response would have data like the following for each version that matches your query:

    <Version>
        <Key>my-image.jpg</Key>
        <VersionId>3/L4kqtJl40Nr8X8gdRQBpUMLUo</VersionId>
        <IsLatest>true</IsLatest>
         <LastModified>2009-10-12T17:50:30.000Z</LastModified>
        <ETag>&quot;fba9dede5f27731c9771645a39863328&quot;</ETag>
        <Size>434234</Size>
        <StorageClass>STANDARD</StorageClass>
        <Owner>
            <ID>75aa57f09aa0c8caeab4f8c24e99d10f8e7faeebf76c078efc7c6caea54ba06a</ID>
            <DisplayName>mtd@amazon.com</DisplayName>
        </Owner>
    </Version>

In terms of the aws cli this would be aws s3api get-object --version-id ... and aws s3api list-object-versions ... respectively for the API calls mentioned above.

Thank you, while not an quick easy solution, this will help me write a script to perform the task I need to complete. Thanks for taking the time to reply. — Nada_Surf, Jul 04 '14 at 08:34

Alberto Spelta · Answer 2 · 2014-06-15T11:48:26.383

3

Using Duplicity you can do a restore from a point in time by including the parameter --time. You can refer to this link for some examples of use with Amazon S3 service.

duplicity --time 2012-10-01T01:00:00 s3+http://backup_bucket_url /local_restore_path

edited Jun 15 '14 at 11:48

answered Jun 14 '14 at 12:50

Alberto Spelta

131
4

1

Thanks for your comment. Though I'd prefer to not have to use Duplicity, given that I will need to deal with the hassle of having a huge set of Duplicity files (my data set is about 900gb). – Nada_Surf Nov 03 '14 at 12:56

Is there a way to retrieve all contents from a versioned Amazon S3 bucket as of a particular datetime

2 Answers2