8

I'm trying to backup a directory to a pre-existing Amazon S3 bucket using the following command:

duplicity --no-encryption system/ s3+http://MY_BUCKET_NAME/backup

However, I'm getting the following error consistently:

S3CreateError: S3CreateError: 409 Conflict
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>BucketAlreadyOwnedByYou</Code><Message>Your previous request to create the named bucket succeeded and you already own it.</Message><BucketName>vacationlabs</BucketName><RequestId>3C1B8C49469E3374</RequestId><HostId>4dU1TKf3Td6R0yvG9MaLKCYvQfwaCpdM8FUcv53aIOh0LeJ6wtVHHduPSTqjDwt0</HostId></Error>
  1. The S3 bucket is empty and does NOT have the backup directory
  2. The bucket is in Singapore region
Saurabh Nanda
  • 449
  • 1
  • 7
  • 17

3 Answers3

12

Duplicity also supports Amazon S3 URLs in this format:

s3://host/bucket_name[/prefix]

where host is the S3 endpoint for your region, which you can find in Amazon's list of endpoints.

In your case, the URL you want is:

s3://s3-ap-southeast-1.amazonaws.com/MY_BUCKET_NAME/backup

Note that:

  • Duplicity can still only create buckets in the US Standard and EU regions, which means you'll need to manually create the bucket (which you already have).

  • For regions other than US Standard, you will want to have Duplicity use Amazon's "new style" subdomain bucket addressing, by adding --s3-use-new-style to your Duplicity command line.

Jeffery To
  • 331
  • 3
  • 7
  • 2
    With s3+http I got duplicity error "Access Denied". Got fixed with s3://host/bucket. Used same IAM user / policy. Go figure.... – michaelbn May 22 '13 at 14:33
2

@mgorven is right, and I'm going to try to get this fixed properly. If you'd like to see this fixed, please mark yourself affected and subscribe at https://bugs.launchpad.net/duplicity/+bug/1003159

The s3://<region>/<bucket> syntax doesn't work (any more), you can only do s3+http://<bucket> which uses us-east-1 or eu-west-1 if you specify --s3-european-buckets. You can't get to any other regions.

Ian Gibbs
  • 31
  • 3
  • I encountered this too and I agree it should be handled properly. There is also a Zone-Infrequent S3 class now that could be used if one wanted to. – Andrea Richiardi Oct 08 '18 at 00:07
  • endpoints like `s3://s3.ap-southeast-1.amazonaws.com/bucket-name` are working fine in 0.7.17, haven't tried current version 0.7.19 – Mr5o1 May 25 '19 at 20:47
  • 1
    I found that `s3://s3.ap-northeast-1.amazonaws.com/bucket-name` works but only if I don't also say `--s3-use-multiprocessing`. With this flag present, we get the UnsupportedBackendScheme error which seems wrong and confusing. Hope this helps someone. On version 0.8.19. – Mateusz Kowalczyk May 11 '21 at 03:50
  • I confirm the need for http: `s3+http://mybucket`. – tash Feb 14 '22 at 00:16
1

Based on the duplicity manpage, it looks like it only supports the US and EU S3 regions. I'd guess that it's talking to the US endpoint, detects that the bucket doesn't exist in the US region, and then tries to create it and fails.

Try creating a bucket in the US or EU region and test whether that works (using --s3-european-buckets if appropriate). If that does work and you really want to use the Singapore region, you'll have to hack Duplicity to support it.

mgorven
  • 30,036
  • 7
  • 76
  • 121