7

I'm trying to setup sync between two buckets on different AWS accounts.

I got cp working with:

@ubuntu:~$ s3cmd cp -v s3://src/dir/ s3://dest/folder --recursive

I am the user who owns /src/dir and I've added:

{
"Version": "2012-10-17",
"Id": "Policy1477299702471",
"Statement": [
    {
        "Sid": "Stmt1477299696163",
        "Effect": "Allow",
        "Principal": {
            "AWS": "arn:aws:iam::awsid:user/name"
        },
        "Action": "s3:*",
        "Resource": "arn:aws:s3:::bucket/*"
    }
]}

To the bucket permissions policy on the test bucket.

Now I'm after aws sync to work, as using s3 cp is not recommended for using with cron.

I tried

user@ubuntu:~$ aws s3 sync --dryrun s3://src/ s3://dest/ --region eu-central-1

but I get access denied:

fatal error: An error occurred (AccessDenied) when calling the ListObjects operation: Access Denied

How can I get access denied if this is the user who owns the bucket, and get sync running?

kenorb
  • 5,943
  • 1
  • 44
  • 53
OrigamiEye
  • 182
  • 1
  • 1
  • 7

1 Answers1

14

The cause of your ListObjects error is that you assigned permission to access the contents of your bucket (arn:aws:s3:::bucket/*) but you did not give permissions to the bucket itself (arn:aws:s3:::bucket). The ListObjects command requires access to the bucket.

To test this, I did the following:

  • Used two AWS accounts: Account A, Account B
  • Created bucket-a in Account A
  • Created bucket-b in Account B
  • Created an IAM User user-a in Account A with permissions to access bucket-a

  • Added a Bucket Policy to bucket-b:

    {
  "Id": "CopyBuckets",
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1",
      "Action": "s3:*",
      "Effect": "Allow",
      "Resource": [
        "arn:aws:s3:::bucket-b",
        "arn:aws:s3:::bucket-b/*"
      ],
      "Principal": {
        "AWS": [
          "arn:aws:iam::<account-a-id>:user/user-a"
        ]
      }
    }
  ]
}

I then triggered the sync by using user-a in Account A:

aws s3 sync s3://bucket-a s3://bucket-b --profile user-a

It worked successfully.

John Rotenstein
  • 821
  • 6
  • 16
  • Hi are the regions the bucket are in not relevant? – OrigamiEye Oct 31 '16 at 07:28
  • Amazon S3 bucket names are globally unique, so ARNs (Amazon Resource Names) for S3 buckets do not need the account, nor the region (since they can be derived from the bucket name). However, when calling the `aws s3 sync` command, the region is important because you should send the request to the bucket that is *doing* the copy (the source bucket). – John Rotenstein Oct 31 '16 at 10:28
  • Watch out for missing permissions on items that are synchronised. When performing a sync you also need to use the --grants option to specify that the canonical name of the Account B is granted permission otherwise the objects that are synced will only be accessible to the original owner from Account A. – CarlR May 31 '18 at 14:08
  • If you want to have the correct owner on the objects copied into the destination bucket, you must run the sync command from the destination account not the source account. So the logic here would be flipped. – Josh Bernfeld Jun 14 '18 at 07:00
  • 1
    Or use `--acl bucket-owner-full-control` – John Rotenstein Jun 14 '18 at 07:11