13

If I have a VPC running and some servers located in the private portion of that network that do backend processing by downloading files from amazon s3, can I access S3 internally to get at those files? Or do I have to access the public internet via NAT, download s3 files over https, and process that way?

EEAA
  • 108,414
  • 18
  • 172
  • 242
The Internet
  • 473
  • 1
  • 8
  • 17
  • See my answer to http://stackoverflow.com/questions/25539057/restricting-s3-bucket-access-to-a-vpc for one way to do this, although my answer relates more to setting up an S3 bucket so that it can ONLY be accessed from a specific VPC, so it may only partially answer your question. – Eddie Jun 12 '15 at 02:15

5 Answers5

30

With a username like "The Internet", I'd expect you to know this. But since you asked...

:)

VPC's are truly private. Only traffic that you explicitly allow can transit the borders of the VPC.

So, inside a VPC, instances needing access to external resources either need to be assigned an EIP (in which case they can access external resources using AWS's infrastructure), or you need to provide a NAT host (in which case all of the traffic egresses the VPC via your own NAT).

If you opt to provide your own NAT host, remember that you'll need to disable source/dest checking on that instance as well as adding a default route to your private subnet, pointing to the NAT host.

UPDATE (2015-05-10): As of May 11th, 2015, AWS has released a "VPC Endpoint" for S3, which allows access to S3 directly from a VPC without having to go through a proxy host or NAT instance. Thankfully out of respect for the truly private nature of VPC, this feature is off by default, but can be easily turned on using the AWS Console or through their API.

EEAA
  • 108,414
  • 18
  • 172
  • 242
  • How about access control? When the traffic comes through NAT-instance, how do S3 bucket policies work (as the origin is NATted, how can S3 know what role or user is requesting the data)? – Tuukka Mustonen Dec 11 '14 at 11:13
  • @TuukkaMustonen The only concern is if you have policies based on source IP. In that case you'll need to use the NAT instance's public IP as the source IP. – filipenf Jan 09 '15 at 13:41
  • Has anyone made this work really in an isolated subnet? ie: without NAT or IGW? It seems s3 endpoint does not allow private DNS even though there is an interface type, but it seems it still requires public IPs on instances, right? – Efren May 26 '21 at 08:30
3

If your instance is in Public Subnet of VPC then:

  • Either you should have public IP address assigned to your instance
  • OR you should have elastic IP assigned to your instance

If your instance is in private subnet of VPC then:

  • You need to have a NAT device running in public subnet. So that the instance in private subnet of VPC can access internet via NAT and access S3. You can use AWS VPC NAT or you can configure your own (google for this in case you want to set-up your own NAT)

Bottom line, to access S3, You must be able to access internet.

slayedbylucifer
  • 494
  • 3
  • 7
  • 24
0

You should be using endpoint from VPC to achieve this Create a VPC endpoint for Amazon S3

  1. Open the Amazon VPC console.

  2. Using the Region selector in the navigation bar, set the AWS Region to the same Region as your VPC.

  3. From the navigation pane, choose Endpoints.

  4. Choose Create Endpoint.

  5. For Service category, verify that "AWS services" is selected.

  6. For Service Name, select the "s3" service name and "Gateway" type. For example, the service name in the US East (N. Virginia) Region is com.amazonaws.us-east-1.s3. Ref: This screenshot, if you are confused about this step

  7. For VPC, select your VPC (The VPC where you private subnet resides).

  8. For Configure route tables, select the route tables based on the associated subnets that you want to be able to access the endpoint from.

  9. For Policy, verify that Full Access is selected.

  10. Choose Create endpoint.

  11. Note the VPC Endpoint ID. You'll need this endpoint ID for a later step.

Update your bucket policy with a condition that allows users to access the S3 bucket when the request is from the VPC endpoint that you created.

To allow those users to download objects ( s3:GetObject), use a bucket policy like this one:

{
   "Version": "2012-10-17",
   "Id": "Policy1415115909152",
   "Statement": [
     {
       "Sid": "Access-to-specific-VPCE-only",
       "Principal": "*",
       "Action": "s3:GetObject",
       "Effect": "Allow",
       "Resource": ["arn:aws:s3:::DOC-EXAMPLE-BUCKET/*"],
       "Condition": {
         "StringEquals": {
           "aws:sourceVpce": "vpce-1a2b3c4d"
         }
       }
     }
   ]
}

For the value of aws:sourceVpce, make sure to enter the VPC endpoint ID of the endpoint that you previously created.

Nafsin Vk
  • 101
  • 1
0

SInce few days ago you are now able to access S3 via your VPC without using NAT or public IP.

http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-endpoints.html?sc_ichannel=em&sc_icountry=global&sc_icampaigntype=launch&sc_icampaign=em_137702700&sc_idetail=em_332195307&ref_=pe_411040_137702700_31

golja
  • 1,611
  • 10
  • 14
-3

You do not have to go "out" and back "in" or change anything about the way you transfer data within AWS regions. No fee for a transfer to/from buckets in the same region. You do have to pay for storage.

quadruplebucky
  • 5,041
  • 18
  • 23
  • This does not answer the OP's question. – EEAA Feb 26 '14 at 23:26
  • It's not clear that he's referring to a network outside of AWS. Thanks for the dv! – quadruplebucky Feb 26 '14 at 23:29
  • You're missing the point. He has EC2 instances in a private VPC subnet. By default, these don't have access to *anything* outside that VPC. So, as I stated in my answer, he'll need to take one of two options to give them access to S3. It has nothing to do with whether or not networks are inside or outside AWS, rather, it has to do with accessing outside his VPC. – EEAA Feb 26 '14 at 23:31
  • @quadruplebucky, I make clear that I'm accessing the "public internet" to get files over HTTPS. – The Internet Feb 26 '14 at 23:45
  • I'm going to bow out here, I don't have a dog in this fight. EEAA answered your question. You pay if you cross regions. – quadruplebucky Feb 26 '14 at 23:50