6

I learned a long time ago that if you want an EC2 instance to be able to talk to AWS services like SQS, Kinesis and the like, the instance has to either have a public IP address or there needs to be something within the VPC doing NAT.

When I first heard about VPC Endpoints earlier this year it seemed like a game-changer -- a way to access these services from instances that only had private IPs. But the fact that the only supported service is S3 seems really limiting. Every EC2 instance I currently manage has some other dependency on a non-S3 AWS resource that requires traffic to go out through a public IP.

I'm curious if there is any other benefit to using a VPC Endpoint for S3 anyway, with the understanding that the rest of the stack will still depend on public IPs and will for some time. Is there any measurable improvement in speed/throughput, or any other benefit that can outweigh the management overhead?

smitelli
  • 1,214
  • 1
  • 10
  • 16

3 Answers3

3

This is an old thread but they just released VPC endpoints for DynamoDB. I hope they add them for SQS. Right now since Lambda functions running in a VPC don't have internet access, they cannot access any public AWS services without setting up a NAT Gateway. So if you want your Lambda functions to access a resource in a VPC, RDS for example, and send something to SQS, SNS, SES, any other public AWS service without a VPC Endpoint or call another Lambda function, you can't.

If they added VPC endpoints for SQS you could access your VPC resources with a VPC lambda function, send results to a SQS queue, then have a public Lambda function watch and consume the queue messages, accessing any public AWS service (SNS, SES) as needed. Currently I am stuck 'watching' S3/DynamoDB since I don't want to incur the costs of a managed NAT Gateway.

Cyrus
  • 31
  • 1
2

VPC nodes without a public IP have to traverse either a NAT Gateway instance or NAT Service to reach S3 (ignoring VPC Endpoints for a moment). The former is an availability and bandwidth constraint, the the latter cost per-byte to utilize.

With a VPC Endpoint, your private hosts do not need to traverse either to get to S3 and can do so more cheaply and potentially faster.

Jason Martin
  • 4,865
  • 15
  • 24
  • 2
    How much faster? Have you run any tests to determine the speed difference? – Arjun Prasad Feb 09 '17 at 07:01
  • I haven't specifically tested it. The scenario where it would help is where you have nodes in a private-subnet. Lacking the endpoint they would need to traverse a NAT instance or NAT Service to reach S3, which is a significant / potential (respectively) constraint. With the endpoint that constraint is removed. – Jason Martin Feb 09 '17 at 15:51
  • I have more than 1 TB of data which is in my server. I need to push it into my S3 bucket. Will it cost less if I use vpc endpoint rather than using the normal transfer? – Arjun Prasad Feb 10 '17 at 12:34
  • @ArjunPrasad - this is an old comment - but if nothing else, you should save on outbound data transfer costs - which adds up fast.... – Dan G Jun 01 '21 at 19:50
0

The benefit to VPC endpoints is primarily security. The ability to store/retrieve data in S3 within a closed private VPC is an attractive one to those not using AWS for public facing uses.

47cloud
  • 181
  • 1
  • 5