12

Right now all our systems are in a traditional data center and traditional network topology.

We're planning to migrate to AWS and in doing so we're trying to figure out how to implement our dev/test/staging/production environments. Should we:

1) Have separate AWS accounts for each environment and use consolidated billing?

2) Create separate VPCs per environment? - keep in mind we'll likely want multiple regions/availability zones for High Availability

3) Separate subnets per environment. Or should we use separate subnets for web severs/database servers/management-administration type servers?

I've done some googling and reading and I can't find any clear cut answers. There seems to be a lot of mixed opinions. Mind you most of the posts I found weren't necessarily security minded folks so I thought I would ask here.

What are other folks doing for cloud security, network topology, and a layered approach to network security with AWS.

Brad
  • 613
  • 6
  • 12

2 Answers2

5

Based on our AWS DevSecOps oriented experience a mixture of 1, 2 and 3 would probably be the most effective solution.

1) Have separate AWS accounts for each environment and use consolidated billing? Absolutely agree this was the official rec we've gotten from the AWS support at early 2017. The multi-account architecture segregating user-mgmt, security-audit-logging and shared-resources-services and project-app accounts will reduce the blast radius to the maximum possible.

Consider the following AWS official links as reference since the AWS Organization structure will depend on your specific requirements.

For it's automation you'll have some alternatives like AWS python SDK + bash or terraform (consider that if your org has less than 10 accounts managing it from the AWS Web Console will be very straight forward). However keeping in id an IaC (InfraAsCode approach with Terraform or AWS Cloud Formation is almost a must do from our point of view).

2) Create separate VPCs per environment? - keep in mind we'll likely want multiple regions/availability zones for High Availability Since we assume we're going to implement multi-account AWS Org approach then will have at least 1 VCP per account x environment. In our case having one VPC in a certain region it's enough since for High-Availability, Self-healing, Redundancy and Resilience we use a multi-availability-zone (multi-az) configuration (eg: us-east-1a, us-east-1b and us-east-1c, will be extended in 3). Keep in mind choosing your region evaluation both network performance (latency - https://www.cloudping.info/) and costs (N.Virginia-us-east-1, Ohio-us-east-2, Orego-us-west-2 - https://www.concurrencylabs.com/blog/choose-your-aws-region-wisely/).

Post steps to have in mind:

Let's consider the following info to evaluate if multi-region is also needed. As defined in the official doc -> https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html Amazon EC2 is hosted in multiple locations world-wide. These locations are composed of Regions and Availability Zones. Each Region is a separate geographic area. Each Region has multiple, isolated locations known as Availability Zones

So for as hosting our HA Apps in a region but within isolated locations (datacenters) it's enough referening to the per region AWS SLA of:

  • AWS will use commercially reasonable efforts to make the Included Services each available for each AWS region with a Monthly Uptime Percentage of at least 99.99%
  • availability: 99.99%
  • durability: 99.999999999% (depends on the storage class)

If because of compliance frameworks you still need a Multi-Region active-active method: Then by architecting multi region applications and using DNS to balance between them in normal production status, you can adjust the DNS weighting and send all traffic to the AWS region that is available, this can even be performed automatically with Route53 or other DNS services that provide health check mechanisms as well as load balancing. For multi-region architecture please consider: https://aws.amazon.com/blogs/aws/latency-based-multi-region-routing-now-available-for-aws/

ref-sla-links:

3) Separate subnets per environment. Or should we use separate subnets for web severs/database servers/management-administration type servers? To get the desired multi-az deploying for Production Grade environments at least 3 Private Subnets (through a redundant AWS Nat-GW - at least 2 of them in different az) and 3 Public Subnets (through Internet GW) of course each one in a separate AZ. And for Dev/Qa/Stg (can be consolidated in a single account for cost efficiency) and have at least 2 private and 2 public subnets in different AZs. This applies for EC2 & RDS (https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.MultiAZ.html), for S3 versioning and multi-region replication it's recommended

Based on the before presented let's present our reference architecture.

AWS org multi-account architecture

  • user-mgmt-acct (centralized iam roles based cross-org approach - iam roles per accout to be assumed still needed).
  • security-audit-logging-acct (centralized account to have centralized logs like cloudtrail, vpc-flow-logs, alb and cloudfront access logs, among others, centralized firewall manager setup, or any other AWS service or sec&audit tool). NOTE: To simplify the architecture the before mentioned accounts can be consolidated in only 1 account.
  • shared-resources-services-acct 1 x VPC + 2 prv-sub-nw + 2 public-sub-nw (multi-az) + 1 x nat-gw + VPC-Peering with project-X-app-accts (here you'll orchestrate your Software Development shared tools like, VPN Solution, Monitoring tools like prometheus and graphana, CI/CD server, maybe jenkins, drone, spinnaker, etc, multi-purpose infra K8s cluster, centralized logging solution like ELK or self hosted Splunk).
  • project-A-dev-stg-acct: 1 x VPC + 2 prv-sub-nw + 2 public-sub-nw (multi-az) + 1 x nat-gw + VPC-Peering with shared-resources-acct
  • project-A-prod-acct: 1 x VPC + 3 prv-sub-nw + 3 public-sub-nw (multi-az) + 2 x nat-gw + VPC-Peering with shared-resources-acct
  • project-B-dev-stg-acct: 1 x VPC + 2 prv-sub-nw + 2 public-sub-nw (multi-az) + 1 x nat-gw + VPC-Peering with shared-resources-acct
  • project-B-prod-acct: 1 x VPC + 3 prv-sub-nw + 3 public-sub-nw (multi-az) + 2 x nat-gw + VPC-Peering with shared-resources-acct

SECURITY - Miscellaneous

NOTE1: Remember to encrypt every service storage at rest at least with AWS encryption - https://docs.aws.amazon.com/aws-technical-content/latest/efs-encrypted-file-systems/encryption-of-data-at-rest.html

NOTE2: The 5 Pillars of the AWS Well-Architected Framework - https://aws.amazon.com/blogs/apn/the-5-pillars-of-the-aws-well-architected-framework/

NOTE3: AWS Tagging Strategies - https://aws.amazon.com/answers/account-management/aws-tagging-strategies/

NOTE4: Prevent AWS Secrets to git repository - https://www.binbash.com.ar/archives/1038

Exequiel Barrirero
  • 226
  • 2
  • 6
2

With my security hat on, I think your optimal solution is to use #1. The inconvenience factor is certainly there, but 2 and 3 create a single point of failure - all environments would be compromised if a single AWS authorized user's credentials were compromised.

Jesse K
  • 1,068
  • 6
  • 13
  • One concern with multiple accounts is data synchronization between environments. We have data we push from production -> staging -> test -> dev regularly. I think that would be easier to do if all the data was in a single account but like you point out - one compromised account compromises everything. That could be mitigated by MFA though? Does having multiple aws accounts mean we have to have multiple user accounts? That sounds like a management headache unless they can be synced - but then that negates the benefit of separate aws accounts doesn't it? – Brad Aug 24 '17 at 13:37
  • 2
    You can allow external entities access to things like your S3 buckets without giving them IAM accounts. – Jesse K Aug 24 '17 at 21:28