18

I want to create an ASG such that I can reuse existing EBS volumes (because I have persistent data on those volumes) to new EC2 instances that are instantiated via ASG. Is this possible?

Update copied from comment

To clarify my original question, I am not trying to attach an EBS volume to two EC2 instances at the same time. Rather, when one EC2 instance is terminated, I want launch a new EC2 instance with the previous EBS volume instead of creating a brand new EBS volume

Tim
  • 30,383
  • 6
  • 47
  • 77
steve landiss
  • 389
  • 1
  • 3
  • 8

3 Answers3

14

IMO, you can try to do it like this (I do something similar but with Elastic Network Interfaces). This assumes either a knowledge of AWS CloudFormation service or AWS CLI:

  1. Create an EBS volume
  2. Create a bare AutoScaling group of min/max size set to 1. This will ensure that the only instance when it gets unhealthy will be replaced by a new and healthy one.
  3. Attach the volume to the instance from UserData section of AutoScaling LaunchConfiguration resource.
  4. Now, what happens with the volume when the instance will be terminated? A new one is launched by AutoScaling group. The volume will stay attached/in-use until the old one is terminated. Then, the volume will be available again and it can be attached to a new instance.
  5. Attaching/reattaching a volume is more about timing such action. You can blindly assume UserData will be executed after the volume is available again (will the instance be terminated "quickly"?). Or you can do some status checks like describe the volume metadata, check its state and when it is not in use reattach it.

A quick example in Linux bash shell:

INST_ID='i-xyzxyzxyz' 
VOL_ID='vol-xyzxyzxyz'
VOL_STATUS=''
until [[ $VOL_STATUS == '"available"' ]]; do
    VOL_STATUS=$(aws ec2 describe-volumes --volume-ids $VOL_ID --query 'Volumes[0].State')
    sleep 5
done

aws ec2 attach-volume --volume-id $VOL_ID --instance-id $INST_ID --device /dev/sdh
  1. If you want it more resilient, you can try to use AutoScaling lifecycle hooks. Before the instance is put into InService state, you can attach the volume. It will stay in this state until you tell the group to continue (otherwise it has 1h timeout by default). This is for scale-out events or when a new instance is launched. When it is terminated or scaled-in it can also wait for a signal. You would detach the volume and signal it to continue terminating it.
  2. All AWS CLI commands assumes that your instance has attached a proper IAM instance profile which allows attaching/dettaching EBS volumes or describing them.
Fermin
  • 103
  • 4
dsmsk80
  • 5,757
  • 17
  • 22
  • the `until [ "$VOLUME_STATUS" == 'attached' ]` block continually loops on AWS' Linux 2 AMIs in Bash. I've tried everything I can think of. Any idea why? – CDub Feb 13 '19 at 03:52
  • 1
    The example script sets VOL_STATUS but checks VOLUME_STATUS. Change the until line to use "x$VOL_STATUS" == "xattached" – Phil Hayward Mar 16 '20 at 16:22
  • this is basically what the [EC2 Volume Manager](https://github.com/binxio/ec2-volume-manager) does. – cowbert Apr 01 '21 at 20:35
10

No.

Autoscaling groups start multiple EC2 instances. Each EBS volume can only be attached to a single EC2 instance. Therefore you shouldn't do this, and it's probably not possible. If you have multiple volumes and multiple instances you can probably implement some complex logic to make it work, but you shouldn't - simple architectures tend to be best.

What you can do is run an EC2 instance sharing data over NFS that the ASG accesses, or you can use AWS Elastic File System, but beware latency is relatively high.

If you expand on your requirements and use case you may get more useful answers.

Update

Based on the additional requirements, it's probably possible. It's not a standard requirement so I don't think there's any explicit support for this.

I would probably look at writing some kind of startup script which associated the volume with the EC2 instance then mounted it. There's an answer here which could be of some use. It's probably possible with some kind of setup with SNS and Lambda, but the EC2 instance would likely still need something to run to map the volume.

Someone else may be able to give you more information, I've never had to do anything like this.

Community
  • 1
Tim
  • 30,383
  • 6
  • 47
  • 77
  • EFS latency is marginally higher than EBS, but still low and consistent. Anecdotal reports of high latency seem exaggerated. For large filesystems, EFS also offers more aggregate throughput than EBS. Still, this answer is essentially correct. – Michael - sqlbot Feb 11 '17 at 17:29
  • 1
    To clarify my original question, I am not trying to attach an EBS volume to two EC2 instances at the same time. Rather, when one EC2 instance is terminated, I want launch a new EC2 instance with the previous EBS volume instead of creating a brand new EBS volume – steve landiss Feb 11 '17 at 19:14
0

If your requirement is to merely share the data across ASG instances, you should go ahead and follow below article:

https://aws.amazon.com/blogs/storage/delivering-instant-data-sharing-with-multi-attach-enabled-amazon-ebs/

Once, you have a shared/Multi-Attach EBS volume available, ensure to have all your application data on its mount point. So, next launch of instance from ASG shall pickup the same data in that mount point.

However, its required that the application hosted should support shared-filesystem approach or atleast majority of its configuration should be able to lie on a shared-filesystem. If application makes lockfiles, this may not work.

In case you just want to persist an image with base application configuration(no runtime/dynamic data) then option would be to create a custom AMI from an EC2 instance where you have completed the application configuration and use that in your ASG

Arvind Kumar
  • 1