Launch new EC2 instance with a specified EBS volume attached

Question

I would like to launch an EC2 instance using the python boto library, with a previously created EBS volume attached (in addition to the root volume).

After connecting, I look up the volume by its name tag, and try to create a block device list containing the volume. I was getting all kinds of errors, but eventually created the following code which at least succeeds in launching an instance.

volumes=conn.get_all_volumes(filters={'tag:Name':'TestVolume'})
vol=volumes[0]
print repr(vol)

disks=ec2.blockdevicemapping.BlockDeviceMapping()
xvdf=ec2.blockdevicemapping.BlockDeviceType(volume_id=vol.id,
    size=vol.size,volume_type=vol.type)
disks['/dev/xvdf']=xvdf

base_image="ami-9a562df2" #ubuntu 14.04
reservation=conn.run_instances(
    base_image,
    instance_type="t2.micro",
    block_device_map=disks)

The print statement shows it was able to lookup the volume and reports the correct volume id. However when the instance launches a new unformatted 32GB volume is created and attached to the volume instead.

I'm having trouble finding documentation that makes this clear. What is the correct way to add a specific EBS Volume to the block_device_map ?

Answer 1

It's not possible to launch a new EC2 instance with an existing EBS volume attached.

Instead, attach your existing EBS volume after the new EC2 instance has launched. You have two options:

1. Attach your existing EBS volume during the initial boot sequence of your instance. Using cloud init, you have an initial script run that attaches the volume.

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html

2. After you launch your new EC2 instance, wait for it to finish launching, then attach the existing EBS volume from within your Python script.

Edit: Added more details based on comments.

All the various language SDKs call the base AWS API. Look at the base AWS API documentation for RunInstances:

http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_RunInstances.html

On that command, there is a BlockDeviceMappingparameter such that if you follow it's data structure, does not include a volume ID property.

I'm not familiar with the Python SDK, but it's possible that the SDK is overloading the BlockDeviceMapping class to represent both a specification for launch (which does not permit volume ID) and a mapping for a running instance (which can include a volume ID).

I usually work with the AWS SDK for .NET and I rarely refer to the documentation specific for that. I usually refer to the base API docs (like I referenced above), then I "find" the appropriate methods to use in the .NET SDK. But I take the base API docs as correct if there is a discrepancy.

Answer 2

Unlike Matt says, it is perfectly possible to attach existing EBS volumes to AMI during its creation.

Firstly you should have the snapshot of your volume, it can be done both through console and aws-cli.

Then you create AMI based on this EBS snapshot:

#!/bin/bash

aws --region=sa-east-1 ec2 \
register-image \
--name "image" \
--description "new_image" \
--architecture x86_64 \
--kernel-id aki-912fbcfd \
--root-device-name "/dev/xvda" \
--block-device-mappings "[
{
 \"DeviceName\": \"/dev/xvda\",
 \"Ebs\": {
 \"SnapshotId\": \"snap-0dfgkhdfjkhg77\",
 \"DeleteOnTermination\": false
  }
}
]"

Then you start the instance based on the above AMI. Here I drop the sample of creating spot-instance, but creating On-Demand instance is even simpler:

#!/bin/bash
AWS_DEFAULT_OUTPUT="text"
ami="ami-xxxxxxx"
price="0.0022"
region="sa-east-1"
sgroup="sg-54654765"
zone="sa-east-1b"
key="key"

aws ec2 request-spot-instances  \
    --region $region \
    --spot-price $price  \
    --type "persistent" \
    --instance-interruption-behavior "stop" \
    --launch-specification \
    "{ \"KeyName\": \"$key\", 
       \"ImageId\": \"$ami\" , 
       \"InstanceType\": \"t1.micro\"
     }"

All this is easily scriptable and programmable.