I'm trying to create a report about our EC2 instances usage and utilization over time. I would like to examine all of the instance that have or had been running in my environment for the past X days. As part of the report I would like to include instances that were created during a scale-group scale out but later were terminated as the group scaled in.
Using a simple python script with boto3
I can get the list of instances for a region:
session = Session(aws_access_key_id=AWS_ACCESS_KEY_ID, aws_secret_access_key=AWS_SECRET_ACCESS_KEY, region_name=self.name)
ec2 = session.resource('ec2')
cloudwatch = session.resource('cloudwatch')
By listing the instances, using the following command, I do not see terminated instances after a few hours (probably the same time you stop seeing them in the dashboard):
ec2.instances.all()
Using cloudwatch, when I use the following command, I do see the metrics for terminated instances:
metric = cloudwatch.Metric('AWS/EC2', 'CPUUtilization')
result = metric.get_statistics(
Dimensions=[{'Name': 'InstanceId', 'Value': instanceId}],
StartTime=timeRange.start,
EndTime=timeRange.end,
Period=300,
Statistics=['Average'],
) # This returns a complete list of data points for the instance was live
It means the metrics do exist for terminated instances but you have to have the id of those instances in order to retrieve them. Is there a way to retrieve terminated instances ids?