Bulk delete ec2 snapshots from Jenkins

Question

Use case:
Export list of snapshots in txt file (old_snapshots.txt) to S3 bucket. From Jenkins, use aws cp command copy file to Jenkins /tmp/directory

--dry-run did not show any error

However, when passed the following bash line with aws delete command in Jenkins it shows SUCCESS, at the same says error has occurred and the snapshots are not deleted.

## actual deletion
        file="/tmp/old_snapshots.txt"
         while read delete_data
         do
        aws ec2 --region eu-west-1 delete-snapshot --snapshot-id $file
        echo "Deleting snapshot $delete_data"
        done <"$file"

Output

An error occurred (InvalidParameterValue) when calling the DeleteSnapshot operation: Value (/tmp/old_snapshots.txt) for parameter snapshotId is invalid. Expected: 'snap-...'.
Deleting snapshot snap-xxxx81xxxxxx7fxxx

An error occurred (InvalidParameterValue) when calling the DeleteSnapshot operation: Value (/tmp/old_snapshots.txt) for parameter snapshotId is invalid. Expected: 'snap-...'.
Deleting snapshot snap-xxxacc49xxxxxxc26


An error occurred (InvalidParameterValue) when calling the DeleteSnapshot operation: Value (/tmp/old_snapshots.txt) for parameter snapshotId is invalid. Expected: 'snap-...'.
Deleting snapshot snap-04xxxxxxxx3fa3cxxxxxxf4

[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS

Request: This error An error occurred (InvalidParameterValue) when calling the DeleteSnapshot operationseems too generic? Have anyone experienced seem issue? Also, I did a manual test of deleting just a single snapshot with aws cli by manually adding the snapshot id in Jenkins and that works fine. Any suggestion would be greatly appreciated.

Including aws command

aws ec2 --region eu-west-1 delete-snapshot --snapshot-id snap-01x1618xxxxxxa51x
Found snapshotid: snap-01x1618xxxxxxa51x in the uploaded file: /tmp/old_snapshots.txt
Now deleting snapshot id snap-01x1618xxxxxxa51x

score 1 · Answer 1 · answered Jan 28 '20 at 06:14

You need to change your script a bit.

$ sh abc.sh

What is in delete_data snap-xxxx81xxxxxx7fxxx
What is in file /tmp/old_snapshots.txt
Deleting snapshot snap-xxxx81xxxxxx7fxxx
What is in delete_data snap-xxxacc49xxxxxxc26
What is in file /tmp/old_snapshots.txt
Deleting snapshot snap-xxxacc49xxxxxxc26
What is in delete_data snap-04xxxxxxxx3fa3cxxxxxxf4
What is in file /tmp/old_snapshots.txt
Deleting snapshot snap-04xxxxxxxx3fa3cxxxxxxf4

$ cat abc.sh
## actual deletion

    file="/tmp/old_snapshots.txt"
    while read delete_data
    do
    #aws ec2 --region eu-west-1 delete-snapshot --snapshot-id $file
    echo "What is in delete_data $delete_data"
    echo "What is in file $file"
    echo "Deleting snapshot $delete_data"
    done < $file

Your script is calling file variable is constant but you need to pass content of file line by line. So, replace below

aws ec2 --region eu-west-1 delete-snapshot --snapshot-id $file

With

aws ec2 --region eu-west-1 delete-snapshot --snapshot-id $delete_data

@askyagi, thanks a lot for all the effort scanning through the code. Yes, I tested both with the `file` variable and `delete_data` and the result is still the same. The txt file contains only the list of snapshot-ids in a list format. — Sunny J, Jan 28 '20 at 14:36
Can you update question what value is passed with aws command just echo `aws ` in script it will give us more idea. — asktyagi, Jan 28 '20 at 15:37

score 0 · Accepted Answer · answered Jan 28 '20 at 18:59

For the benefit of other community users, I used the --debug flag to discover that additional character '\r' were being added to the snap-#

Debug output

2020-01-28 18:09:02,295 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.ec2.delete-snapshot.snapshot-id: calling handler <awscli.paramfile.URIArgumentHandler object at >
2020-01-28 18:09:02,296 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.ec2.delete-snapshot: calling handler <awscli.argprocess.ParamShorthandParser object at >
2020-01-28 18:09:02,296 - MainThread - awscli.arguments - DEBUG - Unpacked value of u'snap-0xxxxxxxxxxxxxxxx\r' for parameter "snapshot_id": u'snap-0xxxxxxxxxxxxxxxx\r'

To solve the issue: I passed tr -d '\r' to variable holding the value, to make the conversation.

Example:

tr -d '\r' < input > output

Updated script

file="/tmp/old_snapshots.txt"
cat $file | tr -d '\r' | while read -r line; 
do 
aws ec2 --region eu-west-1 delete-snapshot --snapshot-id $line                        
done

Bulk delete ec2 snapshots from Jenkins

Including aws command

2 Answers2