AWS Instance Created from Snapshot Failing "Instance reachability check"

0

I have a dev environment hosted by AWS (t2.large, RHEL 7.4) that is backed up with a snapshot every couple of hours. Today, I tried to create a new instance from one of those snapshots to run some tests but when I started the instance I got a "Instance reachability check failed" error.

Things I have tried so far:

  1. Double checked security groups
  2. Rebooted the instance
  3. Made sure the instance OS was the exact same OS as the instance the snapshot was taken of
  4. Followed this guide on troubleshooting an unreachable instance, none of the error messages mentioned were in the system log
  5. Got a screenshot of the unreachable instance, it just shows a RHEL login screen
  6. Ran the EC2 Rescue tool, it's been running for 45 minutes now and has returned no results.
  7. SSH-ing into the server gives a "Network error: Connection timed out" error
  8. Creating an image from the snapshot and launching that still produces an instance that fails the reachability check

What else can I try to find more information about this error?

Note: This instance contains irreplaceable data, nuking it and starting over isn't an option.

Josh Hamilton

Posted 2019-09-27T20:48:54.423

Reputation: 1

"Made sure the instance OS was the exact same OS as the instance the snapshot was taken of" What does this mean? I'd expect you to create an AMI from your snapshot and use that as the OS. – Anon Coward – 2019-09-27T22:33:06.167

I launched a new instance with the OS set to the OS of the snapshot, and then swapped out the volumes. I also tried launching it from an AMI. – Josh Hamilton – 2019-09-27T22:59:14.710

Swapping out mounted volumes generally doesn't end well. If you can't launch it from an AMI, I'd have to fall back onto launching an image with a new OS, and mounting an EBS volume from the snapshot as a second volume to get the data off of it. – Anon Coward – 2019-09-27T23:14:15.323

Have you, for some reason, configured the original instance not to use DHCP? That would explain the failed reachability check on the new machine, which (if I remember right) is a test of whether the instance responds to ARP requests for its assigned IP address. It wouldn't respond, if you had done something like that. – Michael - sqlbot – 2019-09-28T03:24:55.340

@Michael-sqlbot haven't modified the network settings for any instance – Josh Hamilton – 2019-09-30T18:15:17.307

1@AnonCoward Weird, I've swapped volumes hundreds of times in the last year and this is the first I'm hearing of this. – Josh Hamilton – 2019-09-30T18:20:02.650

@AnonCoward I just tried creating an image from the snapshot and after launching it it still fails the reachability check – Josh Hamilton – 2019-09-30T19:06:07.993

No answers