0
I hope anybody has experience with EC2. I'm in a spot.
I accidentally removed glibc from my Amazon Linux 2 instance. Well, not entirely on accident, I was trying to replace the one I had with the official Amzn2 repo package (I had manually tried to upgrade it earlier via rpm).
Although I had done it before, I followed this guide: https://www.rootusers.com/how-to-repair-an-aws-ec2-instance-without-console/
I created a new Amazon Linux 2 instance, mounted a new volume from the snapshot I made, mounted dev/nvme1n1p1 and chrooted. Repaired glibc and yum. Did yum distro-sync (no problems left now). I umounted everything carefully. Shut down the instance. Created a new snapshot. Created a new volume from that snapshot. Mounted that volume to my first (broken) instance as dev/xvda. And it doesn't work. 1/2 system checks passed (like after a reallly long time) and it's unreachable. Looked through the kernel log but couldn't find anything out of the ordinary. I tried re-attaching Elastic IPs.
I even tried to mount the fixed volume to my temporary instance as root directly after repairing it. Still didn't work. Even tried to create AMI from the snapshot. I would sincerely appreciate any help, I dont't know what to do.
psisis
Posted 2019-06-27T20:00:18.120
Reputation: 9
P.S. could it be that I actually never repaired glibc? I had to mount lib and lib64 from the host into the chroot environment because /bin/bash wouldn't work with chroot. I suspect that if yum installed anything into these directories it would be null, since it would just be umounted – psisis – 2019-06-27T20:04:22.953
It's not an experimental procedure, it's standard chroot. The reason I have to snapshot and create a new volume instead of just mounting the broken volume in my working host instance is that the instances are on two different availability zones, i.e. their SSD volumes aren't interchangeable. It actually works now. The problem was that I couldn't mount /lib64. But instead I "cp -n /lib64/* /mnt/lib64/" and it ran flawlessy – psisis – 2019-06-28T13:20:52.757
Ah great. Please post your solution as an answer, to help others with the same problem in future. Another option if others have problems in the future is to create a new AL2 image, mount the old drive as a data drive, copy the data over, and set up the software again. I prefer Ubuntu over AL because it has a LOT more community support. – Tim – 2019-06-28T18:41:30.903
You're right. Amazon Linux has given me a lot of headaches what with its limited repos and documentation. Upgrading a package could also be hell and requires manual compilation. But I had already started with it and it also has its good qualities; I recently set up logging for Cloudwatch and it was super easy installing it to AmznLinux2 – psisis – 2019-06-29T15:14:22.813
That's pretty easy with Ubuntu as well. – Tim – 2019-06-30T00:13:20.383