0

I have an instance in Google Compute Engine (cloud server) running ubuntu 16.04.

I noticed filesystem corruption on the root filesystem as follows:

==>ls -l data/vocabulary/
ls: cannot access 'data/vocabulary/Makefile': Permission denied
ls: cannot access 'data/vocabulary/vocab-count.txt': Permission denied
ls: cannot access 'data/vocabulary/vocab-random-access.db': Permission denied
ls: cannot access 'data/vocabulary/vocab-list.txt': Permission denied
ls: cannot access 'data/vocabulary/vocab.db': Permission denied
ls: cannot access 'data/vocabulary/CVS': Permission denied
total 0
d????????? ? ? ? ?            ? CVS
-????????? ? ? ? ?            ? Makefile
-????????? ? ? ? ?            ? vocab-count.txt
-????????? ? ? ? ?            ? vocab-list.txt
-????????? ? ? ? ?            ? vocab-random-access.db
-????????? ? ? ? ?            ? vocab.db

However, my attempts to "force fsck" and reboot were unsuccessful. I would touch file "/forcefsck" and reboot but fsck just simply would not occur.

I also tried to set the fsck remaining count and that also does not seem to be effective.

What should I do to proceed with fsck?

Igor Chudov
  • 21
  • 1

1 Answers1

1

It looks like there's likely an issue with your filesystem.

There's a procedure you can use in order to run the fsck utility to attempt recovery, but before getting into this I would like to mention that if you have a backup snapshot for this instance from when it was working, or if this machine is just created from a standard template or image, then restoring from that might be a more practical option. It's definitely guest OS related, so a simple revert to an older working image should fix all your problems.

If for whatever reason that's not an option, then you can continue to attempt repairs on this instance.

In order to run the fsck utility on your disk, you'll need to safely detach the disks and attach them to another instance, as fsck only works on unmounted filesystems.

First, go ahead and stop (don't delete, just stop) the instance, and detach the boot disk.

Once that's done, follow steps 2-6 under "Verify that your disk has a valid file system".

Once that's complete, you can stop (not delete, just stop) the debug instance, detach your disk, and reattach it to your original instance.

After you've verified the disk has safely been moved over, then you can go ahead and delete the debug instance.

Proceed to start the original instance. If the fsck was successful and a filesystem issue is the root cause here, then it should start and work as normal.

Mario Jiménez Feijóo
  • 11
  • 2