1
I have an Amazon EC2 Ubuntu t2.nano at sa-east-1 region. It was running fine until 2017-06-14 maintenance.
Today, while trying to access it via ssh, I've noted it was not possible. I get ssh connection refused. Then I moved to aws console and looked two things:
Instance Screenshot:
and System Log (partly shown below):
[ 5.500346] systemd[1]: Starting Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling...
Starting Monitoring of LVM2 mirrors... dmeventd or progress polling...
[ 5.513090] systemd[1]: Reached target Encrypted Volumes.
[[0;32m OK [0m] Reached target Encrypted Volumes.
[ 5.519554] systemd[1]: Listening on Journal Audit Socket.
[[0;32m OK [0m] Listening on Journal Audit Socket.
[ 5.527501] systemd[1]: Starting Journal Service...
Starting Journal Service...
[ 5.533351] systemd[1]: Set up automount Arbitrary Executable File Formats File System Automount Point.
[ 5.540655] Loading iSCSI transport class v2.0-870.
[[0;32m OK [0m] Set up automount Arbitrary Executab...ats File System Automount Point.
[ 5.551818] systemd[1]: Mounted Debug File System.
[[0;32m OK [0m] Mounted Debug File System.
[ 5.559413] systemd[1]: Mounted Huge Pages File System.
[[0;32m OK [0m] Mounted Huge Pages File System.
[ 5.567656] systemd[1]: Mounted POSIX Message Queue File System.
[[0;32m OK [0m] [ 5.573542] iscsi: registered transport (tcp)
Mounted POSIX Message Queue File System.
[ 5.578385] systemd[1]: Started Set console keymap.
[[0;32m OK [0m] Started Set console keymap.
[ 5.587286] systemd[1]: Started Create list of required static device nodes for the current kernel.
[[0;32m OK [0m] Started Create list of required sta...ce nodes for the current kernel.
[ 5.600884] systemd[1]: Started Uncomplicated firewall.
[[0;32m OK [0m] Started Uncomplicated firewall.
[ 5.607939] systemd[1]: Started Remount Root and Kernel File Systems.
[[0;32m OK [0m] Started Remount Root and Kernel File Systems.
[ 5.627380] systemd[1]: Started Nameserver information manager.
[[0;32m OK [0m[ 5.635695] iscsi: registered transport (iser)
] Started Nameserver information manager.
[ 5.642535] systemd[1]: Started Load Kernel Modules.
[[0;32m OK [0m] Started Load Kernel Modules.
[ 5.660181] systemd[1]: Started Journal Service.
[[0;32m OK [0m] Started Journal Service.
Starting Apply Kernel Variables...
Mounting FUSE Control File System...
[[0;32m OK [0m] Started LVM2 metadata daemon.
Starting Flush Journal to Persistent Storage...
Starting Initial cloud-init job (pre-networking)...
Starting udev Coldplug all Devices...
Starting Load/Save Random Seed...
Starting Create Static Device Nodes in /dev...
[[0;32m OK [0m] Mounted FUSE Control File System.
[[0;32m OK [0m] Started Apply Kernel Variables.
[[0;32m OK [0m] Started Load/Save Random Seed.
[[0;32m OK [0m] Started udev Coldplug all Devices.
[[0;32m OK [0m] Started Flush Journal to Persistent Storage.
[[0;32m OK [0m] Started Create Static Device Nodes in /dev.
Starting udev Kernel Device Manager...
[[0;32m OK [0m] Started udev Kernel Device Manager.
[[0;32m OK [0m] Started Dispatch Password Requests to Console Directory Watch.
[[0;32m OK [0m] Started Monitoring of LVM2 mirrors,...ng dmeventd or progress polling.
[[0;32m OK [0m] Reached target Local File Systems (Pre).
[[0;32m OK [0m] Found device /dev/ttyS0.
[[0;32m OK [0m] Found device /sys/subsystem/net/devices/eth0.
[[0;32m OK [0m] Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.
[ 8.686223] cloud-init[415]: Cloud-init v. 0.7.8 running 'init-local' at Thu, 15 Jun 2017 13:02:25 +0000. Up 8.20 seconds.
[[0;32m OK [0m] Started Initial cloud-init job (pre-networking).
[[0;32m OK [0m] Reached target Network (Pre).
[[0m[0;31m* [0m] A start job is running for dev-xvdf.device (8s / 1min 30s)
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (9s / 1min 30s)
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (9s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (10s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (10s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for dev-xvdf.device (11s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m] A start job is running for dev-xvdf.device (11s / 1min 30s)
[K[ [0;31m*[0m] A start job is running for dev-xvdf.device (12s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m] A start job is running for dev-xvdf.device (12s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for dev-xvdf.device (13s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (13s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (14s / 1min 30s)
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (14s / 1min 30s)
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (15s / 1min 30s)
[K[[0m[0;31m* [0m] A start job is running for dev-xvdf.device (15s / 1min 30s)
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (16s / 1min 30s)
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (16s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (17s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (17s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for dev-xvdf.device (18s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m] A start job is running for dev-xvdf.device (18s / 1min 30s)
[K[ [0;31m*[0m] A start job is running for dev-xvdf.device (19s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m] A start job is running for dev-xvdf.device (19s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for dev-xvdf.device (20s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (20s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (21s / 1min 30s)
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (21s / 1min 30s)
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (22s / 1min 30s)
[K[[0m[0;31m* [0m] A start job is running for dev-xvdf.device (22s / 1min 30s)
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (23s / 1min 30s)
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (23s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (24s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (24s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for dev-xvdf.device (25s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m] A start job is running for dev-xvdf.device (25s / 1min 30s)
[K[ [0;31m*[0m] A start job is running for dev-xvdf.device (26s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m] A start job is running for dev-xvdf.device (26s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for dev-xvdf.device (27s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (27s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (28s / 1min 30s)
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (28s / 1min 30s)
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (29s / 1min 30s)
[K[[0m[0;31m* [0m] A start job is running for dev-xvdf.device (29s / 1min 30s)
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (30s / 1min 30s)
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (30s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (31s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (31s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for dev-xvdf.device (32s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m] A start job is running for dev-xvdf.device (32s / 1min 30s)
[K[ [0;31m*[0m] A start job is running for dev-xvdf.device (33s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m] A start job is running for dev-xvdf.device (33s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for dev-xvdf.device (34s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (34s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (35s / 1min 30s)
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (35s / 1min 30s)
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (36s / 1min 30s)
[K[[0m[0;31m* [0m] A start job is running for dev-xvdf.device (36s / 1min 30s)
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (37s / 1min 30s)
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (37s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (38s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (38s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for dev-xvdf.device (39s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m] A start job is running for dev-xvdf.device (39s / 1min 30s)
[K[ [0;31m*[0m] A start job is running for dev-xvdf.device (40s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m] A start job is running for dev-xvdf.device (40s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for dev-xvdf.device (41s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (41s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (42s / 1min 30s)
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (42s / 1min 30s)
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (43s / 1min 30s)
[K[[0m[0;31m* [0m] A start job is running for dev-xvdf.device (43s / 1min 30s)
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (44s / 1min 30s)
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (44s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (45s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (45s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for dev-xvdf.device (46s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m] A start job is running for dev-xvdf.device (46s / 1min 30s)
[K[ [0;31m*[0m] A start job is running for dev-xvdf.device (47s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m] A start job is running for dev-xvdf.device (47s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for dev-xvdf.device (48s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (48s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (49s / 1min 30s)
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (49s / 1min 30s)
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (50s / 1min 30s)
[K[[0m[0;31m* [0m] A start job is running for dev-xvdf.device (50s / 1min 30s)
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (51s / 1min 30s)
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (51s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (52s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (52s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for dev-xvdf.device (53s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m] A start job is running for dev-xvdf.device (53s / 1min 30s)
[K[ [0;31m*[0m] A start job is running for dev-xvdf.device (54s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m] A start job is running for dev-xvdf.device (54s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for dev-xvdf.device (55s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (55s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (56s / 1min 30s)
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (56s / 1min 30s)
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (57s / 1min 30s)
[K[[0m[0;31m* [0m] A start job is running for dev-xvdf.device (57s / 1min 30s)
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (58s / 1min 30s)
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (58s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (59s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (59s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for dev-xvdf.device (1min / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m] A start job is running for dev-xvdf.device (1min / 1min 30s)
[K[ [0;31m*[0m] A start job is running for dev-xvdf.device (1min 1s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m] A start job is running for dev-xvdf.device (1min 1s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for dev-xvdf.device (1min 2s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 2s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 3s / 1min 30s)
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 3s / 1min 30s)
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 4s / 1min 30s)
[K[[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 4s / 1min 30s)
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 5s / 1min 30s)
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 5s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 6s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 6s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for dev-xvdf.device (1min 7s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m] A start job is running for dev-xvdf.device (1min 7s / 1min 30s)
[K[ [0;31m*[0m] A start job is running for dev-xvdf.device (1min 8s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m] A start job is running for dev-xvdf.device (1min 8s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for dev-xvdf.device (1min 9s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 9s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 10s / 1min 30s)
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 10s / 1min 30s)
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 11s / 1min 30s)
[K[[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 11s / 1min 30s)
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 12s / 1min 30s)
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 12s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 13s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 13s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for dev-xvdf.device (1min 14s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m] A start job is running for dev-xvdf.device (1min 14s / 1min 30s)
[K[ [0;31m*[0m] A start job is running for dev-xvdf.device (1min 15s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m] A start job is running for dev-xvdf.device (1min 15s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for dev-xvdf.device (1min 16s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 16s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 17s / 1min 30s)
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 17s / 1min 30s)
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 18s / 1min 30s)
[K[[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 18s / 1min 30s)
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 19s / 1min 30s)
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 19s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 20s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 20s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for dev-xvdf.device (1min 21s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m] A start job is running for dev-xvdf.device (1min 21s / 1min 30s)
[K[ [0;31m*[0m] A start job is running for dev-xvdf.device (1min 22s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m] A start job is running for dev-xvdf.device (1min 22s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for dev-xvdf.device (1min 23s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 23s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 24s / 1min 30s)
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 24s / 1min 30s)
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 25s / 1min 30s)
[K[[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 25s / 1min 30s)
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 26s / 1min 30s)
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 26s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 27s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for dev-xvdf.device (1min 27s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for dev-xvdf.device (1min 28s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m] A start job is running for dev-xvdf.device (1min 28s / 1min 30s)
[K[ [0;31m*[0m] A start job is running for dev-xvdf.device (1min 29s / 1min 30s)
[K[ [0;31m*[0;1;31m*[0m] A start job is running for dev-xvdf.device (1min 29s / 1min 30s)
[K[[0;1;31m TIME [0m] Timed out waiting for device dev-xvdf.device.
[[0;1;33mDEPEND[0m] Dependency failed for /mnt/xvdf.
[[0;1;33mDEPEND[0m] Dependency failed for Local File Systems.
Starting Set console font and keymap...
[[0;32m OK [0m] Stopped getty on tty2-tty6 if dbus and logind are not available.
[[0;32m OK [0m] Stopped Getty on tty1.
[[0;32m OK [0m] Stopped Daily apt activities.
[[0;32m OK [0m] Closed Unix socket for apport crash forwarding.
[[0;32m OK [0m] Stopped Trigger resolvconf update for networkd DNS.
[[0;32m OK [0m] Stopped Daily Cleanup of Temporary Directories.
[[0;32m OK [0m] Closed UUID daemon activation socket.
[[0;32m OK [0m] Stopped Execute cloud user/final scripts.
[[0;32m OK [0m] Stopped Apply the settings specified in cloud-config.
[[0;32m OK [0m] Closed ACPID Listen Socket.
[[0;32m OK [0m] Stopped target Graphical Interface.
[[0;32m OK [0m] Stopped target Multi-User System.
[[0;32m OK [0m] Reached target Cloud-init target.
[[0;32m OK [0m] Stopped LSB: daemon to balance interrupts for SMP systems.
[[0;32m OK [0m] Stopped D-Bus System Message Bus.
[[0;32m OK [0m] Stopped Docker Application Container Engine.
[[0;32m OK [0m] Closed Docker Socket for the API.
[[0;32m OK [0m] Stopped LSB: MD monitoring daemon.
[[0;32m OK [0m] Stopped Terminate Plymouth Boot Screen.
[[0;32m OK [0m] Stopped LXD - container startup/shutdown.
[[0;32m OK [0m] Stopped System Logging Service.
[[0;32m OK [0m] Stopped OpenBSD Secure Shell server.
[[0;32m OK [0m] Stopped Pollinate to seed the pseudo random number generator.
[[0;32m OK [0m] Stopped LSB: Set the CPU Frequency Scaling governor to "ondemand".
[[0;32m OK [0m] Closed LXD - unix socket.
[[0;32m OK [0m] Stopped Auto import assertions from block devices.
[[0;32m OK [0m] Closed Syslog Socket.
[[0;32m OK [0m] Stopped Snappy daemon.
[[0;32m OK [0m] Stopped LSB: Set up cgroupfs mounts..
[[0;32m OK [0m] Stopped Service for virtual machines hosted on VMware.
[[0;32m OK [0m] Stopped Serial Getty on ttyS0.
[[0;32m OK [0m] Stopped Hold until boot process finishes up.
[[0;32m OK [0m] Stopped /etc/rc.local Compatibility.
[[0;32m OK [0m] Stopped Permit User Sessions.
Starting Set console scheme...
[[0;32m OK [0m] Reached target Login Prompts.
[[0;32m OK [0m] Stopped LSB: Record successful boot for GRUB.
[[0;32m OK [0m] Stopped noip2.service.
[[0;32m OK [0m] Stopped Deferred execution scheduler.
[[0;32m OK [0m] Stopped ACPI Events Check.
[[0;32m OK [0m] Stopped Login Service.
[[0;32m OK [0m] Stopped FUSE filesystem for LXC.
[[0;32m OK [0m] Stopped Accounts Service.
[[0;32m OK [0m] Stopped Timer to automatically refresh installed snaps.
[[0;32m OK [0m] Reached target Timers.
[[0;32m OK [0m] Stopped LSB: automatic crash report generation.
[[0;32m OK [0m] Closed D-Bus System Message Bus Socket.
[[0;32m OK [0m] Stopped Regular background program processing daemon.
[[0;32m OK [0m] Closed Socket activation for snappy daemon.
[[0;32m OK [0m] Stopped LSB: Apache2 web server.
[[0;32m OK [0m] Stopped target Basic System.
[[0;32m OK [0m] Reached target Paths.
[[0;32m OK [0m] Reached target Sockets.
[[0;32m OK [0m] Stopped target System Initialization.
Starting LSB: AppArmor initialization...
Starting Create Volatile Files and Directories...
[[0;32m OK [0m] Started Emergency Shell.
Starting Tell Plymouth To Write Out Runtime Data...
[[0;32m OK [0m] Reached target Emergency Mode.
[[0;32m OK [0m] Started Set console font and keymap.
[[0;32m OK [0m] Started Set console scheme.
[[0;32m OK [0m] Started Create Volatile Files and Directories.
Starting Network Time Synchronization...
Starting Update UTMP about System Boot/Shutdown...
[[0;32m OK [0m] Created slice system-getty.slice.
[[0;32m OK [0m] Started Tell Plymouth To Write Out Runtime Data.
[[0;32m OK [0m] Started Network Time Synchronization.
[[0;32m OK [0m] Reached target System Time Synchronized.
[[0;32m OK [0m] Started Update UTMP about System Boot/Shutdown.
Starting Update UTMP about System Runlevel Changes...
[[0;32m OK [0m] Started Update UTMP about System Runlevel Changes.
[[0;32m OK [0m] Started LSB: AppArmor initialization.
Starting Raise network interfaces...
[[0;32m OK [0m] Started ifup for eth0.
[[0;32m OK [0m] Started Raise network interfaces.
Starting Initial cloud-init job (metadata service crawler)...
[[0;32m OK [0m] Reached target Network.
[ 97.469232] cloud-init[923]: Cloud-init v. 0.7.8 running 'init' at Thu, 15 Jun 2017 13:03:54 +0000. Up 96.85 seconds.
[ 97.476693] cloud-init[923]: ci-info: +++++++++++++++++++++++++++++++++++++Net device info++++++++++++++++++++++++++++++++++++++
[ 97.484692] cloud-init[923]: ci-info: +--------+------+----------------------------+---------------+-------+-------------------+
[ 97.496661] cloud-init[923]: ci-info: | Device | Up | Address | Mask | Scope | Hw-Address |
[ 97.504676] cloud-init[923]: ci-info: +--------+------+----------------------------+---------------+-------+-------------------+
[ 97.512656] cloud-init[923]: ci-info: | eth0 | True | 172.31.28.229 | 255.255.240.0 | . | 02:92:48:51:32:d5 |
[ 97.520690] cloud-init[923]: ci-info: | eth0 | True | fe80::92:48ff:fe51:32d5/64 | . | link | 02:92:48:51:32:d5 |
[ 97.532598] cloud-init[923]: ci-info: | lo | True | 127.0.0.1 | 255.0.0.0 | . | . |[[0;32m OK [0m] Started Initial cloud-init job (metadata service crawler).
[[0;32m OK [0m
[ 97.540937] cloud-init[923]: ci-info: | lo | True | ::1/128 | . | host | . |] Reached target Cloud-config availability.
[[0;32m OK [0m] Reached target Network is Online.
[ 97.549254] cloud-init[923]: ci-info: +--------+------+----------------------------+---------------+-------+-------------------+ Starting iSCSI initiator daemon (iscsid)...
[ 97.557931] cloud-init[923]: ci-info: +++++++++++++++++++++++++++++Route IPv4 info+++++++++++++++++++++++++++++
[ 97.569483] cloud-init[923]: ci-info: +-------+-------------+-------------+---------------+-----------+-------+
[ 97.575302] cloud-init[923]: ci-info: | Route | Destination | Gateway | Genmask | Interface | Flags |
[ 97.579340] cloud-init[923]: ci-info: +-------+-------------+-------------+---------------+-----------+-------+
[ 97.583365] cloud-init[923]: ci-info: | 0 | 0.0.0.0 | 172.31.16.1 | 0.0.0.0 | eth0 | UG |
[ 97.587524] cloud-init[923]: ci-info: | 1 | 172.31.16.0 | 0.0.0.0 | 255.255.240.0 | eth0 | U |
[ 97.591777] cloud-init[923]: ci-info: +-------+-------------+-------------+---------------+-----------+-------+
[[0;32m OK [0m] Started iSCSI initiator daemon (iscsid).
Starting Login to default iSCSI targets...
[[0;32m OK [0m] Started Login to default iSCSI targets.
[[0;32m OK [0m] Reached target Remote File Systems (Pre).
[[0;32m OK [0m] Reached target Remote File Systems.
Welcome to emergency mode! After logging in, type "journalctl -xb" to view
system logs, "systemctl reboot" to reboot, "systemctl default" or ^D to
try again to boot into default mode.
Give root password for maintenance
(or press Control-D to continue):
It seems user intervention is required (not sure why).
I tried to detach the volume and attach it to another instance but then the other instance also refused to boot with similar issue. So I guess the issue is related to that volume.
May I ask what's really going on and how I got into this situation?
How should I proceed?
Is this related to Ubuntu only? Just started an Amazon Linux and attached volume to it, here at least I can mount it.
KcFnMi
Posted 2017-06-15T17:33:09.983
Reputation: 396
1something crashed badly. what about "Welcome to emergency mode! After logging in, type "journalctl -xb" to view system logs" ? – Federico Galli – 2017-06-20T13:55:26.440
I have had the same issue with same last few messages in logs, on an Ubuntu 16.04 instance on EC2. Not sure why, other than perhaps iscsi getting confused as those remote filesystem messages are the last ones before emergency mode message. I have one non-root EBS volume which I configured in
/etc/fstab, but no iSCSI. One guess is that mounting from /etc/fstab isn't working for some reason, which can sometimes cause this I think. – RichVel – 2018-05-26T10:09:36.963