I intended to setup encrypted LUKS arch linux system on a cloud server. I installed the latest arch linux version onto LUKS-encrypted partition, configured the initramfs (installed build hooks), then installed grub bootloader and configured it to decrypt the cryptroot. I used this manual from official Arch Linux wiki. Now I am able to boot into the encrypted system using the cloud provider's console to enter the password for encrypted partition. All works great, I also can connect to the server via SSH as expected.
Now I want to be able to unlock the root partition remotely without using of the cloud provider's console, but using SSH instead. I got problems here. I followed this manual. The exact steps I did:
Installed packages:
pacman -S busybox dropbear mkinitcpio-dropbear mkinitcpio-utils mkinitcpio-netconfGenerated a key pair on the client for LUKS unlocking:
ssh-keygen -t rsa -f unlock_luksUploaded the public key (
unlock_luks.pub) to the server and moved it to/etc/dropbear/root_keyRegenerated OpenSSH keys in PEM format.
[root@host ~]# rm /etc/ssh/ssh_host_* [root@host ~]# ssh-keygen -A -m PEM ssh-keygen: generating new host keys: RSA DSA ECDSA ED25519Converted OpenSSH rsa key to dropbear SSH key:
[root@host ~]# dropbearconvert openssh dropbear /etc/ssh/ssh_host_rsa_key /etc/dropbear/dropbear_rsa_host_key Key is a ssh-rsa key Wrote key to '/etc/dropbear/dropbear_rsa_host_key' [root@host ~]# dropbearconvert openssh dropbear /etc/ssh/ssh_host_ed25519_key /etc/dropbear/dropbear_ed25519_host_key Key is a ssh-ed25519 key Wrote key to '/etc/dropbear/dropbear_ed25519_host_key' [root@host ~]# dropbearconvert openssh dropbear /etc/ssh/ssh_host_ecdsa_key /etc/dropbear/dropbear_ecdsa_host_key Key is a ecdsa-sha2-nistp256 key Wrote key to '/etc/dropbear/dropbear_ecdsa_host_key' [root@host ~]# dropbearconvert openssh dropbear /etc/ssh/ssh_host_dsa_key /etc/dropbear/dropbear_dsa_host_key Error: unable to create key structure Error reading key from '/etc/ssh/ssh_host_dsa_key'Inserted hooks
netconf dropbear encryptsshbeforefilesystemsinHOOKSof/etc/mkinitcpio.conf. Now it looks like:HOOKS=(base udev autodetect modconf block mdadm_udev lvm2 netconf dropbear encryptssh filesystems keyboard fsck)Installed and configured GRUB:
pacman -S grub grub-install /dev/sdaSet these options in
/etc/default/grub(/dev/sda3is the LUKS encrypted partition):GRUB_CMDLINE_LINUX="cryptdevice=/dev/sda3:cryptroot ip=dhcp" GRUB_ENABLE_CRYPTODISK=y
Network configuration on my server uses DHCP, so I use ip=dhcp as well.
Generated grub configuration file:
grub-mkconfig -o /boot/grub/grub.cfg
echo "cryptroot /dev/sda3 none luks" >> /etc/crypttab(/dev/sda3is the LUKS encrypted partition)Regenerated initramfs:
mkinitcpio -p linux. Output:==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'default' -> -k /boot/vmlinuz-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux.img ==> Starting build: 5.18.16-arch1-1 -> Running build hook: [base] -> Running build hook: [udev] -> Running build hook: [autodetect] -> Running build hook: [modconf] -> Running build hook: [block] ==> WARNING: Possibly missing firmware for module: xhci_pci -> Running build hook: [mdadm_udev] -> Running build hook: [lvm2] -> Running build hook: [netconf] -> Running build hook: [dropbear] Key is a ssh-rsa key Wrote key to '/etc/dropbear/dropbear_rsa_host_key' Error: unable to create key structure Error reading key from '/etc/ssh/ssh_host_dsa_key' Key is a ecdsa-sha2-nistp256 key Wrote key to '/etc/dropbear/dropbear_ecdsa_host_key' dropbear_rsa_host_key : SHA256:J9v2M8Lso02myd7Ah5Gk2itcNOq0dWVyARVSpBzx8R0 dropbear_ecdsa_host_key : SHA256:1c9URey3Z3JnUAWXKGFIVruzcWnhBtiqiQsEe/TdLXM -> Running build hook: [encryptssh] ==> WARNING: Possibly missing firmware for module: qat_4xxx -> Running build hook: [filesystems] -> Running build hook: [keyboard] -> Running build hook: [fsck] ==> Generating module dependencies ==> Creating zstd-compressed initcpio image: /boot/initramfs-linux.img ==> Image generation successful ==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'fallback' -> -k /boot/vmlinuz-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux-fallback.img -S autodetect ==> Starting build: 5.18.16-arch1-1 -> Running build hook: [base] -> Running build hook: [udev] -> Running build hook: [modconf] -> Running build hook: [block] ==> WARNING: Possibly missing firmware for module: qed ==> WARNING: Possibly missing firmware for module: qla2xxx ==> WARNING: Possibly missing firmware for module: bfa ==> WARNING: Possibly missing firmware for module: qla1280 ==> WARNING: Possibly missing firmware for module: wd719x ==> WARNING: Possibly missing firmware for module: aic94xx ==> WARNING: Possibly missing firmware for module: xhci_pci -> Running build hook: [mdadm_udev] -> Running build hook: [lvm2] -> Running build hook: [netconf] ==> WARNING: Possibly missing firmware for module: cfg80211 ==> WARNING: Possibly missing firmware for module: wcn36xx ==> WARNING: Possibly missing firmware for module: mt7603e ==> WARNING: Possibly missing firmware for module: rsi_usb ==> WARNING: Possibly missing firmware for module: rsi_sdio ==> WARNING: Possibly missing firmware for module: mwl8k ==> WARNING: Possibly missing firmware for module: usb8xxx ==> WARNING: Possibly missing firmware for module: libertas_sdio ==> WARNING: Possibly missing firmware for module: libertas_spi ==> WARNING: Possibly missing firmware for module: libertas_cs ==> WARNING: Possibly missing firmware for module: mwifiex_sdio ==> WARNING: Possibly missing firmware for module: mwifiex_usb ==> WARNING: Possibly missing firmware for module: zd1201 ==> WARNING: Possibly missing firmware for module: zd1211rw ==> WARNING: Possibly missing firmware for module: p54spi ==> WARNING: Possibly missing firmware for module: p54pci ==> WARNING: Possibly missing firmware for module: p54usb ==> WARNING: Possibly missing firmware for module: orinoco_usb ==> WARNING: Possibly missing firmware for module: rtl8723ae ==> WARNING: Possibly missing firmware for module: b43 ==> WARNING: Possibly missing firmware for module: b43legacy ==> WARNING: Possibly missing firmware for module: ipw2200 ==> WARNING: Possibly missing firmware for module: ipw2100 ==> WARNING: Possibly missing firmware for module: atmel ==> WARNING: Possibly missing firmware for module: at76c50x_usb ==> WARNING: Possibly missing firmware for module: mlxsw_spectrum ==> WARNING: Possibly missing firmware for module: nfp ==> WARNING: Possibly missing firmware for module: liquidio ==> WARNING: Possibly missing firmware for module: bnx2x ==> WARNING: Possibly missing firmware for module: bna ==> WARNING: Possibly missing firmware for module: softing_cs -> Running build hook: [dropbear] Key is a ssh-rsa key Wrote key to '/etc/dropbear/dropbear_rsa_host_key' Error: unable to create key structure Error reading key from '/etc/ssh/ssh_host_dsa_key' Key is a ecdsa-sha2-nistp256 key Wrote key to '/etc/dropbear/dropbear_ecdsa_host_key' dropbear_rsa_host_key : SHA256:J9v2M8Lso02myd7Ah5Gk2itcNOq0dWVyARVSpBzx8R0 dropbear_ecdsa_host_key : SHA256:1c9URey3Z3JnUAWXKGFIVruzcWnhBtiqiQsEe/TdLXM -> Running build hook: [encryptssh] ==> WARNING: Possibly missing firmware for module: qat_4xxx -> Running build hook: [filesystems] -> Running build hook: [keyboard] -> Running build hook: [fsck] ==> Generating module dependencies ==> Creating zstd-compressed initcpio image: /boot/initramfs-linux-fallback.img ==> Image generation successful
Here you can see a weird dropbear error:
Error: unable to create key structure
Error reading key from '/etc/ssh/ssh_host_dsa_key'
I don't know why this happens and if this can be my problem why I can't connect to dropbear SSH, because I use RSA key instead of DSA to establish the dropbear SSH connection, but anyway I attach some more details.
[root@host ~]# ls -l /etc/dropbear
total 16
-rw------- 1 root root 140 Aug 11 14:01 dropbear_ecdsa_host_key
-rw------- 1 root root 83 Aug 11 14:00 dropbear_ed25519_host_key
-rw------- 1 root root 1189 Aug 11 13:13 dropbear_rsa_host_key
-rw-r--r-- 1 root root 563 Aug 11 11:15 root_key
[root@host ~]# ls -l /etc/ssh
total 536
-rw-r--r-- 1 root root 505489 Apr 8 14:34 moduli
-rw-r--r-- 1 root root 1531 Apr 8 14:34 ssh_config
-rw-r--r-- 1 root root 3131 Aug 11 09:50 sshd_config
-rw------- 1 root root 668 Aug 11 12:57 ssh_host_dsa_key
-rw-r--r-- 1 root root 599 Aug 11 12:57 ssh_host_dsa_key.pub
-rw------- 1 root root 227 Aug 11 12:57 ssh_host_ecdsa_key
-rw-r--r-- 1 root root 171 Aug 11 12:57 ssh_host_ecdsa_key.pub
-rw------- 1 root root 399 Aug 11 12:57 ssh_host_ed25519_key
-rw-r--r-- 1 root root 91 Aug 11 12:57 ssh_host_ed25519_key.pub
-rw------- 1 root root 2459 Aug 11 12:57 ssh_host_rsa_key
-rw-r--r-- 1 root root 563 Aug 11 12:57 ssh_host_rsa_key.pub
[root@host ~]# cat /etc/ssh/ssh_host_dsa_key
-----BEGIN DSA PRIVATE KEY-----
MIIBuwIBAAKBgQDrcndi0o+A7wKsoc5jL57HgEOJwxKSYBMJWC5314/e3vJ3ADz9
8L4fYbRSmpJrSYvkQsHjALYJsAz7P3KsD6a1NqFlrgX/3aUeEaxQ6yGH1OdX7QeB
GwXSxHnGt07PgbkuQPulASu+d0XLXciCg65d+7HxNw+EKrf+h5qMh8tOHwIVAJD2
2Js/dX/DOC7GTO5EkRMZ9WcBAoGAedXP6N5g2LPh5NuT6dLZA6FMmSkS6RZ+/ofN
5gC1g/zDUcX1xctZzPcaQt94fY+lVLERyBb8kf1/nAFpoHTSEyzmlbw16DsFYYjG
T5E1cOd2NGnmR+mPyQJCkW95HvFdjQreomA1k7QKdndnIwIkjcyWCrZyMw4xydnT
WOUF+aYCgYEAlcX9jnnKzfUmP2bF5MkBeKsMzEYG8/SvECblhuM4J6+wCR5pOhSg
owToGa6pPIsIsLR/DXsHpd8xij+D0oRbKtc2Al3c8HbPJ8tcPfnPPjOpsSMd8YUv
dkj0zg5sIh6ujTcWM5BbDvGKpU+5eW0HHvHU6E/G9tet76/HKpgQtl0CFA/0lVMi
J5VrlqWYS/T94pZqwL8B
-----END DSA PRIVATE KEY-----
Reboot.
Server started and waiting for passphrase. Screenshot from cloud provider's console screenshot
I compared my output to output from this tutorial. It seems the only difference is that my output contains a line:SIOCADDRT: Network is unreachableTrying to connect to the dropbear SSH server from the client:
$ eval $(ssh-agent) Agent pid 20578 $ ssh-add unlock_luks Identity added: unlock_luks (<USER>) $ ssh -v root@<SERVERIP>
Output:
$ ssh -v root@<SERVERIP>
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: /etc/ssh/ssh_config line 21: Applying options for *
debug1: Connecting to <SERVERIP> [<SERVERIP>] port 22.
Never connects. Nmap says that SSH port is filtered:
$ nmap <SERVERIP> -p 22
Note: Host seems down. If it is really up, but blocking our ping probes, try -Pn
Nmap done: 1 IP address (0 hosts up) scanned in 3.08 seconds
$ nmap <SERVERIP> -p 22 -Pn
Nmap scan report for <HOST> (<SERVERIP>)
Host is up.
PORT STATE SERVICE
22/tcp filtered ssh
Nmap done: 1 IP address (1 host up) scanned in 2.12 seconds
Traceroute of nmap (sudo nmap <SERVERIP> -p 22 -Pn --traceroute) shows that the packet doesn't reach my server, but some other server instead. But when the disk is unlocked and the system fully started then the packet reaches my server as expected and nmap says that SSH port is open.
Hope for your help!
