9

Any time I stand up a VM in EC2 and you ssh into it for the first time, I always get this message:

The authenticity of host 'ec2-xxxxxx.compute-1.amazonaws.com (n.n.n.n)' can't be established.
RSA key fingerprint is xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx.

And like everybody else, I just say yes

Are you sure you want to continue connecting (yes/no)? yes

I understand what a fingerprint is, why it's good and all that. What I'm wondering is, years ago, since it was a physical machine I was setting up... I could check on the physical machine and validate yes, this is the fingerprint.

Is there some way in the EC2 console to independently verify "yes, this is the fingerprint"? If so, how do you find it?

slf
  • 247
  • 2
  • 7

2 Answers2

13

You can verify the fingerprint using the AWS console for instances with cloud-init.

The following is on an instance running Amazon Linux.

There is a init.d script called cloud-init:

cloud-init is the distribution-agnostic package that handles early initialization of a cloud instance.

Some of the things it configures are:

  • setting a default locale
  • setting hostname
  • generate ssh private keys
  • adding ssh keys to user's .ssh/authorized_keys so they can log in
  • setting up ephemeral mount points
  • preparing package repositories and performs a variety of at-boot customization actions based on user-data

Once you launch an instance, you can view the system log output via the AWS console without using SSH. (Which avoids your catch-22 - you can see the fingerprint before you access the instance).

You can do this by

  1. going to your EC2 Management Console page,

  2. clicking the "Instances" link on the sidebar,

  3. selecting the instance you want to log into, and

  4. navigating to Actions > Instance Settings > Get System Log.

Get System Log

If you scroll through that output, you will see something like the following:

Running cloud-init
...

cloud-init:  sshGenerating public/private rsa key pair.
Your identification has been saved in /etc/ssh/ssh_host_rsa_key.
Your public key has been saved in /etc/ssh/ssh_host_rsa_key.pub.
The key fingerprint is:
aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa root@example.com
The key's randomart image is:
+--[ RSA 2048]----+
|       aa        |
|  a    a         |
|     aa   a   a  |
|  a    aaa       |
|     aaaaa   a   |
| a   a aa        |
|aaaa   a  aa     |
|aaaaaa     a     |
| aa      a       |
+-----------------+
Generating public/private dsa key pair.
Your identification has been saved in /etc/ssh/ssh_host_dsa_key.
Your public key has been saved in /etc/ssh/ssh_host_dsa_key.pub.
The key fingerprint is:
bb:bb:bb:bb:bb:bb:bb:bb:bb:bb:bb:bb:bb:bb:bb:bb root@example.com
The key's randomart image is:
+--[ DSA 1024]----+
|                 |
|                 |
|                 |
|       b  b      |
|        b     b  |
|    b    b b     |
|  bb b bb        |
| b  bbb bb    b  |
|  bbbb   bbb     |
+-----------------+
ec2: 
ec2: #############################################################
ec2: -----BEGIN SSH HOST KEY FINGERPRINTS-----
ec2: 2048 aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa  root@example.com (RSA)
ec2: 1024 bb:bb:bb:bb:bb:bb:bb:bb:bb:bb:bb:bb:bb:bb:bb:bb  root@example.com (DSA)
ec2: -----END SSH HOST KEY FINGERPRINTS-----
ec2: #############################################################
[  OK  ]

System Log Output

Above each "randomart" image is the type of signature, such as RSA or DSA. Find the kind you are given (RSA in your case), and check the fingerprint above it.

If you wish to verify that the same key is the one on the instance (as a proof of concept, after you have SSH'd in), you can run:

ssh-keygen -lf /etc/ssh/ssh_host_rsa_key.pub
2048 aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa  root@example.com (RSA)

Alternate Approach:

In order to avoid the problem, you can use your own keys - since you generated the key, you know the fingerprint. The issue then becomes one of getting your new key onto the instance without using SSH.

Most instances use cloud-init and will support user-data. This applies to any such instance. Since you need to stop an instance in order to modify user-data, this approach requires that you either are a: launching an instance for the first time (and are setting yourself up to use known keys) or b: can stop the instance, modify the user-data, and restart the instance.

  1. Generate a key For example, using PuttyGen on Windows:

  2. Create a user-data script:

    If you are launching a new instance, you just need to specify the keys you want to use, if you are restarting an instance that is already running, you need to have cloud-init reconfigure SSH to pull in the new keys. By default, the SSH config module of cloud-init is run once per instance, so you need to set it to run always (each boot) (This might not be ideal in some circumstances, but can be modified after you know the key, if required)

    The user-data would take the following form:

    #cloud-config
    cloud_config_modules: #only needed if restarting an instance, omit if launching a new instance
      ...list all existing modules
      - [ssh, always]  #this is changed to always (default is once per instance)
    
    ssh_keys:
      rsa_private: |
        -----BEGIN RSA PRIVATE KEY-----
        -----END RSA PRIVATE KEY-----
    
        ssh-rsa #public_key
    For example:
    #cloud-config
    cloud_config_modules:
      - locale
      - [ssh, always]
      - set-passwords
      - mounts
      - yum-configure
      - yum-add-repo
      - package-update-upgrade-install
      - timezone
      - puppet
      - disable-ec2-metadata
      - runcmd
    
    ssh_keys: #you can specify rsa, dsa, and ecdsa keys
      rsa_private: |
        -----BEGIN RSA PRIVATE KEY-----
        MIIEoQIBAAKCAQEAopbE8beKaKajF/SFOtntO9xt5XVZW5rlQCW6PVY1jXCq5dbj
        nEQoBGBIp6jsqLcnwYQW/tU4zXi7T0kX6NlVywiMOtjnyoOkLCX2R5OjMap3hlyj
        AO/PCKW7pE4vAHd7HyYvGW/gPezGW0WeFshp7J7dTXZdSmDquZI15rEsz07QsKWy
        /SH/rjYVObAQJN78CuU7C41LRshEeTSBM0jBSnp3jL1Ocw66qe4sV6jbcQN6QzK3
        77e+KzpUDmcxaB7plTWDSpjxVFWbY6PQcsz5d/h60wSKu90Ia9fNMHWs7cbyELhK
        VPBRs4JtWKndjtISCd5T34UnKmtTpq6g/ocrrwIBJQKCAQB7Ck/Zg/oKAZAtzcyb
        PSI7I1oVbY+68cI+Yb1e2XSiYxmLVoK7cdkYEYMXGA0KDhA/auD4MqeGvDq4ildI
        bR5UdSvZgYzQmvjHdqyJMXSUSaaPMVjCcEmlrdobeW+tU3/Ei5lDrpvb1caKQoV5
        Bl3/LB0YBovJlXNb//FwTrogVhYFexcda+DxN5a2oNSCwMosdgCP4gz+hX9zTAl9
        k/VOkLaj2h7URfsuAmwwZ+m24Bpz1r7vEtec0PraoKkBpVxBeNDPwMdosTrpGS49
        V+YRRiM8yShuRPF9mAwo62kcD7K5bToppyb6CLdCi06CTcAmQ5Sb+UwHqC/rdZI0
        wmnNAoGBAPxj6ecjM0AwrPf2TPJOtdEUHvFnc6bB23C32Yr7IWjNhij0BGG/D8Cv
        smCXDwYDH7Ss4CN/mMzG43QhfyyAz0T0BjpFmZEYqYOJAB7cwpdx4zjHzoc7WKiI
        vXPml2hdd37iVRNq6raUgDLpKfVkpY8FKcJjzFuiCXDOU1+mNxPbAoGBAKTqD/+X
        oDGsf6hkV7vgPLIXc3/BZco0l9kNkemto9RVIsr3D40bfe4PuJg3fjwFYDTq9s79
        WFR5sG/eSpNJtSGTz6LN5TQoL5xLMCIysajc+JN64w4TYCDGSEk4Xgv0X0cCgJfF
        RCedv9qObGT9/KCI/9Y/5jlZsVphNsAk4Xm9AoGBAKO2bjUP6eRymbWY1/ceTGvx
        YC3iPS3lh2u1hjCi5T0PsPf4OjGQsEWiZd3JxI5HNykWMIW6jKCBAj19guxvOlY9
        a9LFXLEkwPtfyLoSp7wuMoWyCWx5hZ3AeuNlI/CrVG36mAyYYOUiDfeCfBTLqajg
        wSQlDu9UWSaTq7OqFeNdAoGAFkkkway0yHFBrvjNle3esEhbt1F8dUVgoMp7gHFp
        KogLnuMdxvXglcrF6w5rATEorTSCN6Wx/ZPnaRAzl1z8zS+mb/JPZ+nBPqJgc1L1
        adir+EEJ7SU2gPgzSCo2OPeCfzewgzZVUXYurtT55CJSkjwG5Znurc3ZskR9BTVq
        k+kCgYARZXWS0Wyy5piq7WX7w2Hc6bVEPMCU+yJcbJ8E+F8meoQ1kXIRqIB+cAsq
        /3z3JmU19yOGms557POXgEFseMMFai3i2wTQ4mGPOM3a7yEeeKl1Zg5eHVUdVd+n
        PDzs9D+9umc6mXrRiRwJPQWo8pIKXb7SqjIA73M7H+98CBtb5w==
        -----END RSA PRIVATE KEY-----
    
        ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAopbE8beKaKajF/SFOtntO9xt5XVZW5rlQCW6PVY1jXCq5dbjnEQoBGBIp6jsqLcnwYQW/tU4zXi7T0kX6NlVywiMOtjnyoOkLCX2R5OjMap3hlyjAO/PCKW7pE4vAHd7HyYvGW/gPezGW0WeFshp7J7dTXZdSmDquZI15rEsz07QsKWy/SH/rjYVObAQJN78CuU7C41LRshEeTSBM0jBSnp3jL1Ocw66qe4sV6jbcQN6QzK377e+KzpUDmcxaB7plTWDSpjxVFWbY6PQcsz5d/h60wSKu90Ia9fNMHWs7cbyELhKVPBRs4JtWKndjtISCd5T34UnKmtTpq6g/ocrrw== rsa-key-20140716

  3. Stop your instance and modify the user-data:

  4. Start the instance, and connect to it:

    Note that the fingerprint shown here matches the one displayed when the key was generated.

If you are stopping the instance and have access to another instance, you can also mount the root volume and modify the keys directly without using user-data (the keys will not be overwritten since cloud-init only runs SSH config once per instance by default). An alternate approach, if you are planning ahead, is to setup cloud-init (or an init script) to log the SSH keys each boot, which increases the probability you will find them in the console log after a restart.

cyberx86
  • 20,620
  • 1
  • 60
  • 80
  • 1
    But this works only for the first run of the instance, when the log includes key generation. When you restart the instance you cannot find the fingerprints there anymore. Any other way? – Martin Prikryl Jul 16 '14 at 04:58
  • You shouldn't need to explicitly check the fingerprint any other time, since it shouldn't change. In most cases, your SSH client will remember the fingerprint if it matches the fingerprint you previously approved - which means that if you restart (when the key shouldn't change), the fingerprint won't change and your SSH client already knows the key. If the key does change, it either should be in the logs (new key generation), or something is wrong. Of course, once you are logged in, it is easy to get the fingerprint (if logs are cleared, etc.) – cyberx86 Jul 16 '14 at 18:43
  • 1
    As an aside - I think (haven't had a chance to confirm) that the fingerprint is actually displayed independently of the key generation - and should be displayed on every boot, not just the first one. Cloud init runs 'ssh-authkey-fingerprints' as part of the cloud_final_modules section of /etc/cloud/cloud.cfg.d/00_defaults.cfg - which will run /usr/libexec/cloud-init/write-ssh-key-fingerprints – cyberx86 Jul 16 '14 at 18:45
  • But what if i loose my SSH client cache for whatever reason (I can loose my notebook, my hdd can crash, anything). Also, I checked and I'm pretty sure the fingerprints are not logged on consecutive starts. – Martin Prikryl Jul 16 '14 at 19:19
  • 1
    Anyway, see my solution at http://stackoverflow.com/a/24773982/850848 – Martin Prikryl Jul 16 '14 at 19:19
  • True enough - the best solution in that case is to use your own keys (see my edit, above), or setup your instance to write the fingerprints to the logs on each boot. – cyberx86 Jul 17 '14 at 06:05
  • Using own keys is also a solution, thanks for the guide. But for example in case you have clients connecting to the server, you cannot change the keys. But you cannot ask your clients for the fingerprint either, it wouldn't look good :) – Martin Prikryl Jul 18 '14 at 05:56
-1

I could check on the physical machine and validate yes, this is the fingerprint.

Yes. On the server cat /etc/ssh/ssh_host_rsa_key.pub. You can even copy this into your ~/.ssh/known_hosts file on your client to avoid ever seeing the "error" message.

Is there some way in the EC2 console to independently verify "yes, this is the fingerprint"?

Yes, either manually as above, or you can publish the key to DNS and have the ssh client verify it that way (note: you'd only want to rely on DNS if you have DNSSEC configured properly for your domain).

You can configure the client to do the DNS lookup automatically via the configuration file option VerifyHostKeyDNS or on the command line with something like ssh -o "VerifyHostKeyDNS Yes" user@system.example.com. See man ssh_config for more info.

As a side note, you can configure public keys in DNS for a variety of such services - HTTPS and GPG spring to mind, though HTTPS certificate stapling is not enabled by default in any major web browser.

Chris S
  • 77,337
  • 11
  • 120
  • 212
  • This sounds like a no. Both of those are a catch 22. I have to ssh into the server to do those things. – slf Mar 11 '14 at 22:14
  • My apologies, I should not have assumed you knew the thumbprint was dumped in the logs. – Chris S Mar 12 '14 at 13:36