In all cases, the party which is being authenticated holds the private key (actually both keys, the whole set). The party which is verifying authentication only needs to know the public key.
Both SSH and TLS/SSL actually support authenticating both the client and the server, so each connection may use two different keypairs – one to verify the server's identity (server holds the private key) and one to verify the client's identity (client holds the private key).
For example, in SSH, the server's authorized_keys
file holds client public keys, but the client's known_hosts
file holds server public keys.
When you're being asked to verify the "hostkey fingerprint" when making an SSH connection, you're seeing the server's public key, and the server stores its private key in /etc/ssh. Once that step is done, the client presents its own public key to the server.
(The same thing exists in TLS as "client certificate authentication", but it is somewhat rare in comparison – but even though it's very rare on the web, many browsers support it anyway.)
1The private key is always kept by the party that generated the key pair. Public keys are handed out to everyone (the public) who might want to send something to the person with the private key. – Mokubai – 2019-12-05T18:57:58.910