How can I verify the hash of the plain text without being able to decrypt the cipher text?

Question

I am building an E2EE chat app where there is one asymmetric key pair per group. Each user also has one asymmetric key pair. All messages in a group chat are encrypted with the group public key and decrypted with the group private key.

When Alice is added to the group, Bob (one of the group admins) encrypts the group's private key for Alice with her public key. For want of a better term, let's call this encrypted form of the group private key the "encrypted group private key". The encrypted group private key can only be decrypted using Alice's private key. It is therefore safe to store on my server. Bob creates this encrypted group private key locally and then sends it up for storage on my server.

// on Bob's device
encryptedGroupPrivateKey = encrypt(data: groupPrivateKey, publicKey: alicePublicKey)

My server in this case is just a plain key-value store of encrypted private keys on the cloud. Later, Alice can download the encrypted group private key and decrypt it to reveal the raw group private key and use that to read messages of the group.

When Bob sends up the encrypted group private key to my server, how can I verify that what Bob has sent me does indeed contain the group private key and not some garbage? Otherwise, my server would dutifully store the garbage for Alice but when Alice comes to decrypt the encrypted group private key she finds out the contents are not the group private key and is not able to read the messages of the group.

My first thought is to hash the group private key when the group is created and store the hashed version on my server and somehow later compare this hash with that of the encrypted group private key. How might this work?

I'm looking to have a verify function on my server which is able to

// on my cloud server
verify(encryptedGroupPrivateKey, groupPrivateKeyHash) // returns true
verify("anything else", groupPrivateKeyHash) // returns false

Putting it all together and more generally this is the set of functions I'm looking for:

messageHash = hash(message)
cipherText = encrypt(message, publicKey)

verify(cipherText, messageHash) # returns true
verify("anything else", messageHash) # returns false

Answer 1

There are a few things to unpack here.

First, as @MechMK1 points out in the comments, private keys are normally not sent over the network (even after they've been encrypted). Usually in group chat applications, a random symmetric key is generated by one of the participants (e.g. Bob), then as other participants join (e.g. Alice), Bob encrypts the symmetric key using Alice's public key and sends the encrypted symmetric key to Alice. This way, all of the participants share the same symmetric key, and this symmetric key is used to encrypt and decrypt messages sent by participants to the group using a symmetric encryption cipher such as AES.

In this case, your server is simply acting as a conduit for passing encrypted messages and encrypted keys between the participants, and therefore your server has 'zero knowledge' of these messages and keys. If one of the participants tries to distribute a bogus key, the other participants will notices this when they try to use this bogus key to decrypt a message, as @schroeder pointed out. To this end, you might want to consider using an authenticated encryption algorithm (e.g. AES-GCM), which combines both integrity verification and confidentiality.

If you still prefer that your server does some sort of integrity checking, then the only other solution that comes to mind is that the group admin (e.g. Bob) must be trusted by the server, and Bob signs the encrypted key with his private key. Then, the server can verify the signature using Bob's public key.

Answer 2

What you are asking is for Bob (the prover) proves to a server (the verifier) that he is sharing the correct secret, while not revealing said secret.

This is called a Zero-knowledge proof.

You can read about the Fiat-Shamir heuristic which enables a non-interactive verification: Bob - or Peggy in the article - send a sets of elements to the server once, instead of having a back and forth with the server.

I have quickly checked for existing implementation of this on GitHub, and while some exists, this does not seems to be widely used, so I would take extra care using it ; and implementing your own cryptography is almost always a bad idea.