I have an interesting case regarding protection against replay attacks. Under the assumption that Alice and Bob have used Diffie-Hellman key exchange to establish a secure temporary session, would it be safe for them to use the public portion of their DH keys as nonce seeders in combination with HMAC to protect from replay attacks? In pseudo code -
Establishing a session:
a:
(dhA, dhSecret) = DH_GEN(common)
a->b (dhA)
b:
(dhB, dhSecret) = DH_GEN(common)
b->a (dhB)
a:
a<-b (dhB)
sessionKey = DH_KEY(dhB, dhSecret)
b:
b<-a (dhA)
sessionKey = DH_KEY(dhA, dhSecret)
Now when Alice wants to send some data to Bob:
a:
data = ENCRYPT(sessionKey, data)
dhA++
hmac = HMAC(sessionKey, data | dhA)
a->b (data, hmac)
And on Bob's side the process is reversed:
b:
b<-a (data, hmac)
if (hmac == HMAC(sessionKey, data | dhA+1)) {
dhA++
data = DECRYPT(sessionKey, data)
}
The same thing happens in the opposite direction:
b:
data = ENCRYPT(sessionKey, data)
dhB++
hmac = HMAC(sessionKey, data | dhB)
b->a (data, hmac)
a:
a<-b (data, hmac)
if (hmac == HMAC(sessionKey, data | dhB+1)) {
dhB++
data = DECRYPT(sessionKey, data)
}
And the second question - would it be easier and just as safe to use a self-incrementing nonce starting from a fixed number (say 0) instead of using fairly large DH public keys for the initial nonce?
What are the perils (if any) of such approach, aside from a possible MITM attack during the session establishment which can be thwarted using some form of authenticated Diffie-Hellman KE (e.g. having the generated public keys signed by trusted certificates during the exchange)?
Thanks