Securing non-consumer machine-to-machine APIs with Encryption over SSL

Question

I'm working on an API that is intended to be consumed by another system. The data is not owned by the requesting machine and the user(s) that the data pertains to are not driving the transaction. You could think of it as the same use-case as a hospital web service that transfers sensitive data and the people that own the data are not involved in the transaction. Or a banking web service that is used to perform analytical queries against lots of user data but the users in the data are not driving the transaction.

Problem

I'm researching authentication & Authorization options for this use case but nearly every document I come across for securing APIs jumps to OAuth 2 and sometimes OpenID Connect. I don't think either are applicable, if I read the RFCs correctly. They seem to center around delegating authentication to a 3rd party with the assumption that a user will then authenticate.

Solution I'm Considering

I've been considering an HMAC token in the Authorization header of the request in which the request details are encoded and signed with a key the client app was given once. Same flow as AWS's authentication scheme for their apis. The only thing I can think of more secure than this is exchanging public encryption keys and having the client encrypt the request with my public key and having me encrypt the response with theirs (PGP/RSA Encryption). Either option would be over TLS with certificate verification. Am I re-inventing the wheel here? Is there a comparable standard of rules to follow for this use case?

Side-note about Encryption over SSL

I've heard it remarked once that encryption over SSL was redundant. I just watched Moxie Marlinspike tear apart SSL at Defcon through a man in the middle and certificate chain spoofing. There is definitely a big difference between the certificate exchange process included in SSL that happens on every session and the one that only takes place once for what I outlined above.

Answer 1

You need to separate authentication from authorization. OAuth 2.0 and OpenID Connect is a system for managing primarily Authorization, not Authentication.

HMAC is a method for symmetric signing. It gives you integrity and partial authentication, but not non-repudiation.

Pubic key encryption is asymmetric encryption. It gives you confidentiality . Note however that encryption by itself doesn't guarantee integrity or authentication. To add these aspects to a PKI encryption system, digital signature and certificates are used. Digital signature also provides limited non-repudiation.

There's really no difference between users to machines, user to user, or machines to machines auth. A machine to machine communication is less likely to complain about the inconvenience of some schemes, but the tactics used to mitigate specific threats models are exactly the same.

Most user-to-machine authentication uses password. In machine-to-machine, this is called pre-shared key (PSK) authentication. TLS can be configured to use TLS-PSK, though this is a relatively rare configuration.

A PSK can also be used to create a message authentication code (MAC). An MAC asserts that the data hasn't been altered in transit. TLS signs the data with MAC for integrity, or you can add HMAC to any messages in any protocol, but it's not a replacement for non-repudiable signature.

In some higher security settings, the system might need a scheme that prevents requests/document from being tampered and prevents the user/machine from later denying the request/document. This non repudiation is provided by digital signature. You can use GPG or S/MIME signature for this. Note that there's limitation on non repudiation from digital signature, in that the signer can just claim that their private key is compromised. In many situations, claiming key compromise often have less severe consequences than admitting to have signed the request/document.

An asymmetric encryption can be used whenever you need asymmetric capability. For example, if you want to allow a machine to be able to write to a data storage, but prevent said machine from reading stored data in the future (or prevent compromise of said machine to allow attacker to decrypt the stored data), then you can use asymmetric encryption.

In many distributed system scenario, you'd also want to be able to prove sequencing or timing of events. Many systems used blockchain to prove sequencing of messages. Alternatively, some may use a timestamping authority. A widely distributed blockchain, like bitcoin's block chain, can also be used as a decentralized timestamping authority.

Answer 2

Either option would be over TLS with certificate verification. Am I re-inventing the wheel here? Is there a comparable standard of rules to follow for this use case?

Your basic concern seems to be one you could in fact guard against by using key pinning - that would guard against the kind of MITM/spoofing you mention in your question.

Assuming of course it is correctly implemented.

While adding another layer of encryption just for the sake of it would seem redundant, there may be cases where it could be warranted - you need to see if this is your case. For example, I can imagine there may be cases where you might want to store some data in its encrypted form, so that only authorised consumers can read it.

That, though, has nothing to do with why you would want to use TLS.

You could consider JSON Web tokens for some of your underlying needs within your application. It would seem to be a fairly close match for:

an HMAC token in the Authorization header of the request in which the request details are encoded and signed with a key the client app was given once