Model: bulk transfer of data from one server to another, you'd want to prevent information leakage by a MITM attacker
Solution: TLS or any protocol over TLS such as HTTPS is a good choice. You'd want to make sure you have proper authentication. Server need to authenticate client, and client need to authenticate server. Two common ways to do this: 1) TLS mutual authentication, 2) TLS server certificate authentication + API Key within message headers. Other possible solution is alternative stream encryption like SSH (scp or sftp) or IPSec VPN.
Model: low latency messaging or multimedia streams, threat model as above
Solution: DTLS with Web Socket or other protocol designed for low latency like XMPP. Authentication concerns as above.h
Model: the sender server don't necessarily trust the recipient server, you'd want to be able to mutually prove that a message comes from the sender server and hasn't been tampered by the recipient
Solution: the sender need to cryptographically sign the messages, e.g. using GPG. Alternatively, you might want to use a merkle chain/block chain to prove message sequencing. You might want to use trusted time stamping as well.
Model: you want to have zero trust on any third parties, such as public/commercial CA
Solution: run your own CA, use self signed certificate. Now you're responsible for all the verification process that a CA would normally do.
SSH may be more suitable for this purpose, but you'll need to do your server fingerprint validation properly, otherwise you expose yourself to first use attack due to TOFU.