Is it conceptually possible to allow in the server a specific self signed client certificate for mutual TLS?
If possible but not recommended. Why?
I have a client to who I have to provide a server that does mutual TLS auth. But they say they wont sign our server certificate nor will they let us sign their client certificate. How should I approach this?
Is it conceptually possible to allow in the server a specific self signed client certificate for mutual TLS?
Yes. A self-signed certificate is nothing special. Using the trust chain against a trusted root CA is not the only way a certificate can be verified, but one can for example simply explicitly trust the given certificate or the public key inside it.
Note that a self-signed certificate still need to be verified against the expected value. Blindly trusting arbitrary self-signed certificates (as done in many examples) is insecure. How such validation needs to be implemented is different between programming languages and frameworks.
If possible but not recommended. Why?
Certificates issued by an already trusted CA can rely on this CA as the trust anchor. For self-signed certificates there is no such pre-existing trust anchor. In this case the certificate, its fingerprint or similar must be distributed using some existing trusted mechanism to all the peers which should trust the certificate. And it must be removed from all these clients if it needs to be revoked. This is doable with a few peers but does not scale to many.
I have a client to who I have to provide a server that does mutual TLS auth. But they say they wont sign our server certificate nor will they let us sign their client certificate. How should I approach this?
As I said, self-signing certificates are perfectly acceptable to use if the trust relationship was distributed over a trusted medium. It simply does not scale well with many parties and it might be a problem to properly verify the certificate in the applications. But, if this is not a problem in your specific case, then self-signed certificates can be used.
Yes, it's entirely possible for clients to authenticate with your web server using self-signed client certificates.
See https://cweiske.de/tagebuch/ssl-client-certificates.htm for a good write-up on how this is done using Apache and PHP. In this write-up, the author uses a CA-signed client cert. However, there is no reason why the client certificate could not be self signed. To determine whether or not to authenticate a client based on the client's certificate, the server checks the values in a few of the fields in the leaf certificate and compares these with the expected values for that client. If the values in the certificate match the expected values, the client is authenticated. (*Of course, the client must be able to complete a TLS handshake with the server, proving that it has possession of the private key that corresponds with the certificate). But, to answer your question, other certificates higher up in the chain are not relevant, so it doesn't matter if the client certificate is self-signed or signed by a CA.
