The answer tends to involve encryption of the secrets. This is an overly simplistic so let's explore that further, because different solutions approach the problem quite differently.
There are (as my current research is telling me) largely two or three common tools for this:
- Sealed Secrets
- Helm-Secrets (which is Helm-specific, but uses Mozilla SOPS, which is just about secrets and YAML etc., and thus not Kubernetes specific)
- KSOPS (which is a Kustomize plugin to use SOPS)
I don't have any practical experience with these at this time, but I can tell you that one of the biggest differences is that with Sealed Secrets, the decryption is done inside the cluster at the time when the SealedSecret object enters the system. The keying material lives inside the cluster. The encrypted SealedSecret (a type of CRD) can live happily in Git with whatever confidence you have in the encryption. Developers don't require access to the private-key, but can't view the contents of the secret [without it].
Compare this to SOPS, whereby the decryption occurs on the client-side, and the secret will generally be maintained in some external vault, such as HashiCorp Vault, Amazon KMS, or similar.
Comparing the two, one significant aspect would appear to be the use of public CI/CD infrastructure, and which parties would need to handle your secrets; this is one area in which Sealed Secrets has an advantage, but there is pros and cons either way, particularly in terms of what access all/some of your developers might require.
Unsurprisingly, both SOPS (in one of its guises) and Sealed Secrets are both widely used, which explains why the likes of Kuberenetes (or even opinionated distributions such as OpenShift) don't come with any particular opinions with regard to how to do it; which is frustrating!
If you're familiar with Ansible Vault, you could use SOPS with a PGP key in a similar way, potentially alongside the use of other KMS solutions.