Azure Key Vault does not only offer crypto solutions. It is also a user-based secret store. This solves one problem of secret management, which is quick remediation. Imagine that you have a SQL database that has to be accessed by multiple hosts, let's say hosts A and B. If the database password is compromised (typically because one host was attacked) and you put them in environment variables at both hosts, you have to heal both hosts.
With a user-based secret store, A and B have distinct credentials. If A is attacked, the attacker may be able to retrieve the SQL database password, but you can immediately revoke the authorization of A from the Key Vault, change the password at SQL database, and update the Key Vault with the new password. Note that no action was performed on B. When B identifies itself to the secret store, it will retrieve the updated SQL password and resume operation.
Of course, if your database password is used by single web app, then this doesn't help much. Still, you may have crypto applications that benefit from never seeing the private key, as the other answer explained.