I'm trying to set up a Windows 2019 system as an SMB server to work with third-party software on an external, non-Windows system that uses Kerberos to authenticate to the SMB server. This SMB server is joined to an existing Active Directory domain on a separate Domain Controller server. Although the software is successful in getting both user and SPN credentials from the Windows AD Kerberos server, the SMB server rejects those credentials with the error KRB5KRB_AP_ERR_MODIFIED.
The SMB user is defined in AD, and there is a computer account for the SMB server in AD as well. Although the SPN list in the DC account does not have this SMB server, the SPN list in the computer account for the SMB server does. I tried to manually add an SPN entry for the SMB server on the DC, but it was rejected because it's a duplicate, so I presume that the DC takes the SMB server computer account into consideration when looking for duplicates.
I've seen this kind of problem before, and it's usually an issue with DNS entries where the SMB server thinks its host name is one thing but the Kerberos server and/or third-party software think it's something else. In this case, though, the situation is slightly different--the rejection in the Session Setup Response shows the SMB server name as its short host name followed by a dollar sign. This is despite the fact that the full computer name of the SMB server on the SMB server is its DNS FQDN (this is according to the System program in the Control Panel). I also verified that the DNS server (which is also the AD DC) correctly resolves the SMB server's host name and IP address.
Here is the sequence of events:
- The client software on a non-Windows system connects to the SMB server over TCP and, using SMBv2, successfully performs protocol negotiation.
- The client software requests credentials for the SMB user from the Kerberos server (AD DC) via an AS-REQ and receives a positive AS-REP with the credentials. (I verified that the correct Kerberos realm name is used, and that the sname values in the request and response match up.)
- The client software then requests credentials for the SPN of the SMB server in a TGS-REQ sent to the Kerberos server. The SPN value it uses is "cifs/DNS_FQDN", where "DNS_FQDN" is the fully qualified DNS host name of the SMB server. It receives those credentials in a positive TGS-REP. (I verified that the correct Kerberos realm name is used, and that the sname values in the request and response match up.)
- The client software puts those credentials in a GSS-API security blob inside an SMB Session Setup Request and sends the request to the SMB server.
- The SMB server rejects the credentials with the error KRB5KRB_AP_ERR_MODIFIED. The realm name in the security blob in the Session Setup Response packet is correct, but the sname string of the SMB server in the response is the short host name of the server followed by a dollar sign.
Clearly, I haven't configured something correctly. The third-party software works with SMB shares on the AD DC; it just doesn't work with shares on the separate SMB server, and then only because the SMB server rejects the credentials given to the third-party software by the Kerberos server.
What am I doing wrong? Should I remove the SMB computer account on the AD DC and then manually add an SPN entry for the SMB server on it? Is there something I need to add to or modify in the SMB computer account? Do I need to change something on the SMB server so that it recognizes "cifs/DNS_FQDN" as being valid?