Testing setup:
- Weblogic 12.2.1.4 running on a Windows 10 machine joined to an active directory
- JVM 1.8.0_281
- The java web application is using Java GSSAPI to access the fileshare over Samba essentially using the code from https://github.com/hierynomus/smbj/issues/304#issuecomment-375603115
- This is a developer machine - the java system property "user.name" shows the developer's windows username
- Linux fileserver running Red Hat Enterprise Linux 7, configured with sssd to connect to Active Directory
- Linux fileserver running Red Hat Enterprise Linux 6, not using sssd, not using winbind (unclear how precisely it is configured for Active Directory)
From what I've been able to gather third-hand on conversations with sysadmins, MIT Kerberos is somehow involved in connecting the linux servers to Active Directory - but I have no more information on that.
(note: serverfault is having markdown table rendering issue - table displays fine in preview, but not in actual posted question, so surrounded in code block for now so that it doesn't all run together)
Test Results
| Source (all on same Windows 10 machine) | Target FileServer | Result |
|------------------------------------------|-----------------------|---------|
| Weblogic application | RHEL 6 | Success |
| Weblogic application | RHEL 7 | **Fail:** Server not found in Kerberos database |
| Windows Explorer | RHEL 6 | Success |
| Windows Explorer | RHEL 7 | Success |
All tests done using the same user credentials.
The trace results from Weblogic (after setting the java system property sun.security.krb5.debug
to true
) are:
KrbAsReqBuild: PREAUTH FAILED/REQ, re-send AS-REQ
Using builtin default etypes for default_tkt_enctypes
default etypes for default_tkt_enctypes: 18 17 16 23
Using builtin default etypes for default_tkt_enctypes
default etypes for default_tkt_enctypes: 18 17 16 23
>>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
>>> KrbAsReq creating message
getKDCFromDNS using UDP
>>> KrbKdcReq send: kdc=***************. UDP:88, timeout=30000, number of retries =3, #bytes=233
>>> KDCCommunication: kdc=***************. UDP: 88, timeout=30000,Attempt =1, #bytes=233
>>> KrbKdcReq send: #bytes read=100
>>> KrbKdcReq send: kdc=*****************. TCP:88, timeout=30000, number of retires =3, #bytes=233
>>> KDCCommunication: kdc=****************. TCP:88, timeout=30000,Attempt =1, #bytes=233
>>>DEBUG: TCPClient reading 2695 bytes
>>>KrbKdcReq send: #bytes read=2695
>>>KdcAccessibility: remove **********************.:88
>>>Etype: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
>>>KrbAsRep cons in KrbAsReq.getReply ******
Found ticket for ******@******** to go to krbtgt/******@****** expiring on ******
Entered Krb5Context.initSecContext with state=STATE_NEW
Found ticket for ******@******** to go to krbtgt/******@****** expiring on ******
Service ticket not found in the subject
>>> Credentials serviceCredsSingle: same realm
Using builtin default etypes for default_tgs_enctypes
default etypes for default_tgs_enctypes: 18 17 16 23
>>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
>>> CksumType: sun.security.krb5.internal.crypto.HmacSha1Aes256CksumType
>>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
getKDCFromDNS using UDP
>>> KrbKdcReq send: kdc=************. TCP:88, timeout=30000, number of retries =3, #bytes=2633
>>> KDCCommunication: kdc=************. TCP:88, timeout=30000,Attempt =1, #bytes=2633
>>>DEBUG: TCPClient reading 104 bytes
>>> KrbKdcReq send: #bytes read=104
>>> KdcAccessibility: remove *************.:88
>>> KDCRep: init() encoding tag is 126 req type is 13
>>>KRBError:
STime is **********
suSec is **********
error code is 7
error Message is Server not found in Kerberos database
sname is cifs/***********@***********
msgType is 30
Searching for "Server not found in Kerberos database" yields a number of possibilities (DNS seems to be most common suggestion, other answers have suggested SPN registrations, TLS certs, not using FQDN, invalid host to realm mapping, host not part of domain, IPV4 vs IPV6)
The network admins say DNS is correct, which would appear to be substantiated by the fact that Windows explorer can connect to the RHEL 7 server just fine. But neither am I prepared to just blame the java code, since it does successfully connect to the RHEL 6 server.
I'm having difficulty finding a clear explanation of what Kerberos entries need to be configured where.
Is this "Kerberos database" on the Windows 10 machine, the fileserver, or the Active Directory KDC? Or are there multiple copies of this Kerberos database that each need entries?
Edit - Additional Details I've learned a few new things and can provide some additional details.
The following are all from a Window's command prompt on the Developer machine.
Realm is determined by:
C:\>echo %userdnsdomain%
DC1.DC2.DC3
For the successful connection (RHEL 6 server)
C:\>powershell Get-ADComputer server1
DistinguishedName : CN=SERVER1,OU=ou1,OU=ou2,OU=ou3,DC=dc1,DC=dc2,DC=dc3
DNSHostName : server1.dc2.dc3
Enabled : True
Name : SERVER1
ObjectClass : Computer
ObjectGUID : aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa
SamAccountName : server1$
SID : ************************************
UserPrincipalName :
C:\>setspn -L server1
Registered ServicePrincipalNames for CN=SERVER1,OU=ou1,OU=ou2,OU=ou3,DC=dc1,DC=dc2, DC=dc3:
HOST/server1.dc2.dc3
HOST/SERVER1
C:\>nslookup server1
Server: aa1.dc2.dc3
Address: 123.456.789.01
Name: server1.dc2.dc3
Address: 123.456.7.890
C:\>nslookup 123.456.7.890
Server: aa1.dc2.dc3
Address: 123.456.789.01
Name: server1.dc2.dc3
Address: 123.456.7.890
For the failing connection (RHEL 7 server)
C:\>powershell Get-ADComputer server2
DistinguishedName : CN=SERVER2,OU=ou4,DC=dc1,DC=dc2,DC=dc3
DNSHostName : server2.dc1.dc2.dc3
Enabled : True
Name : SERVER2
ObjectClass : Computer
ObjectGUID : bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb
SamAccountName : SERVER2$
SID : ************************************
UserPrincipalName :
C:\>setspn -L server2
Registered ServicePrincipalNames for CN=SERVER2,OU=ou4,DC=dc1,DC=dc2,DC=dc3:
RestrictedKrbHost/SERVER2
HOST/SERVER2
RestrictedKrbHost/SERVER2.dc1.dc2.dc3
HOST/SERVER2.dc1.dc2.dc3
C:\>nslookup server2
Server: aa1.dc2.dc3
Address: 123.456.789.01
Name: server2.dc1.dc2.dc3
Address: 12.345.6.78
C:\>nslookup 12.345.6.78
Server: aa1.dc2.dc3
Address: 123.456.789.01
Name: server2.dc2.dc3
Address: 12.345.6.78
Comparing the Weblogic trace results for the success vs failure:
success:
>>> DEBUG: ----Credentials----
client: username@DC1.DC2.DC3
server: cifs/server1.dc2.dc3@DC1.DC2.DC3
ticket: sname: cifs/server1.dc2.dc3@DC1.DC2.DC3
failure:
>>>KRBError:
...
error code is 7
error Message is is Server not found in Kerberos database
sname is cifs/server2.dc2.dc3@DC1.DC2.DC3
msgType is 30
So I note a few things:
- The successful (RHEL 6) server omits "dc1" in a number of places (but not all)
- The successful and failing
sname
differ only in the server portion - they otherwise line up on the "dc" values. - The two servers are in different Organizational Units (though I don't think this is the issue)
- There are a few difference in capitalization in the command results
- There appears to be a difference in the
nslookup
results forserver2
depending on whether it is looked up by server name (includesdc1
) or ip address (excludesdc1
)