The first thing you may want to do is check that your server is presenting it's certificates properly. You can do this by trying to connect to your server using OpenSSL. On a client machine with access, run:
openssl s_client –connect target_server_fqdn:636
This should return a nice print out of the server's certificate. The key here is checking the "Verify return code" printed at the end. You may get different codes, but generally speaking, you should get 0 for a valid certificate, and possibly 19 if you're self-signing.
If this fails, go back and check to ensure you have imported your server side certificates properly.
If you've passed this test, move on to testing your TLS connections from the client side.
On a client machine, run
ldapsearch -z -ZZ '(uid=<testusername>)'
This will force an LDAP lookup over an encrypted connection. If that's successful, you should get some user information back, and a check into the DS logs should yield the following:
[23/Sep/2011:07:48:57 -0500] conn=1631 op=0 EXT oid="X.X.X.X.XX.X.XX" name="startTLS"
[23/Sep/2011:07:48:57 -0500] conn=1631 op=0 RESULT err=0 tag=120 nentries=0 etime=0
[23/Sep/2011:07:48:57 -0500] conn=1631 SSL 256-bit AES
If this fails, you'll want to ensure the certificates were properly imported on the client side.
When troubleshooting, some common areas I've found myself looking frequently are:
1.) On the clients, in some cases (which someone here may be able to better explain), you might try to require signing by editing ldap.conf and including the line
TLS_REQCERT allow
2.) If the authentication GUI is giving you problems, you might try just explicitly turning on TLS for LDAP with
authconfig --enableldaptls --update
I've had problems with the GUI before, so I tend to stick to using CLI commands.
3.) And a final thing you might try (again , just for testing), is calling
cacertdir_rehash <dir where certs are stored>
Update
If you're looking for more help in actually creating self sign certificates, try the following:
1.) Create your own, self-signed CA Certificate:
certutil -S -n "<CA Certificate Name Here>" -s "cn=<CN Name Here>, dc=<Your DC's FQDN>" -2 -x -t "CT,," -m 1000 -v 120 -d . -k rsa
2.) Create a server certificate for the directory server
certutil -S -n "Cert-Name" -s "cn=<Server FQDN>" -c "<Name of CA Certificate>" -t "u,u,u" -m 1001 -v 120 -d . -k rsa
3.) Import both of these certificates into the directory server in the "Manage Certificates" section, selected under "Tasks"
4.) Enable TLS encryption
5.) Create an exportable certificate for clients and output it to a .pem file
certutil -d . -L -n "<CA Certificate Name>" -a > cacert.pem
6.) By means of your choosing - download the client certificate onto each client.
7.) Rehash the certificates by using the previously mentioned command
cacertdir_rehash <dir where certs are stored>
