I question the logic of this question, as even if you are only allowing Linux boxes to connect, there is nothing to stop those Linux boxes natting Windows boxes etc. I believe that doing this for security reasons must be misguided, particularly as OpenVPN is opensource, and can thus be hacked to appear to be something its not.
You haven't specified the parameters for the OpenVPN clients, so the question is open to interpretation. You could, for instance, allow the client to connect but then force a connection (via SSH or somesuch) back to the machine which connected and run some tests to ensure it is a Linux machine.
Alternatively, depending on how robust a solution you need, you could also do a "Fingerprint Scan" scan for OS using, for example, NMAP OS Detection.
Another option, depending on your usage case would be to use parameters which are only implemented under Linux. You might be able to set up P2P tunnel with 192.168.1.1 on the one end and 192.168.1.5 upwards on the other end - I suspect Windows will not handle this correctly while Linux will.
Hi David, thanks for the response. Yes, I know a VM could be natted inside Linux, but I'm not trying to prevent such determination. Do you know how I can add a script that runs NMAP & pareses the output to ID the OS? I know how to create the script, the thing is how I can make the OpenVPN server run it & wait for a response. Thanks. – Just Starriol – 2013-10-07T13:35:29.017