log4j-over-slf4j only translates logging calls from LOG4J API to SLF4J API. But to really create log entries, you need a library that implements SLF4J API, for instance, Logback. In such case the application code believes the logging is done by LOG4J, but actually it is done by Logback. It looks as follows:
your code -> LOG4J API -> log4j-over-slf4j -> Logback -> file (database, Splunk, Elastic Search, ...)
To your question:
How can I know whether the "underlying implementation" is log4j 2.x?
If you developed the application you refer, then you must know how the logging is implemented in your application, and you know if particular library is used or not.
If you have not developed it, then check if the affected class is contained in the application binaries. Search for file "JndiLookup.class" in all files of your application, unpackaging all archives: .jar, .zip, .war, .ear etc. Don't be surprised if you find it not in "log4j-core-2.*.jar", but in "log4j-core.jar" or even in "<your application>.jar" (in case "fat jar" is used). If the file exists, find out its version. Then you know if you have a problem or not.
If the application uses standard libraries without repackaging them, e.g. without making a "fat jar", then the class in question is contained in log4j-core.jar or log4j-core-.jar. If you don't develop and don't know the structure, here is the source code. As you can see, class "JndiLookup.class" is contained in the module "log4j-core".
You write that the only have "log4j-over-slf4j". Thus you don't have "log4j-core" and your application is not affected by the log4shell problem.