You'll have to ask your ISP to setup the PTR for you.
You can have something like mail.example.com for your mail server and have that as the PTR. It is important because many mail servers will otherwise flat out refuse your emails (like mine for example.)
Otherwise you can test that the mail.example.com is valid by pinging it to make sure that the DNS works as expected. If you are under Linux, you can use dig (with my domain):
dig -t MX m2osw.com
That will show you something like this:
m2osw.com. 86400 IN MX 10 mail.m2osw.com.
Which looks like it is working.
To see the IP address and make sure that it points to the correct server:
dig mail.m2osw.com
Note that when you change an IP in a DNS it can take "a while" before it fully updates because other servers will have the old IP cached. Servers you have no direct control over.