The answer depends to some degree on what you mean by "safe".
First, your summary doesn't quite capture the difference between SSL/TLS and STARTTLS.
- With SSL/TLS, the client opens a TCP connection to the "SSL port" assigned to the application protocol it wants to use, and starts speaking TLS immediately.
- With STARTTLS, the client opens a TCP connection to the "cleartext port" associated with the application protocol it wants to use, then asks the server "what protocol extensions do you support?". The server then responds with a list of extensions. If one of those extensions is "STARTTLS", the client can then say "okay, let's use TLS" and the two start speaking TLS.
If the client is configured to require TLS, the two approaches are more-or-less equally safe. But there are some subtleties about how STARTTLS must be used to make it safe, and it's a bit harder for the STARTTLS implementation to get those details right.
On the other hand, if the client is configured to use TLS only when TLS is available, and use cleartext when TLS is not available, what the client might do is first try to connect to the SSL port used by the protocol, and if that fails, then connect to the cleartext port and try to use STARTTLS, and finally fall back to cleartext if TLS isn't available in either case. It's fairly easy for an attacker to cause the SSL port connection to fail (all it takes is some well-timed TCP RST packets or blocking of the SSL port). It's a little bit harder - but only a little bit - for an attacker to defeat the STARTTLS negotiation and cause the traffic to stay in cleartext. And then the attacker not only gets to read your email but also gets to capture your username/password for future use.
So the simple answer is if you're connecting to a server which you already know supports TLS (as should be the case when you're sending or reading email), you should use SSL/TLS. If the connection is being attacked, the connection attempt will fail, but your password and email won't be compromised.
On the other hand, if you're connecting to some service which you don't know whether it supports TLS, STARTTLS may be marginally better.
When STARTTLS was invented, "passive" listening-only attacks were very common, "active" attacks in which the attacker would inject traffic to try to lower security were less common. In the 20 or so years since then, active attacks have become more feasible and more common.
For instance, if you are trying to use a laptop in an airport or some other public place and try to read your mail via the wifi that's provided there, you have no idea what that wifi network is doing with your traffic. It's very common for wifi networks to route certain kinds of traffic to "proxies" that interpose themselves between your client applications and the servers they're trying to talk to. It's trivial for those proxies to disable both STARTTLS and "try one port then another" in an attempt to get your client to fall back to cleartext. Yes, this happens, and it's just one example of how your traffic can be spied on by a network. And such attacks aren't limited to three-letter state-supported agencies, they can be pulled off by a teenager with a $35 computer that is hidden in a public place somewhere.