No.
Generally they can't do this since still very large majority of mail addresses are handled by non-TLS capable servers.
However some large providers (and the first was Gmail I believe) started "scoring" or "marking" mail received from TLS-incapable servers, and these mail may possibly end up in spam more easily than others.
As for your "putting another way" it doesn't quite work that way, since the servers actully have to announce their capabilities, so if they announce STARTTLS but it's unsuccessful then it's a hard protocol error, and may result a bounce similar to any 5xx (permanent) or persistent 4xx (temporary) errors; for others it is not a question of "success" but support, but I haven't seen disconnects due to non-TLS-support on the few servers I have without it.