Is TCP more secure than UDP?
Yes, but we have to be very clear about what we are talking about when we are talking about "security" and not generalize this statement to upper layer protocols.
Currently, security is often associated to the CIA triad:
- Confidentiality.
- Integrity.
- Availability.
The version 4 of the IP protocol, which is still the most commonly used now, is a very old beast which was developed during the 70's and 80's.
At that time, confidentiality was not really a subject and what was really targeted was to achieve integrity and availability (the Arpanet network, the ancestor of the Internet which gave birth to the IP protocol, was designed to ensure a continuity of service even in the event of a nuclear war, not to protect in-transit data).
In this optic, two transport protocols were developed over the IP layer: TCP and UDP.
TCP was the one designed to ensure both the integrity and availability properties. It includes what was at that times advanced techniques such as a three-way handshake, parameters negotiation, various connection state handling, transparent packet reordering, acknowledgement windows and retry mechanisms. This brings good guaranties to the sender that the data it sent has been received in a complete an uncorrupted form (ie. no missing, altered or unordered parts).
Remind though that what this protocol was targeting was a technical disaster, not a malicious tampering of the data in transit. Such issue was completely out-of-scope at that time.
UDP on the contrary was designed to be a fast protocol. It has none of the above mentioned feature, and therefore none of their overhead too. In particular, when the sender sends some data using UDP, the data received may be incomplete, unordered or not received at all: the UDP protocol by itself does not provide any mechanism to prevent or detect this neither on the sender or receiver sides.
In this way, when focusing on the data integrity and availability properties of security, TCP is indeed more secure than UDP.
Is an application protocol relying on TCP more secure than one relying on UDP?
Certainly not.
This only means that people developing an application protocol relying on UDP will have more work since they may have to implement in their application protocol workarounds for the features missing in the UDP protocol. They will have to take into account that data sent may not be necessarily received, that data received may not be in the right order, etc. These are all well-known problems.
OpenVPN for instance, while it supports TCP mostly for compatibility with restrictive firewalls, runs by default and runs best over UDP. Switching it to TCP is possible but will not increase its security in any way as the difference between the two transport layer protocol UDP and TCP is fully handled by OpenVPN itself. Switching it to TCP will only add the TCP overhead to the OpenVPN protocol, thus reducing its performance.
Is TCP better than UDP?
No, this is entirely an application protocol design choice.
UDP is a more raw protocol. When used correctly and carefully it may achieve better performances than TCP at the price of a more difficult development and maintenance of the application protocol.
- When the application is not time sensitive, TCP imposes itself as the natural choice as there is no need to reinvent the wheel.
- When the application is time sensitive, a discussion must occur on which to choose knowing that each will come with its drawback: a potential performance penalty with TCP against a certainly more complex application with UDP.
Most protocols are not time sensitive and therefore TCP is most widely used protocol. Indeed, when you load a web page or receive an email, there is no difference if you receive it 10 milliseconds sooner or later.
Two classical examples of protocols using UDP, in addition to the OpenVPN previously cited, are media streaming and DNS.
With media streaming, you don't really care if one video frame or a few milliseconds of video or audio is missing, as long as the video or audio is playing smoothly and synchronously. In such case, you don't want to induce repetitive pauses because TCP detected a missing packet and asked the sender to resend the content of the last acknowledgement window.
With DNS, requests and answers are usually very short and you want the name resolution process to be as fast as possible (note that longer and less time-sensitive answers such as DNS zone transfers usually still occur on TCP). It is faster to resend the name resolution request to the DNS server on the very few times the request or its answer gets lost than processing a full-fledged three-way handshake for every request.
What about confidentiality and malicious sniffing/tampering (and IPv6)?
All we cared about until now was, in the spirit of this old IPv4, the balance between transmission speed and data integrity + availability. Now, if we want to add confidentiality on top of this, we can do this but it will have to be done at the application layer as, as stated before, IPv4 is not concerned by confidentiality issues.
A modern, full-fledged implementation of security can be implemented at the application layer and rely on either TCP or UDP (or both) protocols without any impact on the application protocol security itself (see the example of OpenVPN above).
However, as stated in the beginning, IPv4 really comes from a another computing age. It got a successor by the name if IPv6, which natively supports IPSec at the IP layer, thus providing more modern security services below transport protocols such as UDP and TCP.
This allows to delegate in-transit data encryption from the application to the network layer, and allows both UDP and TCP to provide exactly the same security guaranties. However, in most scenarios UDP performance gain will be counterbalanced by IPSec overhead, so I'm not sure there would be any advantage in using UDP instead TCP as long as IPv6 IPSec is being used.