71

TLS stands for "transport layer security". And the list of IP protocol numbers includes "TLSP" as "Transport Layer Security Protocol". These two things would leave me to believe that TLS is a transport layer protocol.

However, most people seem to talk about TLS over TCP. Wikipedia lists it as an "application layer" protocol. This is further complicated by the fact that TCP doesn't have anything like a protocol number: it just packages up raw bytes, so how do you parse out that you are getting a TLS packet, vs a packet that just starts with 0x14 - 0x18 or equivalent?

Andrew Spott
  • 862
  • 1
  • 7
  • 8
  • 3
    Related: [Where does SSL encryption take place?](http://security.stackexchange.com/questions/19681/where-does-ssl-encryption-take-place) – StackzOfZtuff Jul 07 '15 at 15:02
  • This doesn't answer your primary question, but... TCP has ports. They're not protocol numbers, but they often fill that role, because on most servers each net-facing service will have a dedicated port, and it knows what sort of packets to expect on that port. – David Z Jul 09 '15 at 12:35

5 Answers5

115

The OSI model, that categorizes communication protocols into successive layers, is just that: a model. It is an attempt at pushing a physical reality into neatly defined labelled boxes. Nobody ever guaranteed that it works...

Historically, that model was built and published when the ISO was pushing for adoption of its own network protocols. They lost. The World, as a whole, preferred to use the much more simple TCP/IP. The "model" survived the death of its initial ecosystem, and many people have tried to apply it to TCP/IP. It is even commonly taught that way. However, the model does not match well TCP/IP. Some things don't fit in the layers, and SSL/TLS is one of them.

If you look at the protocol details:

  • SSL/TLS uses an underlying transport medium that provides a bidirectional stream of bytes. That would put it somewhere above layer 4.
  • SSL/TLS organizes data as records, that may contain, in particular, handshake messages. Handshake messages look like layer 5. This would put SSL/TLS at layer 6 or 7.
  • However, what SSL/TLS conveys is "application data", which is, in fact, a bidirectional stream of bytes. Applications that use SSL/TLS really use it as a transport protocol. They then use their own data representation and messages and semantics within that "application data". Therefore, SSL/TLS cannot be, in the OSI model, beyond layer 4.

Thus, in the OSI model, SSL/TLS must be in layer 6 or 7, and, at the same time, in layer 4 or below. The conclusion is unescapable: the OSI model does not work with SSL/TLS. TLS is not in any layer.

(This does not prevent some people from arbitrarily pushing TLS in a layer. Since it has no practical impact -- this is just a model -- you can conceptually declare that TLS is layer 2, 5, or 17; it won't be proven false.)

Tom Leek
  • 168,808
  • 28
  • 337
  • 475
  • 1
    Thanks. That really helps. Do you know the answer to my last question then? Since TCP doesn't have any protocol numbers, how do I tell that I'm looking at a TLS packet? – Andrew Spott Jul 07 '15 at 15:22
  • 10
    Usually, TLS happens because client and server agree that TLS ought to happen. For instance, when connecting to a `https://' URL, a Web browser just knows that it should begin some TLS, because that's what the 's' in 'https' means. _From the outside_, you can usually infer that TLS is occurring because you recognize the TLS record headers. Each record begins with a five-byte header: one byte for the record type (0x14 to 0x17), then protocol version (0x03 followed by 0x00, 0x01, 0x02 or 0x03), then record length (over two bytes, big-endian). – Tom Leek Jul 07 '15 at 15:38
  • 8
    To sum up, you usually know that TLS is there because of the context (e.g. if it is on TCP port 443, then chances are that it is TLS), and you can _heuristically_ recognize TLS records with a low false-positive rate. E.g., just yesterday, I was looking at some RDP packets, and right in the middle of them I recognized an encapsulated TLS handshake. Notably, the handshake implies the sending of a certificate from the server, and the encoded certificate really stands out in hexadecimal dumps (well, for me at least). – Tom Leek Jul 07 '15 at 15:40
  • One of the most useful concepts in the OSI model is separation of layers. TCP/IP follows this to some degree. So TCP doesn't know anything about what's encapsulated in the packets, thus no protocol number. So you have to imply the packet through what's generally called "deep packet inspection". – Steve Sether Jul 07 '15 at 22:20
  • @AndrewSpott TCP does indeed not have a protocol number but it has port numbers which to some extent serve the same purpose. Usually you will have no more than one service listening on a specific port number, and that service knows which protocol to be speaking on that port. In less common cases one may decide to have a service understand different protocols on a single port and have that service deduce from the bytes sent by the client, which protocol is being used. In some cases it can be a security vulnerability if client and server speaks different protocols. – kasperd Jul 08 '15 at 07:00
  • 1
    Clearly, imaginary numbers should be used for TLS. TLS is layer 4i. – David Conrad Jul 08 '15 at 16:45
  • 2
    It turns out that in the upper layers OSI is too fine-grained (in the "real world", we use the Internet protocols, which compress layers 5-7 into a single layer) while in the lower layers it is too coarse-grained (again, in the "real world", we use IEEE-based protocols such as Ethernet or WiFi, which split OSI layers 1 and 2 into three layers: PHY, MAC, and LLC). – Jörg W Mittag Jul 08 '15 at 19:40
  • 1
    @JörgWMittag: The OSI layers model has additional limitations when communicating through links with different propagation times and reliability characteristics, and probably contributes to a lot of user frustration with WiFi. If a user clicks a button and gets no response, the local machine's WiFi handler will know if the requests associated with the click have yet reached the router, and will also know how long it is since it was last informed that the router has no data available, but there is no mechanism by which it can inform the application. – supercat Jul 11 '15 at 14:30
37

TCP/IP Model

  1. Link Layer
  2. Internet (IP) Layer
  3. Transport Layer
  4. Application

TLS operates between the Transport layer and the Application Layer (kind of). Really it just wraps Application Layer traffic in encryption during transport.

The TLS Key Exchange happens in the in between layers. Here it's not really Transport Layer because things like port numbers, and sequences numbers are already in place at the Transport Layer. It's only sending data to establish encryption protocols so that it can wrap the Application layer.

IPSec sort of does the same thing, only in an earlier layer. IP Security protects everything above the IP layer. Transport Security protects everything above the Transport Layer. The key exchanges for both are in a weird place because they just need to send data to establish the secure layer. A lot like how ICMP is considered an IP protocol, but it still contains data after the IP layer. Does this make it Transport Layer? No.

OSI Model

The OSI model has a bit more granularity.

  1. Physical Layer
  2. Data Link Layer
  3. Network Layer (IP)
  4. Transport Layer (TCP)
  5. Session Layer (TLS)
  6. Presentation Layer
  7. Application Layer (HTTP)

TLS establishes an encrypted session. In the OSI model this is where TLS operates. It sets up its session, and adds a layer of encryption for the Application Layer (HTTP).

A related answer.

Parsing out TLS

In the Transmission Control Protocol Header there is a Data Offset field. This field represents the size of the TCP header in 32-bit words. The minimum size for this value is 5 (20 bytes). This is at a fixed offset from the beginning of the TCP header. Using this value you'll know the size of the TCP, and can use that to calculate the beginning of the TLS portion.

A TLS Record will always start with a content type, and then the SSL version. There are only 5 different content types, and 4 different versions to worry about. So using the first 3 bytes of the data following the TCP header you should be able to determine if SSL/TLS is being used.

I would run Wireshark on your machine (non-work), and filter on "ssl". Then just browse to https://www.google.com. You'll be able to see the entire handshake. All the layers are broken out for you. It'll give a good representation of how the layers are constructed with TLS, and you'll be able to see each of the TLS Records.

Community
  • 1
RoraΖ
  • 12,317
  • 4
  • 51
  • 83
8

This answer only addresses the "TLSP" sub aspect of the question.

the list of IP protocol numbers includes "TLSP" as "Transport Layer Security Protocol".

I had never heard of that protocol. It's not the protocol you use to display HTTPS pages.

TLSP seems to be a very obscure protocol. And it was registered in 1994-10. This was before the protocol that we now commonly refer to as "TLS". Back in that day, the protocol was still called "SSL". The first public draft for "TLS" is dated two years later, on 1996-11-26.

See this question for details on the timeline:

The IANA names "Christer_Oberg@bull.se" as the registrant. That name does not ring a bell for me. The bull.se website is also no longer active.

There is in fact a LinkedIn page for one Swedish Christer Öberg who worked for a "Bull" company as "CTO security solutions" in 1994. I'm guessing that's the guy that once submitted the "TLSP" registration. (And also that he's the only one who can tell you what "TLSP" really was.)

Community
  • 1
StackzOfZtuff
  • 17,783
  • 1
  • 50
  • 86
  • According to Wikipedia "Transport Layer Security Protocol" is synonym for "Transport Layer Security". It might be Wikipedia is wrong. – kasperd Jul 24 '15 at 19:20
  • Yeah. They don't give a citation for that. I think somebody got confused. But I haven't checked. Should be easy to tell from the edit history who made that link. – StackzOfZtuff Jul 24 '15 at 19:23
3

TCP does have an IP protocol number: 0x06

TSL/SSL is an application protocol meaning some program (not the operating system) processes the packets. For example, if a web server is accepting HTTPS requests, then it is the web server that processes the SSL packets which are inside of the TCP packets.

The TLSP 0x38 protocol is Kryptonet-specific. Nobody has used Kryptonet key management for years.

Communication packets are not processed generically based on their contents. The server expects packets to be in the correct protocol from the get go. For example, if an SSL server is listening on a socket and receives a non-SSL packet it will ignore it.

Tyler Durden
  • 1,116
  • 1
  • 9
  • 18
-2

Transport Layer as the name indicates. Transport Layer is responsible for end to end communication, So any data that gets into Network/Internet Layer has to be encrypted this is done by the layer above Network/Internet Layer. And that is Transport Layer

Amar T
  • 101
  • 1
    TLS is not end-to-end. It is easily broken at several points along the way. TLS cannot be used to transport data, merely to encrypt the data being transported, so, no, it is not on the Transport layer. – schroeder Feb 19 '20 at 11:59