Netstring

In computer programming, a netstring is a formatting method for byte strings that uses a declarative notation to indicate the size of the string.[1][2]

Netstrings store the byte length of the data that follows, making it easier to unambiguously pass text and byte data between programs that could be sensitive to values that could be interpreted as delimiters or terminators (such as a null character).

The format consists of the string's length written using ASCII digits, followed by a colon, the byte data, and a comma. "Length" in this context means "number of 8-bit units", so if the string is, for example, encoded using UTF-8, this may or may not be identical to the number of textual characters that are present in the string.

For example, the text "hello world!" encodes as:

<31 32 3a 68 65 6c 6c 6f 20 77 6f 72 6c 64 21 2c>

i.e.

12:hello world!,

And an empty string as:

<30 3a 2c>

i.e.

0:,

The comma makes it slightly simpler for humans to read netstrings that are used as adjacent records, and provides weak verification of correct parsing. Note that without the comma, the format mirrors how Bencode encodes strings.

The length is written without leading zeroes. Empty string is the only netstring that begins with zero. There is exactly one legal netstring encoding for any byte string.

Since the format is easy to generate and to parse, it is easy to support by programs written in different programming languages. In practice, netstrings are often used to simplify exchange of bytestrings, or lists of bytestrings. For example, see its use in the Simple Common Gateway Interface (SCGI) and the Quick Mail Queuing Protocol (QMQP) .

Netstrings avoid complications that arise in trying to embed arbitrary data in delimited formats. For example, XML may not contain certain byte values and requires a nontrivial combination of escaping and delimiting, while generating multipart MIME messages involves choosing a delimiter that must not clash with the content of the data.

Netstrings can be stored recursively. The result of encoding a sequence of strings is a single string. Rewriting the above "hello world!" example to instead be a sequence of two netstrings, itself encoded as a single netstring, gives the following:

17:5:hello,6:world!,,

Parsing such a nested netstring is an example of duck typing, since the contained string ("5:hello,6:world!,") is both a string and a sequence of netstrings. Its effective type is determined by how the application chooses to interpret it, not by any explicit type declaration required by the netstring specification. In general, there are 3 ways that a program expecting a netstring may choose to interpret its contents:

  • As human-readable text with no further automatic processing
  • As encapsulated data in some pre-arranged fixed data serialization format (such as the binary contents of a C or C++ struct)
  • As encapsulated metadata and data, using a tagged union convention to describe the types of nested netstrings, thereby establishing a self-describing hierarchical data serialization format. ("Tagged netstrings" and Bencode can be seen as extensions of netstring that support similar self-describing hierarchical formats[3][4][5][6][7])

Note that since netstrings pose no limitations on the contents of the data they store, netstrings can not be embedded verbatim in most delimited formats without the possibility of interfering with the delimiting of the containing format.

In the context of network programming it is potentially useful that the receiving program is informed of the size of the data that follows, as it can allocate exactly enough memory, avoid the need for reallocation to accommodate more data, and preemptively reject data that would exceed size limits.

See also

Notes and references
  1. defined in a document by D. J. Bernstein.
  2. See e.g. Python Web Programming By Steve Holden, David M. Beazley Published by Sams Publishing, 2002 ISBN 0-7357-1090-2, 978-0-7357-1090-0 691 pages, page 202.
  3. Caolan McMahon. "Bencoding".
  4. "TNetstrings Specification". Archived from the original on 2014-02-10.
  5. tnetstring-rb
  6. "tnetstring: the tagged netstring specification"
  7. "tnetstring: data serialization using typed netstrings"
This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.