< draft-mcquistin-augmented-ascii-diagrams-01.txt   draft-mcquistin-augmented-ascii-diagrams-02.txt >
Network Working Group S. McQuistin Network Working Group S. McQuistin
Internet-Draft V. Band Internet-Draft V. Band
Intended status: Experimental C. S. Perkins Intended status: Experimental C. S. Perkins
Expires: 6 May 2020 University of Glasgow Expires: 7 August 2020 University of Glasgow
3 November 2019 4 February 2020
Describing Protocol Data Units with Augmented Packet Header Diagrams Describing Protocol Data Units with Augmented Packet Header Diagrams
draft-mcquistin-augmented-ascii-diagrams-01 draft-mcquistin-augmented-ascii-diagrams-02
Abstract Abstract
This document describes a machine-readable format for specifying the This document describes a machine-readable format for specifying the
syntax of protocol data units within a protocol specification. This syntax of protocol data units within a protocol specification. This
format is comprised of a consistently formatted packet header format is comprised of a consistently formatted packet header
diagram, followed by structured explanatory text. It is designed to diagram, followed by structured explanatory text. It is designed to
maintain human readability while enabling support for automated maintain human readability while enabling support for automated
parser generation from the specification document. This document is parser generation from the specification document. This document is
itself an example of how the format can be used. itself an example of how the format can be used.
skipping to change at page 1, line 37 skipping to change at page 1, line 37
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/. Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on 6 May 2020. This Internet-Draft will expire on 7 August 2020.
Copyright Notice Copyright Notice
Copyright (c) 2019 IETF Trust and the persons identified as the Copyright (c) 2020 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/ Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this document. license-info) in effect on the date of publication of this document.
Please review these documents carefully, as they describe your rights Please review these documents carefully, as they describe your rights
and restrictions with respect to this document. Code Components and restrictions with respect to this document. Code Components
extracted from this document must include Simplified BSD License text extracted from this document must include Simplified BSD License text
as described in Section 4.e of the Trust Legal Provisions and are as described in Section 4.e of the Trust Legal Provisions and are
provided without warranty as described in the Simplified BSD License. provided without warranty as described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Background . . . . . . . . . . . . . . . . . . . . . . . . . 4 2. Background . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.1. Limitations of Current Packet Format Diagrams . . . . . . 4 2.1. Limitations of Current Packet Format Diagrams . . . . . . 4
2.2. Formal languages in standards documents . . . . . . . . . 6 2.2. Formal languages in standards documents . . . . . . . . . 7
3. Design Principles . . . . . . . . . . . . . . . . . . . . . . 7 3. Design Principles . . . . . . . . . . . . . . . . . . . . . . 7
4. Augmented Packet Header Diagrams . . . . . . . . . . . . . . 8 4. Augmented Packet Header Diagrams . . . . . . . . . . . . . . 9
4.1. PDUs with Fixed and Variable-Width Fields . . . . . . . . 9 4.1. PDUs with Fixed and Variable-Width Fields . . . . . . . . 9
4.2. PDUs That Cross-Reference Previously Defined 4.2. PDUs That Cross-Reference Previously Defined Fields . . . 12
Fields . . . . . . . . . . . . . . . . . . . . . . . . . 11 4.3. PDUs with Non-Contiguous Fields . . . . . . . . . . . . . 15
4.3. PDUs with Non-Contiguous Fields . . . . . . . . . . . . . 14 4.4. Importing PDU Definitions from Other Documents . . . . . 15
5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 14 5. Open Issues . . . . . . . . . . . . . . . . . . . . . . . . . 16
6. Security Considerations . . . . . . . . . . . . . . . . . . . 14 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 16
7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 14 7. Security Considerations . . . . . . . . . . . . . . . . . . . 16
8. Informative References . . . . . . . . . . . . . . . . . . . 15 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 16
Appendix A. ABNF specification . . . . . . . . . . . . . . . . . 16 9. Informative References . . . . . . . . . . . . . . . . . . . 17
A.1. Constraint Expressions . . . . . . . . . . . . . . . . . 16 Appendix A. ABNF specification . . . . . . . . . . . . . . . . . 18
A.2. Augmented packet diagrams . . . . . . . . . . . . . . . . 16 A.1. Constraint Expressions . . . . . . . . . . . . . . . . . 18
Appendix B. Source code repository . . . . . . . . . . . . . . . 16 A.2. Augmented packet diagrams . . . . . . . . . . . . . . . . 18
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 16 Appendix B. Source code repository . . . . . . . . . . . . . . . 19
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 19
1. Introduction 1. Introduction
Packet header diagrams have become a widely used format for Packet header diagrams have become a widely used format for
describing the syntax of binary protocols. In otherwise largely describing the syntax of binary protocols. In otherwise largely
textual documents, they allow for the visualisation of packet textual documents, they allow for the visualisation of packet
formats, reducing human error, and aiding in the implementation of formats, reducing human error, and aiding in the implementation of
parsers for the protocols that they specify. parsers for the protocols that they specify.
Figure 1 gives an example of how packet header diagrams are used to Figure 1 gives an example of how packet header diagrams are used to
skipping to change at page 3, line 46 skipping to change at page 3, line 46
This document describes a consistent packet header diagram format and This document describes a consistent packet header diagram format and
accompanying structured text constructs that allow for the parsing accompanying structured text constructs that allow for the parsing
process of protocol headers to be fully specified. This provides process of protocol headers to be fully specified. This provides
support for the automatic generation of parser code. Broad design support for the automatic generation of parser code. Broad design
principles, that seek to maintain the primacy of human readability principles, that seek to maintain the primacy of human readability
and flexibility in authorship, are described, before the format and flexibility in authorship, are described, before the format
itself is given. itself is given.
This document is itself an example of the approach that it describes, This document is itself an example of the approach that it describes,
with the packet header diagrams and structured text format described with the packet header diagrams and structured text format described
by example. by example. Examples that do not form part of the protocol
description language are marked by a colon at the beginning of each
line; this prevents them from being parsed by the accompanying
tooling.
This draft describes early work. As consensus builds around the This draft describes early work. As consensus builds around the
particular syntax of the format described, both a formal ABNF particular syntax of the format described, both a formal ABNF
specification and code that parses it (and, as described above, this specification (Appendix A) and code (Appendix B) that parses it (and,
document) will be provided. as described above, this document) will be provided.
2. Background
This section begins by considering how packet header diagrams are
used in existing documents. This exposes the limitations that the
current usage has in terms of machine-readability, guiding the design
of the format that this document proposes.
While this document focuses on the machine-readability of packet
format diagrams, this section also discusses the use of other
structured or formal languages within IETF documents. Considering
how and why these languages are used provides an instructive contrast
to the relatively incremental approach proposed here.
2.1. Limitations of Current Packet Format Diagrams
: The RESET_STREAM frame is as follows: : The RESET_STREAM frame is as follows:
: :
: 0 1 2 3 : 0 1 2 3
: 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 : 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ : +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: | Stream ID (i) ... : | Stream ID (i) ...
: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ : +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: | Application Error Code (16) | : | Application Error Code (16) |
: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ : +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
skipping to change at page 4, line 31 skipping to change at page 4, line 48
: :
: Application Protocol Error Code: A 16-bit application protocol : Application Protocol Error Code: A 16-bit application protocol
: error code (see Section 20.1) which indicates why the stream : error code (see Section 20.1) which indicates why the stream
: is being closed. : is being closed.
: :
: Final Size: A variable-length integer indicating the final size : Final Size: A variable-length integer indicating the final size
: of the stream by the RESET_STREAM sender, in unit of bytes. : of the stream by the RESET_STREAM sender, in unit of bytes.
Figure 2: QUIC's RESET_STREAM frame format (from [QUIC-TRANSPORT]) Figure 2: QUIC's RESET_STREAM frame format (from [QUIC-TRANSPORT])
2. Background
This section begins by considering how packet header diagrams are
used in existing documents. This exposes the limitations that the
current usage has in terms of machine-readability, guiding the design
of the format that this document proposes.
While this document focuses on the machine-readability of packet
format diagrams, this section also discusses the use of other
structured or formal languages within IETF documents. Considering
how and why these languages are used provides an instructive contrast
to the relatively incremental approach proposed here.
2.1. Limitations of Current Packet Format Diagrams
Packet header diagrams are frequently used in IETF standards to Packet header diagrams are frequently used in IETF standards to
describe the format of binary protocols. While there is no standard describe the format of binary protocols. While there is no standard
for how these diagrams should be formatted, they have a broadly for how these diagrams should be formatted, they have a broadly
similar structure, where the layout of a protocol data unit (PDU) or similar structure, where the layout of a protocol data unit (PDU) or
structure is shown in diagrammatic form, followed by a description structure is shown in diagrammatic form, followed by a description
list of the fields that it contains. An example of this format, list of the fields that it contains. An example of this format,
taken from the QUIC specification, is given in Figure 2. taken from the QUIC specification, is given in Figure 2.
These packet header diagrams, and the accompanying descriptions, are These packet header diagrams, and the accompanying descriptions, are
formatted for human readers rather than for automated processing. As formatted for human readers rather than for automated processing. As
skipping to change at page 6, line 13 skipping to change at page 6, line 17
only fully expressed if all elements can be parsed. only fully expressed if all elements can be parsed.
Figure 2 highlights the difficulty that machine parsers have in Figure 2 highlights the difficulty that machine parsers have in
chaining structures together. Two fields ("Stream ID" and "Final chaining structures together. Two fields ("Stream ID" and "Final
Size") are described as being encoded as variable-length integers; Size") are described as being encoded as variable-length integers;
this is a structure described elsewhere in the same document. this is a structure described elsewhere in the same document.
Structured text is required both alongside the definition of the Structured text is required both alongside the definition of the
containing structure and with the definition of the sub-structure, containing structure and with the definition of the sub-structure,
to allow a parser to link the two together. to allow a parser to link the two together.
Lack of extension and evolution syntax: Protocols are often
specified across multiple documents, either because the protocol
explicitly includes extension points (e.g., profiles and payload
format specifications in RTP [RFC3550]) or because definition of a
protocol data unit has changed and evolved over time. As a
result, it is essential that syntax be provided to allow for a
complete definition of a protocol's parsing process to be
constructed across multiple documents.
: The format of the "Relay Source Port Option" is shown below: : The format of the "Relay Source Port Option" is shown below:
: :
: 0 1 2 3 : 0 1 2 3
: 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 : 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ : +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: | OPTION_RELAY_PORT | Option-Len | : | OPTION_RELAY_PORT | Option-Len |
: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ : +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: | Downstream Source Port | : | Downstream Source Port |
: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ : +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: :
skipping to change at page 7, line 8 skipping to change at page 7, line 21
problematic for the development of tooling to parse specifications, problematic for the development of tooling to parse specifications,
these, and other, languages serve a range of different use cases. these, and other, languages serve a range of different use cases.
ABNF, for example, is typically used to specify text protocols, while ABNF, for example, is typically used to specify text protocols, while
ASN.1 is used to specify data structure serialisation. This document ASN.1 is used to specify data structure serialisation. This document
specifies a structured language for specifying the parsing of binary specifies a structured language for specifying the parsing of binary
protocol data units. protocol data units.
3. Design Principles 3. Design Principles
The use of structures that are designed to support machine The use of structures that are designed to support machine
readability may potentially interfere with the existing ways in which readability might potentially interfere with the existing ways in
protocol specifications are used and authored. To the extent that which protocol specifications are used and authored. To the extent
these existing uses are more important than machine readability, such that these existing uses are more important than machine readability,
interference must be minimised. such interference must be minimised.
In this section, the broad design principles that underpin the format In this section, the broad design principles that underpin the format
described by this document are given. However, these principles described by this document are given. However, these principles
apply more generally to any approach that introduces structured and apply more generally to any approach that introduces structured and
formal languages into standards documents. formal languages into standards documents.
It should be noted that these are design principles: they expose the It should be noted that these are design principles: they expose the
trade-offs that are inherent within any given approach. Violating trade-offs that are inherent within any given approach. Violating
these principles is sometimes necessary and beneficial, and this these principles is sometimes necessary and beneficial, and this
document sets out the potential consequences of doing so. document sets out the potential consequences of doing so.
skipping to change at page 8, line 9 skipping to change at page 8, line 23
optional, supplementary tools that aid in the authoring machine- optional, supplementary tools that aid in the authoring machine-
readable structures. readable structures.
The immediate impact of requiring specific tooling is that The immediate impact of requiring specific tooling is that
adoption is likely to be limited. A long-term impact might be adoption is likely to be limited. A long-term impact might be
that authors whose workflows are incompatible might be alienated that authors whose workflows are incompatible might be alienated
from the process. from the process.
Canonical specifications: As far as possible, machine-readable Canonical specifications: As far as possible, machine-readable
structures should not replicate the human readable specification structures should not replicate the human readable specification
of the protocol within the same document. Such structures should of the protocol within the same document. Machine-readable
form part of a canonical specification of the protocol. Adding structures should form part of a canonical specification of the
supplementary machine-readable structures, in parallel to the protocol. Adding supplementary machine-readable structures, in
existing human readable text, is undesirable because it creates parallel to the existing human readable text, is undesirable
the potential for inconsistency. because it creates the potential for inconsistency.
As an example, program code that describes how a protocol data As an example, program code that describes how a protocol data
unit can be parsed might be provided as an appendix within a unit can be parsed might be provided as an appendix within a
standards document. This code would provide a specification of standards document. This code would provide a specification of
the protocol that is separate to the prose description in the main the protocol that is separate to the prose description in the main
body of the document. This has the undesirable effect of body of the document. This has the undesirable effect of
introducing the potential for the program code to specify introducing the potential for the program code to specify
behaviour that the prose-based specification does not, and vice- behaviour that the prose-based specification does not, and vice-
versa. versa.
skipping to change at page 8, line 36 skipping to change at page 8, line 50
protocols. If a given language is not sufficiently expressive, protocols. If a given language is not sufficiently expressive,
then adoption is likely to be limited. At the limits of what can then adoption is likely to be limited. At the limits of what can
be expressed by the language, authors are likely to revert to be expressed by the language, authors are likely to revert to
defining the protocol in prose: this undermines the broad goal of defining the protocol in prose: this undermines the broad goal of
using structured and formal languages. Equally, though, using structured and formal languages. Equally, though,
understandable specifications and ease of use are critical for understandable specifications and ease of use are critical for
adoption. A tool that is simple to use and addresses the most adoption. A tool that is simple to use and addresses the most
common use cases might be preferred to a complex tool that common use cases might be preferred to a complex tool that
addresses all use cases. addresses all use cases.
It may be desirable to restrict expressiveness, however, to
guarantee intrinsic safety, security, and computability properties
of both the generated parser code for the protocol, and the parser
of the description language itself. In much the same way as the
language-theoretic security ([LANGSEC]) community advocates for
programming language design to be informed by the desired
properties of the parsers for those languages, protocol designers
should be aware of the implications of their design choices. The
expressiveness of the protocol description languages that they use
to define their protocols can force such awareness.
Broadly, those languages that are more expressive tend to have
parsers that are more complex and less safe. As a result, while
considering the other goals described in this document, protocol
description languages should attempt to be minimally expressive,
and restrict protocol designs to those for which safe and secure
parsers can be generated.
Minimise required change: Any approach should require as few changes Minimise required change: Any approach should require as few changes
as possible to the way that documents are formatted, authored, and as possible to the way that documents are formatted, authored, and
published. Forcing adoption of a particular structured or formal published. Forcing adoption of a particular structured or formal
language is incompatible with the IETF's standardisation process: language is incompatible with the IETF's standardisation process:
there are very few components of standards documents that are non- there are very few components of standards documents that are non-
optional. optional.
4. Augmented Packet Header Diagrams 4. Augmented Packet Header Diagrams
The design principles described in Section 3 can largely be met by The design principles described in Section 3 can largely be met by
skipping to change at page 9, line 38 skipping to change at page 10, line 21
A PDU description is introduced by the exact phrase "A/An _______ is A PDU description is introduced by the exact phrase "A/An _______ is
formatted as follows:" at the end of a paragraph. This is followed formatted as follows:" at the end of a paragraph. This is followed
by the PDU description itself, as a packet diagram within an by the PDU description itself, as a packet diagram within an
<artwork> element in the XML representation, starting with a header <artwork> element in the XML representation, starting with a header
line to show the bit width of the diagram. The description of the line to show the bit width of the diagram. The description of the
fields follows the diagram, as an XML <dl> list, after a paragraph fields follows the diagram, as an XML <dl> list, after a paragraph
containing the text "where:". containing the text "where:".
Each field of the description starts with a <dt> tag comprising the Each field of the description starts with a <dt> tag comprising the
field name and an optional short name in parenthesis. These are field name and an optional short name in parenthesis. These are
followed by a colon, the field length, and a terminating period. The followed by a colon, the field length, an optional presence
following <dd> tag contains a prose description of the field. expression (described in Section 4.2), and a terminating period. The
following <dd> tag contains a prose description of the field. Field
names cannot be the same as a previously defined PDU name.
For example, this can be illustrated using the IPv4 Header Format For example, this can be illustrated using the IPv4 Header Format
[RFC791]. An IPv4 Header is formatted as follows: [RFC791]. An IPv4 Header is formatted as follows:
0 1 2 3 0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Version| IHL | DSCP |ECN| Total Length | |Version| IHL | DSCP |ECN| Total Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Identification |Flags| Fragment Offset | | Identification |Flags| Fragment Offset |
skipping to change at page 10, line 38 skipping to change at page 11, line 13
is shown in the diagram. The field's width -- 4 bits -- is given is shown in the diagram. The field's width -- 4 bits -- is given
in the label of the description list, separated from the field's in the label of the description list, separated from the field's
label by a colon. label by a colon.
Internet Header Length (IHL): 4 bits. This is a shorter field, whose Internet Header Length (IHL): 4 bits. This is a shorter field, whose
full label is too large to be shown in the diagram. A short label full label is too large to be shown in the diagram. A short label
(IHL) is used in the diagram, and this short label is provided, in (IHL) is used in the diagram, and this short label is provided, in
brackets, after the full label in the description list. brackets, after the full label in the description list.
Differentiated Services Code Point (DSCP): 6 bits. This is a fixed- Differentiated Services Code Point (DSCP): 6 bits. This is a fixed-
width field, as previously defined. width field, as previously discussed.
Explicit Congestion Notification (ECN): 2 bits. This is a fixed- Explicit Congestion Notification (ECN): 2 bits. This is a fixed-
width field, as previously defined. width field, as previously discussed.
Total Length (TL): 2 bytes. This is a fixed-width field, as Total Length (TL): 2 bytes. This is a fixed-width field, as
previously defined. Where fields are an integral number of bytes previously discussed. Where fields are an integral number of
in size, the field length can be given in bytes rather than in bytes in size, the field length can be given in bytes rather than
bits. in bits.
Identification: 2 bytes. This is a fixed-width field, as previously Identification: 2 bytes. This is a fixed-width field, as previously
defined. discussed.
Flags: 3 bits. This is a fixed-width field, as previously defined. Flags: 3 bits. This is a fixed-width field, as previously discussed.
Fragment Offset: 13 bits. This is a fixed-width field, as previously Fragment Offset: 13 bits. This is a fixed-width field, as previously
defined. discussed.
Time To Live (TTL): 1 byte. This is a fixed-width field, as Time to Live (TTL): 1 byte. This is a fixed-width field, as
previously defined. previously discussed.
Protocol: 1 byte. This is a fixed-width field, as previously Protocol: 1 byte. This is a fixed-width field, as previously
defined. discussed.
Header Checksum: 2 bytes. This is a fixed-width field, as previously Header Checksum: 2 bytes. This is a fixed-width field, as previously
defined. discussed.
Source Address: 32 bits. This is a fixed-width field, as previously Source Address: 32 bits. This is a fixed-width field, as previously
defined. discussed.
Destination Address: 32 bits. This is a fixed-width field, as Destination Address: 32 bits. This is a fixed-width field, as
previously defined. previously discussed.
Options: (IHL-5)*32 bits. This is a variable-length field, whose Options: (IHL-5)*32 bits. This is a variable-length field, whose
length is defined by the value of the field with short label IHL length is defined by the value of the field with short label IHL
(Internet Header Length). Constraint expressions can be used in (Internet Header Length). Constraint expressions can be used in
place of constant values: the grammar for the expression language place of constant values: the grammar for the expression language
is defined in Appendix A.1. Constraints can include a previously is defined in Appendix A.1. Constraints can include a previously
defined field's short or full label, where one has been defined. defined field's short or full label, where one has been defined.
Short variable-length fields are indicated by "..." instead of a Short variable-length fields are indicated by "..." instead of a
pipe at the end of the row. pipe at the end of the row.
Payload: TL - ((IHL*32)/8) bytes. This is a multi-row variable- Payload: TL - ((IHL*32)/8) bytes. This is a multi-row variable-
length field, constrained by the values of fields TL and IHL. length field, constrained by the values of fields TL and IHL.
Instead of the "..." notation, ":" is used to indicate that the Instead of the "..." notation, ":" is used to indicate that the
field is variable-length. The use of ":" instead of "..." field is variable-length. The use of ":" instead of "..."
indicates the field is likely to be a longer, multi-row field. indicates the field is likely to be a longer, multi-row field.
However, semantically, there is no difference: these different However, semantically, there is no difference: these different
notations are for the benefit of human readers. notations are for the benefit of human readers.
skipping to change at page 11, line 51 skipping to change at page 12, line 27
Binary formats often reference sub-structures that have been defined Binary formats often reference sub-structures that have been defined
earlier in the specification. For example, in RTP [RFC3550], the earlier in the specification. For example, in RTP [RFC3550], the
Contributing Source Identifiers in an RTP Data Packet are defined as Contributing Source Identifiers in an RTP Data Packet are defined as
comprising a list of Source Identifier elements. A Source Identifier comprising a list of Source Identifier elements. A Source Identifier
is formatted as follows: is formatted as follows:
0 1 2 3 0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Source Identifier | | SSRC |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
where: where:
Source Identifier: 32 bits. This is a fixed-width field, as SSRC: 32 bits. This is a fixed-width field, as described previously.
described previously.
The following example shows how a Source Identifier can be referenced The following example shows how a Source Identifier can be referenced
in the description of an RTP Data Packet. It also shows how the in the description of an RTP Data Packet. It also shows how the
presence of some fields in a format may be dependent on the values of presence of some fields in a format may be dependent on the values of
an earlier field. an earlier field.
An RTP Data Packet is formatted as follows: An RTP Data Packet is formatted as follows:
0 1 2 3 0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
skipping to change at page 13, line 15 skipping to change at page 13, line 52
Payload Type (PT): 7 bits. This is a fixed-width field, as described Payload Type (PT): 7 bits. This is a fixed-width field, as described
previously. previously.
Sequence Number (PT): 16 bits. This is a fixed-width field, as Sequence Number (PT): 16 bits. This is a fixed-width field, as
described previously. described previously.
Timestamp (PT): 32 bits. This is a fixed-width field, as described Timestamp (PT): 32 bits. This is a fixed-width field, as described
previously. previously.
Synchronization Source identifier: 1 * Source Identifier. This is a Synchronization Source identifier: 1 * Source Identifier. This is a
field whose structure is a previously defined PDU format. To field whose structure is a previously defined PDU format (Source
indicate this, the width of the field is expressed in terms of Identifier). To indicate this, the width of the field is
cross-referenced structure (here, Source Identifier). When used expressed in terms of cross-referenced structure. When used in
in constraint expressions, PDU names refer to the length of that constraint expressions, PDU names refer to the length of that PDU
PDU structure. structure.
Contributing Source identifiers: CC * Source Identifier. Where a Contributing Source identifiers: CC * Source Identifier. Where a
field is comprised of a sequence of previously defined structures, field is comprised of a sequence of previously defined structures,
square brackets can be used to indicate this in the diagram. The square brackets can be used to indicate this in the diagram. The
length of the sequence can be defined using the constraint length of the sequence can be defined using the constraint
expression grammar as described earlier. expression grammar as described earlier.
In this example, both a PDU name (Source Identifier) and a field
name (CC) are used in the constraint expression. The PDU name
refers to the length of the PDU, while the field name refers to
the value of the field. This is possible because field names
cannot be the same as previously defined PDU names.
Header Extension: 32 bits; present only when X == 1. This is a field Header Extension: 32 bits; present only when X == 1. This is a field
whose presence is predicated on an expression given using the whose presence is predicated on an expression given using the
constraint expression grammar described earlier. Optional fields constraint expression grammar described earlier. Optional fields
can be of any previously defined format (e.g., fixed- or variable- can be of any previously defined format (e.g., fixed- or variable-
width). Optional fields are indicated by the presence of a width). Optional fields are indicated by the presence of ";
"Present only when [expr]." as the first line in their present only when [expr]." at the end of the definition term
description. (i.e., the text contained within the <dt> tag).
[Note that this example deviates from the format as described in [Note that this example deviates from the format as described in
[RFC3550]. As specified in that document, the Header Extension [RFC3550]. As specified in that document, the Header Extension
would be a cross-referenced structure. This is not shown here for would be a cross-referenced structure. This is not shown here for
brevity.] brevity.]
Payload. The length of the Payload is not specified, and hence needs Payload. The length of the Payload is not specified, and hence needs
to be inferred from the total length of the packet and the lengths to be inferred from the total length of the packet and the lengths
of the known fields. There can only be one field of unspecified of the known fields. There can only be one field of unspecified
size in a PDU. size in a PDU.
Padding: Padding Count bytes; present only when (P == 1) and Padding: Padding Count bytes; present only when (P == 1) and
(Padding Count > 0). (Padding Count > 0).
This is a variable size field, with size dependent on a later This is a variable size field, with size dependent on a later
field in the packet. Fields can only depend on the value of a field in the packet. Fields can only depend on the value of a
later field if they follow a field with unspecified size. later field if they follow a field with unspecified size.
Padding Count: 1 byte; present only when P == 1. This is a fixed- Padding Count: 1 byte; present only when P == 1. This is a fixed-
width field, as previously defined. width field, as previously discussed.
4.3. PDUs with Non-Contiguous Fields 4.3. PDUs with Non-Contiguous Fields
In some binary formats, fields are striped across multiple non- In some binary formats, fields are striped across multiple non-
contiguous bits. This is often to allow for backwards compatibility contiguous bits. This is often to allow for backwards compatibility
with previous definitions of the same fields in earlier documents: with previous definitions of the same fields in earlier documents:
striping in this way allows for careful use of the possible range of striping in this way allows for careful use of the possible range of
values. values.
This format is illustrated using the STUN Message Type This format is illustrated using the STUN Message Type
skipping to change at page 14, line 36 skipping to change at page 15, line 36
Method (M): 12 bits. This field is comprised of multiple sub-fields Method (M): 12 bits. This field is comprised of multiple sub-fields
(M0 through MB) as shown in the diagram. That these sub-fields (M0 through MB) as shown in the diagram. That these sub-fields
should be concatenated, after parsing, into a single field is should be concatenated, after parsing, into a single field is
indicated by their being labelled using the 'M' short field name indicated by their being labelled using the 'M' short field name
followed by a single hexadecimal digit, with the least significant followed by a single hexadecimal digit, with the least significant
bit labelled with 0, and subsequent bits labelled in sequence. bit labelled with 0, and subsequent bits labelled in sequence.
Class (C): 2 bits. This field follows the same format as M described Class (C): 2 bits. This field follows the same format as M described
above. above.
5. IANA Considerations 4.4. Importing PDU Definitions from Other Documents
Protocols are often specified across multiple documents, either
because the specification of a protocol's data units has changed over
time, or because of explicit extension points contained in the
protocol's original specification. To allow a document to make use
of a previous PDU definition, it is possible to import PDU
definitions (written in the format described in this document) from
other documents.
A PDU definition is imported using the exact phrase "A/An ________ is
formatted as described in <document identifier>". The document
identifier must refer, unambiguously, to an existing document. An
Internet-Draft is identified by its name. RFCs are identified by
"RFC" followed by their number.
5. Open Issues
* Need a simple syntax for defining a list of identical objects, and
a way of referring to the size of the enclosing packet. The
format cannot currently represent RFC 6716 section 3.2.3, and
should be able to (the underlying type system can do so).
* Need some discussion about the checks that the tooling might
perform, and the implications of those checks. For example, the
tooling checks for consistency between the diagram and the
description list of fields, ensuring that fields match by name and
width. -01 of this draft had a field that mismatched because of
case: is this something that the tooling should identify? More
broadly, what is the trade-off between the rigour that the tooling
can enforce, and the flexibility desired/needed by authors?
* Need to describe the rules governing the import of PDU definitions
from other documents.
6. IANA Considerations
This document contains no actions for IANA. This document contains no actions for IANA.
6. Security Considerations 7. Security Considerations
Poorly implemented parsers are a frequent source of security Poorly implemented parsers are a frequent source of security
vulnerabilities in protocol implementations. Structuring the vulnerabilities in protocol implementations. Structuring the
description of a protocol data unit so that a parser can be description of a protocol data unit so that a parser can be
automatically derived from the specification can reduce the automatically derived from the specification can reduce the
likelihood of vulnerable implementations. likelihood of vulnerable implementations.
7. Acknowledgements As described in Section 3, the expressiveness of a protocol
description language has implications for the safety, security, and
computability properties of the parser for the protocol description
language itself, and on the generated parser code for the protocols
described using it. The language-theoretic security ([LANGSEC])
community explores the security implications of programming language
design; the principles developed in that community should guide the
development of protocol description languages.
8. Acknowledgements
The authors would like to thank David Southgate for preparing a The authors would like to thank David Southgate for preparing a
prototype implementation of some of the ideas described here. prototype implementation of some of the ideas described here.
The authors would like to thank Marc Petit-Huguenin for feedback on The authors would like to thank Marc Petit-Huguenin for feedback on
the draft. the draft.
This work has received funding from the UK Engineering and Physical This work has received funding from the UK Engineering and Physical
Sciences Research Council under grant EP/R04144X/1. Sciences Research Council under grant EP/R04144X/1.
8. Informative References 9. Informative References
[RFC8357] Deering, S. and R. Hinden, "Generalized UDP Source Port [RFC8357] Deering, S. and R. Hinden, "Generalized UDP Source Port
for DHCP Relay", RFC 8357, March 2018, for DHCP Relay", RFC 8357, March 2018,
<https://www.rfc-editor.org/info/rfc8357>. <https://www.rfc-editor.org/info/rfc8357>.
[QUIC-TRANSPORT] [QUIC-TRANSPORT]
Iyengar, J. and M. Thomson, "QUIC: A UDP-Based Multiplexed Iyengar, J. and M. Thomson, "QUIC: A UDP-Based Multiplexed
and Secure Transport", Work in Progress, Internet-Draft, and Secure Transport", Work in Progress, Internet-Draft,
draft-ietf-quic-transport-20, 23 April 2019, draft-ietf-quic-transport-20, 23 April 2019,
<http://www.ietf.org/internet-drafts/draft-ietf-quic- <http://www.ietf.org/internet-drafts/draft-ietf-quic-
skipping to change at page 16, line 19 skipping to change at page 18, line 16
Draft, draft-ietf-tram-stunbis-21, 21 March 2019, Draft, draft-ietf-tram-stunbis-21, 21 March 2019,
<http://www.ietf.org/internet-drafts/draft-ietf-tram- <http://www.ietf.org/internet-drafts/draft-ietf-tram-
stunbis-21.txt>. stunbis-21.txt>.
[RFC791] Postel, J., "Internet Protocol", RFC 791, September 1981, [RFC791] Postel, J., "Internet Protocol", RFC 791, September 1981,
<https://www.rfc-editor.org/info/rfc791>. <https://www.rfc-editor.org/info/rfc791>.
[RFC793] Postel, J., "Transmission Control Protocol", RFC 793, [RFC793] Postel, J., "Transmission Control Protocol", RFC 793,
September 1981, <https://www.rfc-editor.org/info/rfc793>. September 1981, <https://www.rfc-editor.org/info/rfc793>.
[LANGSEC] LANGSEC, "LANGSEC: Language-theoretic Security",
<http://langsec.org>.
Appendix A. ABNF specification Appendix A. ABNF specification
A.1. Constraint Expressions A.1. Constraint Expressions
cond-expr = eq-expr "?" cond-expr ":" eq-expr eq-expr = bool-expr eq-op bool-expr bool-expr = ord-expr bool-op ord-expr ord-expr = add-expr ord-op add-expr add-expr = mul-expr add-op mul-expr mul-expr = expr mul-op expr expr = *DIGIT / field-name / field-name-ws / "(" expr ")" field-name = *ALPHA field-name-ws = *(field-name " ") mul-op = "*" / "/" / "%" add-op = "+" / "-" ord-op = "<=" / "<" / ">=" / ">" bool-op = "&&" / "||" / "!" eq-op = "==" / "!=" cond-expr = eq-expr "?" cond-expr ":" eq-expr
eq-expr = bool-expr eq-op bool-expr
bool-expr = ord-expr bool-op ord-expr
ord-expr = add-expr ord-op add-expr
add-expr = mul-expr add-op mul-expr
mul-expr = expr mul-op expr
expr = *DIGIT / field-name /
field-name-ws / "(" expr ")"
field-name = *ALPHA
field-name-ws = *(field-name " ")
mul-op = "*" / "/" / "%"
add-op = "+" / "-"
ord-op = "<=" / "<" / ">=" / ">"
bool-op = "&&" / "||" / "!"
eq-op = "==" / "!="
A.2. Augmented packet diagrams A.2. Augmented packet diagrams
Future revisions of this draft will include an ABNF specification for Future revisions of this draft will include an ABNF specification for
the augmented packet diagram format described in Section 4. Such a the augmented packet diagram format described in Section 4. Such a
specification is omitted from this draft given that the format is specification is omitted from this draft given that the format is
likely to change as its syntax is developed. Given the visual nature likely to change as its syntax is developed. Given the visual nature
of the format, it is more appropriate for discussion to focus on the of the format, it is more appropriate for discussion to focus on the
examples given in Section 4. examples given in Section 4.
 End of changes. 39 change blocks. 
79 lines changed or deleted 182 lines changed or added

This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/