An Architecture for Transport ServicesApple Inc.One Apple Park WayCupertino, California 95014United States of Americatpauly@apple.comETH ZurichGloriastrasse 358092 ZurichSwitzerlandietf@trammell.chKarlstad UniversityUniversitetsgatan 2651 88 KarlstadSwedenanna.brunstrom@kau.seUniversity of AberdeenFraser Noble BuildingAberdeen, AB24 3UEScotlandgorry@erg.abdn.ac.ukhttp://www.erg.abdn.ac.uk/University of GlasgowSchool of Computing ScienceGlasgow G12 8QQUnited Kingdomcsp@csperkins.orgTU BerlinMarchstraße 2310587 BerlinGermanyphilipp@inet.tu-berlin.deApple Inc.One Apple Park WayCupertino, California 95014United States of Americacawood@apple.com
Transport
TAPS Working GroupInternet-DraftThis document provides an overview of the architecture of Transport Services, a system for exposing the features of transport protocols to applications. This architecture serves as a basis for Application Programming Interfaces (APIs) and implementations that provide flexible transport networking services. It defines the common set of terminology and concepts to be used in more detailed discussion of Transport Services.Many APIs to perform transport networking have been deployed, perhaps the most
widely known and imitated being the BSD socket() interface. The names and
functions between these APIs are not consistent, and vary depending on the
protocol being used. For example, sending and receiving on a stream of data is
conceptually the same between operating on an unencrypted Transmission Control
Protocol (TCP) stream and operating on an encrypted Transport Layer Security
(TLS) stream over TCP, but applications cannot use
the same socket send() and recv() calls on top of both kinds of connections.
Similarly, terminology for the implementation of protocols offering transport
services vary based on the context of the protocols themselves. This variety
can lead to confusion when trying to understand the similarities and
differences between protocols, and how applications can use them effectively.The goal of the Transport Services architecture is to provide a common, flexible, and reusable interface for transport protocols. As applications adopt this interface, they will benefit from a wide set of transport features that can evolve over time, and ensure that the system providing the interface can optimize its behavior based on the application requirements and network conditions.This document is developed in parallel with the specification of the Transport Services API and Implementation documents.The Transport Services architecture is based on the survey of Services Provided by IETF Transport Protocols and Congestion Control Mechanisms , and the distilled minimal set of the features offered by transport protocols . This work has identified common features and patterns across all transport protocols developed thus far in the IETF.Since transport security is an increasingly relevant aspect of using transport protocols on the Internet, this architecture also considers the impact of transport security protocols on the feature set exposed by transport services .One of the key insights to come from identifying the minimal set of features provided by transport protocols was that features either require application interaction and guidance (referred to as Functional Features), or else can be handled automatically by a system implementing Transport Services (referred to as Automatable Features). Among the Functional Features, some were common across all or nearly all transport protocols, while others could be seen as features that, if specified, would only be useful with a subset of protocols, or perhaps even a single transport protocol, but would not harm the functionality of other protocols. For example, some protocols can deliver messages faster for applications that do not require them to arrive in the order in which they were sent. However, this functionality must be explicitly allowed by the application, since reordering messages would be undesirable in many cases.The goal of the Transport Services architecture is to redefine the interface between applications and transports in a way that allows the transport layer to evolve and improve without fundamentally changing the contract with the application. This requires a careful consideration of how to expose the capabilities of protocols.There are several degrees in which a Transport Services system can offer flexibility to an application: it can provide access to multiple sets of protocols and protocol features, it can use these protocols across multiple paths that may have different performance and functional characteristics, and it can communicate with different Remote Endpoints to optimize performance. Beyond these, if the API for the system remains the same over time, new protocols and features may be added to the system’s implementation without requiring changes in applications for adoption.The following considerations were used in the design of this architecture.Functionality that is common across multiple transport protocols should be accessible through a unified set of API calls. An application should be able to implement logic for its basic use of transport networking (establishing the transport, and sending and receiving data) once, and expect that implementation to continue to function as the transports change.Any Transport Services API must allow access to the distilled minimal set of features offered by transport protocols .Since applications will often need to control fine-grained details of transport protocols to optimize their behavior and ensure compatibility with remote peers, a Transport Services system also needs to allow more specialized protocol features to be used. The interface for these specialized options should be exposed differently from the common options to ensure flexibility.A specialized feature may be required by an application only when using a specific protocol, and not when using others. For example, if an application is using UDP, it may require control over the checksum or fragmentation behavior for UDP; if it used a protocol to frame its data over a byte stream like TCP, it would not need these options. In such cases, the API should expose the features in such a way that they take effect when a particular protocol is selected, but do not imply that only that protocol may be used if there are equivalent options.Other specialized features, however, may be strictly required by an application and thus constrain the set of protocols that can be used. For example, if an application requires encryption of its transport data, only protocol stacks that include some transport security protocol are eligible to be used. A Transport Services API must allow applications to define such requirements and constrain the system’s options. Since such options are not part of the core/common features, it should be simple for an application to modify its set of constraints and change the set of allowable protocol features without changing the core implementation.The Transport Services API is envisioned as the abstract model for a family of APIs that share a common way to expose transport features and encourage flexibility. The abstract API definition describes this interface and is aimed at application developers.Implementations that provide the Transport Services API will vary due to system-specific support and the needs of the deployment scenario. It is expected that all implementations of Transport Services will offer the entire mandatory API, but that some features will not be functional in certain implementations. All implementations must offer sufficient APIs to use the distilled minimal set of features offered by transport protocols , including API support for TCP and UDP transport, but it is possible that some very constrained devices might not have, for example, a full TCP implementation.In order to preserve flexibility and compatibility with future protocols, top-level features in the Transport Services API should avoid referencing particular transport protocols. Mappings of these API features in the Implementation document, on the other hand, must explain the ramifications of each feature on existing protocols. It is expected that the Implementation document will be updated and supplemented as new protocols and protocol features are developed.It is important to note that neither the Transport Services API nor the Implementation document defines new protocols that require any changes on remote hosts. The Transport Services system must be deployable on one side only, as a way to allow an application to make better use of available capabilities on a system and protocol features that may be supported by peers across the network.The concepts defined in this document are intended primarily for use in the documents and specifications that describe the Transport Services architecture and API. While the specific terminology may be used in some implementations, it is expected that there will remain a variety of terms used by running code.The architecture divides the concepts for Transport Services into two categories:API concepts, which are meant to be exposed to applications; andSystem-implementation concepts, which are meant to be internally used when building systems that implement Transport Services.The following diagram summarizes the top-level concepts in the architecture and how they relate to one another.Fundamentally, a Transport Services API needs to provide basic objects () that allow applications to establish communication and send and receive data. These may be exposed as handles or referenced objects, depending on the language.Beyond the basic objects, there are several high-level groups of actions that any Transport Services API must provide:Pre-Establishment () encompasses the properties that an application can pass to describe its intent, requirements, prohibitions, and preferences for its networking operations. For any system that provides generic Transport Services, these properties should primarily offer knobs that are applicable to multiple transports. Properties may have a large impact on the rest of the aspects of the interface: they can modify how establishment occurs, they can influence the expectations around data transfer, and they determine the set of events that will be supported.Establishment () focuses on the actions that an application takes on the basic objects to prepare for data transfer.Data Transfer () consists of how an application represents data to be sent and received, the functions required to send and receive that data, and how the application is notified of the status of its data transfer.Event Handling () defines the set of properties about which an application can receive notifications during the lifetime of transport objects. Events can also provide opportunities for the application to interact with the underlying transport by querying state or updating maintenance options.Termination () focuses on the methods by which data transmission is ceased, and state is torn down in the transport.The diagram below provides a high-level view of the actions taken during the lifetime of a connection.Preconnection: A Preconnection object is a representation of a potential connection. It has state that describes parameters of a Connection that might exist in the future: the Local Endpoint from which that Connection will be established, the Remote Endpoint to which it will connect, and Path Selection Properties, Protocol Selection Properties, and Specific Protocol Properties that influence the choice of transport that a Connection will use. A Preconnection can be fully specified and represent a single possible Connection, or it can be partially specified such that it represents a family of possible Connections. The Local Endpoint must be specified if the Preconnection is used to Listen for incoming connections, but is optional if it is used to Initiate connections. The Remote Endpoint must be specified in the Preconnection is used to Initiate connections, but is optional if it is used to Listen for incoming connections. The Local Endpoint and the Remote Endpoint must both be specified if a peer-to-peer Rendezvous is to occur based on the Preconnection.Connection: A Connection object represents an active transport protocol instance that can send and/or receive Messages between a Local Endpoint and a Remote Endpoint. It holds state pertaining to the underlying transport protocol instance and any ongoing data transfer. This represents, for example, an active connection in a connection-oriented protocol such as TCP, or a fully-specified 5-tuple for a connectionless protocol such as UDP.Listener: A Listener object accepts incoming transport protocol connections from Remote Endpoints and generates corresponding Connection objects. It is created from a Preconnection object that specifies the type of incoming connections it will accept.Endpoint: An Endpoint represents one side of a transport connection.
Endpoints can be Local Endpoints or Remote Endpoints, and respectively represent an identity
that the application uses for the source or destination of a connection.
Endpoint can vary in levels of specificity, and can be resolved to more concrete identities.Remote Endpoint: The Remote Endpoint represents the application’s name for a peer that can participate in a transport connection. For example, the combination of a DNS name for the peer and a service name/port.Local Endpoint: The Local Endpoint represents the application’s name for itself that it wants to use for transport connections. For example, a local IP address and port.Path Selection Properties: The Path Selection Properties consist of the options that an application may set to influence the selection of paths between itself and the Remote Endpoint. These options can come in the form of requirements, prohibitions, or preferences. Examples of options which may influence path selection include the interface type (such as a Wi-Fi Ethernet connection, or a Cellular LTE connection), characteristics of the path that are locally known like Maximum Transmission Unit (MTU) or discovered like Path MTU (PMTU), or predicted based on cached information like expected throughput or latency.Protocol Selection Properties: The Protocol Selection Properties consist of the options that an application may set to influence the selection of transport protocol, or to configure the behavior of generic transport protocol features. These options come in the form of requirements, prohibitions, and preferences. Examples include reliability, service class, multipath support, and fast open support.Specific Protocol Properties: The Specific Protocol Properties refer to the subset of Protocol Properties options that apply to a single protocol (transport protocol, IP, or security protocol). The presence of such Properties does not necessarily require that a specific protocol must be used when a Connection is established, but that if this protocol is employed, a particular set of options should be used.Initiate is the primary action that an application can take to create a Connection to a remote endpoint, and prepare any required local or remote state to be able to send and/or receive Messages. For some protocols, this may initiate a client-to-server style handshake; for other protocols, this may just establish local state. The process of identifying options for connecting, such as resolution of the Remote Endpoint, occurs in response the Initiate call.Listen is the action of marking a Listener as willing to accept incoming Connections. The Listener will then create Connection objects as incoming connections are accepted ().Rendezvous is the action of establishing a peer-to-peer connection with a remote endpoint. It simultaneously attempts to initiate a connection to a remote endpoint whilst listening for an incoming connection from that endpoint. This corresponds, for example, to a TCP simultaneous open. The process of identifying options for the connection, such as resolution of the Remote Endpoint, occurs during the Rendezvous call. If successful, the rendezvous call returns a Connection object to represent the established peer-to-peer connection.Message: A Message object is a unit of data that can be represented as bytes that can be transferred between two endpoints over a transport connection. The bytes within a Message are assumed to be ordered within the Message. If an application does not care about the order in which a peer receives two distinct spans of bytes, those spans of bytes are considered independent Messages. Messages may or may not be usable if incomplete or corrupted. Boundaries of a Message may or may not be understood or transmitted by transport protocols. Specifically, what one application considers to be two Messages sent on a stream-based transport may be treated as a single Message by the application on the other side.Send is the action to transmit a Message or partial Message over a Connection to a Remote Endpoint. The interface to Send may include options specific to how the Message’s content is to be sent. Status of the Send operation may be delivered back to the application in an event ().Receive is an action that indicates that the application is ready to asynchronously accept a Message over a Connection from a Remote Endpoint, while the Message content itself will be delivered in an event (). The interface to Receive may include options specific to the Message that is to be delivered to the application.This list of events that can be delivered to an application is not exhaustive, but gives the top-level categories of events. The API may expand this list.Connection Ready: Signals to an application that a given Connection is ready to send and/or receive Messages. If the Connection relies on handshakes to establish state between peers, then it is assumed that these steps have been taken.Connection Finished: Signals to an application that a given Connection is no longer usable for sending or receiving Messages. This should deliver an error to the application that describes the nature of the termination.Connection Received: Signals to an application that a given Listener has passively received a Connection.Message Received: Delivers received Message content to the application, based on a Receive action. This may include an error if the Receive action cannot be satisfied due to the Connection being closed.Message Sent: Notifies the application of the status of its Send action. This may be an error if the Message cannot be sent, or an indication that Message has been processed by the protocol stack.Path Properties Changed: Notifies the application that some property of the Connection has changed that may influence how and where data is sent and/or received.Close is the action an application may take on a Connection to indicate that it no longer intends to send data, is no longer willing to receive data, and that the protocol should signal this state to the remote endpoint if applicable.Abort is an action the application may take on a Connection to indicate a Close, but with the additional indication that the transport system should not attempt to deliver any outstanding data.The Transport System Implementation Concepts define the set of objects used internally to a system or library to provide the functionality of transport networking, as required by the abstract interface.Connection Group: A Connections Group is a set of Connections that share properties. For multiplexing transport protocols, the Connection Group defines the set of Connections that can be multiplexed together.Path: A Path represents an available set of properties of a network route on which packets may be sent or received.Protocol Instance: A Protocol Instance is a single instance of one protocol, including any state it has necessary to establish connectivity or send and receive Messages.Protocol Stack: A Protocol Stack is a set of Protocol Instances (including relevant application, security, transport, or Internet protocols) that are used together to establish connectivity or send and receive Messages. A single stack may be simple (a single transport protocol instance over IP), or complex (multiple application protocol streams going through a single security and transport protocol, over IP; or, a multi-path transport protocol over multiple transport sub-flows).System Policy: System Policy represents the input from an operating system or other global preferences that can constrain or influence how an implementation will gather candidate paths and protocols () and race the candidates during establishment (). Specific aspects of the System Policy may apply to all Connections, or only certain ones depending on the runtime context and properties of the Connection.Cached State: Cached State is the state and history that the implementation keeps for each set of associated endpoints that have been used previously. This can include DNS results, TLS session state, previous success and quality of transport protocols over certain paths.Path Selection: Path Selection represents the act of choosing one or more paths that are available to use based on the Path Selection Properties provided by the application, and a Transport Services system’s policies and heuristics.Protocol Selection: Protocol Selection represents the act of choosing one or more sets of protocol options that are available to use based on the Protocol Properties provided by the application, and a Transport Services system’s policies and heuristics.Protocol Option Racing: Protocol Racing is the act of attempting to establish, or scheduling attempts to establish, multiple Protocol Stacks that differ based on the composition of protocols or the options used for protocols.Path Racing: Path Racing is the act of attempting to establish, or scheduling attempts to establish, multiple Protocol Stacks that differ based on a selection from the available Paths.Endpoint Racing: Endpoint Racing is the act of attempting to establish, or scheduling attempts to establish, multiple Protocol Stacks that differ based on the specific representation of the Remote Endpoint and the Local Endpoint, such as IP addresses resolved from a DNS hostname.While some transports expose a byte stream abstraction, most higher level
protocols impose some structure onto that byte stream. That is, the higher
level protocol operates in terms of messages, protocol data units (PDUs),
rather than using unstructured sequences of bytes, with each message being
processed in turn. Protocols are specified in terms of state machines
acting on semantic messages, with parsing the byte stream into messages
being a necessary annoyance, rather than a semantic concern. Accordingly,
the Transport Services architecture exposes messages as the primary
abstraction. Protocols that deal only in byte streams, such as TCP,
represent their data in each direction as a single, long message. When
framing protocols are placed on top of byte streams, the messages used in
the API represent the framed messages within the stream.Providing a message-based abstraction also provides:the ability to associate deadlines with messages, for transports that
care about timing;the ability to provide control of reliability, choosing what messages to
retransmit in the event of packet loss, and how best to make use of the
data that arrived;the ability to manage dependencies between messages, when some messages
may not be delivered due to either packet loss or missing a deadline, in
particular the ability to avoid (re-)sending data that relies on a previous
transmission that was never received.All require explicit message boundaries, and application-level framing of
messages, to be effective. Once a message is passed to the transport, it
can not be cancelled or paused, but prioritization as well as lifetime and
retransmission management will provide the protocol stack with all needed
information to send the messages as quickly as possible without blocking
transmission unnecessarily. The transport services architecture facilitates
this by handling messages, with known identity (sequence numbers, in the
simple case), lifetimes, niceness, and antecedents.Transport protocols such as SCTP provide a message-oriented API that has
similar features to those we describe. Other transports, such as TCP, do
not. To support a message oriented API, while still being compatible with
stream-based transport protocols, implementations of the transport services
architecture should provide APIs for framing and de-framing messages. That
is, we push message framing down into the transport services API, allowing
applications to send and receive complete messages. This is backwards
compatible with existing protocols and APIs, since the wire format of
messages does not change, but gives the protocol stack additional
information to allow it to make better use of modern transport services.RFC-EDITOR: Please remove this section before publication.This document has no actions for IANA.The Transport Services architecture does not recommend use of specific
security protocols or algorithms. Its goal is to offer ease of use for
existing protocols by providing a generic security-related interface. Each
provided interface mimics an existing protocol-specific interface provided by
supported security protocols. For example, trust verification callbacks are
common parts of TLS APIs. Transport Services APIs will expose similar
functionality.Clients must take care to use security APIs appropriately. In cases where
clients use said interface to provide sensitive keying material, e.g., access
to private keys or copies of pre-shared keys (PSKs), key use must be
validated. For example, clients should not use PSK material created for the
Encapsulating Security Protocol (ESP, part of IPsec) with QUIC, and clients
must not use private keys intended for server authentication as a keys for
client authentication. Moreover, unlike certain transport features such as TCP
Fast Open (TFO) or Explicit Congestion Notification (ECN)
which can fall back to standard configurations, Transport
Services systems must not permit fallback for security protocols. For example,
if a client requests TLS, yet TLS or the desired version are not available,
its connection must fail. Clients are responsible for implementing protocol or
version fallback using a Transport Services API if so desired.This work has received funding from the European Union’s Horizon 2020 research and
innovation programme under grant agreement No. 644334 (NEAT).This work has been supported by Leibniz Prize project funds of DFG - German
Research Foundation: Gottfried Wilhelm Leibniz-Preis 2011 (FKZ FE 570/4-1).This work has been supported by the UK Engineering and Physical Sciences
Research Council under grant EP/R04144X/1.Thanks to Stuart Cheshire, Josh Graessley, David Schinazi, and Eric Kinnear for their implementation and design efforts, including Happy Eyeballs, that heavily influenced this work.The Addition of Explicit Congestion Notification (ECN) to IPThis memo specifies the incorporation of ECN (Explicit Congestion Notification) to TCP and IP, including ECN's use of two bits in the IP header. [STANDARDS-TRACK]IP Encapsulating Security Payload (ESP)This document describes an updated version of the Encapsulating Security Payload (ESP) protocol, which is designed to provide a mix of security services in IPv4 and IPv6. ESP is used to provide confidentiality, data origin authentication, connectionless integrity, an anti-replay service (a form of partial sequence integrity), and limited traffic flow confidentiality. This document obsoletes RFC 2406 (November 1998). [STANDARDS-TRACK]TCP Fast OpenThis document describes an experimental TCP mechanism called TCP Fast Open (TFO). TFO allows data to be carried in the SYN and SYN-ACK packets and consumed by the receiving end during the initial connection handshake, and saves up to one full round-trip time (RTT) compared to the standard TCP, which requires a three-way handshake (3WHS) to complete before data can be exchanged. However, TFO deviates from the standard TCP semantics, since the data in the SYN could be replayed to an application in some rare circumstances. Applications should not use TFO unless they can tolerate this issue, as detailed in the Applicability section.Services Provided by IETF Transport Protocols and Congestion Control MechanismsThis document describes, surveys, and classifies the protocol mechanisms provided by existing IETF protocols, as background for determining a common set of transport services. It examines the Transmission Control Protocol (TCP), Multipath TCP, the Stream Control Transmission Protocol (SCTP), the User Datagram Protocol (UDP), UDP-Lite, the Datagram Congestion Control Protocol (DCCP), the Internet Control Message Protocol (ICMP), the Real-Time Transport Protocol (RTP), File Delivery over Unidirectional Transport / Asynchronous Layered Coding (FLUTE/ALC) for Reliable Multicast, NACK- Oriented Reliable Multicast (NORM), Transport Layer Security (TLS), Datagram TLS (DTLS), and the Hypertext Transport Protocol (HTTP), when HTTP is used as a pseudotransport. This survey provides background for the definition of transport services within the TAPS working group.The Transport Layer Security (TLS) Protocol Version 1.3This document specifies version 1.3 of the Transport Layer Security (TLS) protocol. TLS allows client/server applications to communicate over the Internet in a way that is designed to prevent eavesdropping, tampering, and message forgery. This document updates RFCs 4492, 5705, and 6066 and it obsoletes RFCs 5077, 5246, and 6961. This document also specifies new requirements for TLS 1.2 implementations.A Minimal Set of Transport Services for TAPS SystemsThis draft recommends a minimal set of IETF Transport Services offered by end systems supporting TAPS, and gives guidance on choosing among the available mechanisms and protocols. It is based on the set of transport features in RFC 8303.A Survey of Transport Security ProtocolsThis document provides a survey of commonly used or notable network security protocols, with a focus on how they interact and integrate with applications and transport protocols. Its goal is to supplement efforts to define and catalog transport services [RFC8095] by describing the interfaces required to add security protocols. It examines Transport Layer Security (TLS), Datagram Transport Layer Security (DTLS), Quick UDP Internet Connections with TLS (QUIC + TLS), MinimalT, CurveCP, tcpcrypt, Internet Key Exchange with Encapsulating Security Protocol (IKEv2 + ESP), SRTP (with DTLS), and WireGuard. This survey is not limited to protocols developed within the scope or context of the IETF.Implementing Interfaces to Transport ServicesAn Abstract Application Layer Interface to Transport ServicesThis document describes an abstract programming interface to the transport layer, following the Transport Services Architecture. It supports the asynchronous, atomic transmission of messages over transport protocols and network paths dynamically selected at runtime. It is intended to replace the traditional BSD sockets API as the lowest common denominator interface to the transport layer, in an environment where endpoints have multiple interfaces and potential transport protocols to select from.