INTERNET-DRAFT T. Herbert Intended Status: Informational Quantonium Expires: March 2019 September 25, 2018 Sub-path Transport Layer Problem Statement draft-herbert-sub-path-ps-00 Abstract This document presents the problem statement and use cases for a sub- path transport layer. A sub-path is a portion of a network path that has localized characteristics that are of interest to an operator to manage. The structure for managing a sub-path is the "sub-path transport layer". A sub-path transport layer can provide transport layer mechanisms, such as reliability and congestion control, over the aggregate of packets traversing the sub-path. Additionally, a sub-path transport layer can provide performance measurements of traffic over a sub-path which in turn can be input to operations and management. The sub-path transport layer implies the possibility that traffic may be subject to two transport layer control loops, that of the sub-path and that of a higher layer transport protocol, any such solution must consider the ramifications of that. Status of this Memo This Internet-Draft is submitted to IETF in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/1id-abstracts.html The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html T. Herbert Expires March 29, 2019 [Page 1] INTERNET DRAFT draft-herbert-sub-path-ps Copyright and License Notice Copyright (c) 2018 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1 Passive measurement . . . . . . . . . . . . . . . . . . . . 3 1.2 Transport layer mechanisms . . . . . . . . . . . . . . . . . 3 2 Use cases . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.1 Lossy access network . . . . . . . . . . . . . . . . . . . . 4 2.2 Low latency handover . . . . . . . . . . . . . . . . . . . . 4 2.3 Congestion control for nest of mice . . . . . . . . . . . . 5 3 Existing solutions . . . . . . . . . . . . . . . . . . . . . . 6 3.1 Layer 2 protocols with transport layer functionality . . . . 6 3.2 TCP proxies . . . . . . . . . . . . . . . . . . . . . . . . 6 3.3 In-situ OAM . . . . . . . . . . . . . . . . . . . . . . . . 7 4 Sub-path transport layer solution . . . . . . . . . . . . . . . 7 4.1 Sub-path tunnels . . . . . . . . . . . . . . . . . . . . . . 7 4.2 Nested control loop problem . . . . . . . . . . . . . . . . 7 4.2.1 Avoiding conflict with higher layer transports . . . . . 8 4.2.2 Work in concert with higher layer transports . . . . . . 8 4.3 Advanced features . . . . . . . . . . . . . . . . . . . . . 8 5 References . . . . . . . . . . . . . . . . . . . . . . . . . . 9 5.1 Normative References . . . . . . . . . . . . . . . . . . . 9 5.2 Informative References . . . . . . . . . . . . . . . . . . 9 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 9 T. Herbert Expires March 29, 2019 [Page 2] INTERNET DRAFT draft-herbert-sub-path-ps 1 Introduction This document presents the problem statement that motivates defining a sub-path transport layer. A sub-path is a portion a network path for a flow that is typically contained within some network. For instance, a sub-path may constitute a path from an ingress to an egress point of a transit network. A sub-path may be common to many flows where the aggregate of packets traversing the sub-path could be termed a "sub-path flow". We propose a "sub-path transport layer" that can be used to manage traffic over a sub-path layer with transport layer semantics. There are two goals in this: o Perform passive measurement of packets traversing a sub-path o Implement transport layer mechanisms, like reliability and congestion control, on the sub-path flow 1.1 Passive measurement A network operator may be interested in performance metrics of packets flowing over a sub-path. The motivation for measurement is operations and management (OAM), and the data might be used for such things as traffic engineering and route selection. A sub-path transport layer would provide measurements and statistics for a sub- path flow that are analogous to that provided by a canonical transport layer for a transport flow. The data would include packet and byte counts, latency and latency variance, packet loss, and out of order packets. 1.2 Transport layer mechanisms Sub-paths may exhibit radically different characteristics compared to the rest of the path for a flow. For instance, a radio network portion of a path may be much more susceptible to congestion and packet loss relative to the rest of a flow's path. End-to-end mechanisms for congestion control and reliability don't take this into account, so transport layer end points are often far removed from problems. The result is that congestion control and reliability is sub-optimal and have high response times to problems originating in a sub-path. The proposed solution is to run a congestion control and reliability protocol over sub-paths. This would constitute an in-network transport layer below the end-to-end transport (e.g. TCP) that applies transport layer algorithms to manage a sub-path flow. T. Herbert Expires March 29, 2019 [Page 3] INTERNET DRAFT draft-herbert-sub-path-ps 2 Use cases This section describes some potential use cases of a sub-path transport layer. 2.1 Lossy access network Certain radio technologies, such as mmWave (millimeter wave) cellular systems, offer the opportunity to use an untapped frequency spectrum to achieve high data rates. However, the harsh propagation conditions of mmWave present a considerable challenge. A mmWave link may be both lossy and have high variability in data rates that adversely affects performance of end to end transport protocols like TCP. As shown in the diagram below, a sub-path transport protocol could be used over a mmWave link that provides fast retransmissions for packet loss on the link, as well as responsive congestion control that adapts quickly to changing data rates. In this case, the endpoints of the sub-path transport layer (denoted SPEP in this document) are the end device (UE) and a node in a provider network (PGW). _______ ______ ( ) ( ) +------+ ( Radio ) +------+ ( ) +--------+ | UE |--( Network )--| PGW |--( Internet )--| Server | | Host | ( ) | SPEP | ( ) +--------+ +------+ (_______) +------+ (______) <-------------------------> Sub-path transport <------------------------------------------------------------> End-to-end transport 2.2 Low latency handover In a cellular network, handover is the process of transferring communications from one channel to another. Handover is a very common operation as mobile devices move from one cell to another, so that the point of attachment changes from one cellular base station to another. Handover is not instantaneous, especially considering that most devices can only connect to one channel at a time. Handover latency is defined as the time between the last reception of data to when the device starts receiving packets at the new point of attachment. Handover latency can be in the hundreds of milliseconds. Handover of UE (user equipment) is illustrated below. T. Herbert Expires March 29, 2019 [Page 4] INTERNET DRAFT draft-herbert-sub-path-ps +-- +-----+ ________ ______ | | gNB +--+ ( ) ( ) +----+ +-----+ \ ( Provider ) +-----+ ( ) +------+ | UE | ( RAN )---+ PGW +---( Internet )--+ Peer | +----+ +-----+ / ( ) +-----+ ( ) +------+ | | gNB +--+ (________) (______) +-> +-----+ <--------------------------> Sub-path flow over RAN When a handover occurs, packets for the device that has moved need to be forwarded to new point of attachment. The timing of events to complete negotiation and handover is non-deterministic so that the network does not know the precise time at which events occur. A proposed optimization is that during the handover period, packets are duplicated and forwarded to both the old and new point of attachment. This potentially reduces the handover latency. A sub-path transport protocol facilitates this with sequence numbers to avoid delivery of duplicate packets as well as to maintain ordering during a handover. 2.3 Congestion control for nest of mice Large IoT networks may contain millions of devices and billions of flows that periodically do request/response communications with small amounts of data. Such a collection of flows could be called a "nest of mice". The nest of mice may share a critical link that can become congested due to the aggregate of traffic. End-to-end congestion control is ineffective since each connection generates little data so there is little input to any particular transport control loop. A sub-transport layer could aggregate all the mice flows together and treat them as one single large flow. The control loop for the aggregate flow has sufficient input to perform effective congestion control over sub-path. Aggregation of mice flows over a sub-path transport layer is illustrated in the diagram below. +------+ _____ ______ | Host | ( ) ( ) +------+ +------+ ( ) +------+ ( ) +--------+ ... | SPEP |--( Network )--| SPEP |--( Internet )--| Server | +------+ +------+ ( ) +------+ ( ) +--------+ | Host | (_____) (______) +------+ FLOW\ FLOW \ <--------------------------> .... / Aggregated sub-path flow FLOW/ T. Herbert Expires March 29, 2019 [Page 5] INTERNET DRAFT draft-herbert-sub-path-ps 3 Existing solutions This sections surveys some related solutions. 3.1 Layer 2 protocols with transport layer functionality A number of link layer protocols implement transport layer semantics such as reliability. Link-layer retransmission is a feature of the IEEE 802.11 standard that aims to increase the reliability of data communications. [LLRETRANS] analyzes the impact link layer retransmission on video streaming over a wireless mesh. The conclusion of that work is that link layer retranmission provides the most benefit when the number of retransmissions is limited (to one or two), and the traffic over the mesh network is far below network capacity (in other words there is no risk that retransmitted packets contribute to congestion). 3.2 TCP proxies TCP proxies are often used in mobile networks to enhance performance by running proxy connections over radio sub-paths. [MILLIPROXY] is a proposal to use TCP proxies to optimize communications in 5G mmWave. That paper describes the unique characteristics of high bandwidth mmWave links, and why end-to-end TCP gives sub-optimal performance in that environment. A TCP proxy can split the path over two TCP connections, where one connection is overlaid on the radio network. That connection is then very close to the sub-path of interest (the mmWave link), thus both congestion control and reliability are isolated and are tailored to the characteristics of the link. While TCP proxies can optimize traffic in specific situations like this, as a general solution to the problem they have a number of drawbacks: * They only work with TCP and no other transport protocol. In particular, they don't work with QUIC, SCTP, etc. * Proxies break the end-to-end model. This is most evident in that they break transport layer security like the TCP authentication option. They also ossify TCP since the options that can be used become the least common denominator of what the endpoints and the proxy supports. * Proxies are a single point of failure and potential network bottleneck. * Proxies require considerable transport state to be maintained in T. Herbert Expires March 29, 2019 [Page 6] INTERNET DRAFT draft-herbert-sub-path-ps the network. This is very hard to scale especially in light of 5G scaling goals of a trillion devices attached to the Internet. 3.3 In-situ OAM [IOAM] defines a scheme to measure per hop latencies and queue delay for packets. Whereas in-situ OAM performs low level measurements that assume participation by intermediate nodes to provide data, the sub- path transport layer provides end-to-end measurements over the sub- path. Both mechanisms measure performance of the network, but at different levels. In this sense they may be considered complementary techniques, and it is conceivable that the two techniques could be used in concert to fully characterize the performance of a sub-path. 4 Sub-path transport layer solution Sub-paths within a network can be defined arbitrarily by the network operator. Generally, sub-paths consist of two network nodes, denoted "sub-path end points", and the path between them in both directions. Sub-paths may segregate packets by class or other convenient characteristics; for instance, a sub-path for high priority packets and one for low priority packets could be defined. There is no one to one correspondence between sub-path flows and higher layer transport flows, neither is there any requirement that all packets of a transport flow traverse the same sub-path or sub-path transport layer. 4.1 Sub-path tunnels To achieve a generic, well layered solution, a sub-path transport layer protocol could be specified and implemented for use over network tunnels. That is to use an network overlay to traverse a sub- path. The tunnel endpoints are the sub-path endpoints, and the wire protocol for the sub-path transport protocol can be contained in optional data of an encapsulation protocol for the tunnel. The solution should be generic so that it can be used with any encapsulation protocol that is reasonably extensible: GUE, Geneve, and GTP-U are candidates. It is also plausible that an IPv6 destination option could be used so that sub-path transport layer could generally work with any IPv6 encapsulation. 4.2 Nested control loop problem An important consideration is how a sub-path transport layer interacts with end-to-end transport protocols. Packets of a flow may be subject to two (possibly more) transport control loops, that of the sub-path and that of a higher layer transport protocol. Previous experience has shown this to be problematic and yield poor transport T. Herbert Expires March 29, 2019 [Page 7] INTERNET DRAFT draft-herbert-sub-path-ps behavior. To this end, there are two levels of interaction that could be targeted in a sub-path transport protocol: 1) avoid conflict with higher layer transport protocols 2) work in concert with higher layer transport protocols 4.2.1 Avoiding conflict with higher layer transports To avoid conflict with higher layer transport protocols, one must take into account the effect of an algorithm at a higher layer. For instance, consider that a sub-path transport layer implements retransmissions. If a packet is lost and can quickly be retransmitted within the RTT variance of the end-to-end flow, then the algorithm is potentially beneficial. If a sub-path retransmission timer is long, or multiple retransmissions are required, then the algorithm might interfere with the higher layer algorithms. In the worst case, an endpoint sender might retransmit a packet that is still in the process of retransmitted over a sub-path needlessly increasing congestion in the sub-path. This motivates "best effort" approach to a sub-path transport layer. For instance, a sub-path transport layer might retransmit only once (as motivated by data in [LLRETRANS]). 4.2.2 Work in concert with higher layer transports Creating a solution that works in concert with upper layer transport protocols might be compelling. This would necessitate some signaling between the end-to-end transport and the sub-path transport layer. Explicit congestion notification (ECN) is a signal that could be used for the sub-path transport to signal congestion conditions to the higher transport layer. Some intermediate nodes will do deep packet inspection (DPI) into TCP in order to extract sequence numbers, round trip times (RTT), and acknowledgements in order to deduce things like retransmissions. Parsing into the TCP headers is not a generic solution and generally contributes to protocol ossification, however there are emerging techniques such as in-situ OAM in IPv6 Hop-by-Hop headers that may be useful. If a sub-path transport layer knew the round trip latency and variance of a flow, then it might be able to use that information to determine the number of retransmissions that could be tolerated over the sub-path before a packet is retransmitted at a higher layer. 4.3 Advanced features In addition to the typical transport features of reliability and congestion control, a sub-path transport layer may also allow features and optimizations that that are more specific to T. Herbert Expires March 29, 2019 [Page 8] INTERNET DRAFT draft-herbert-sub-path-ps characteristics of a particular sub-path. These might include: o Forward error correction o Purposely duplicating packets (for low latency handover) o Random packet spraying across a sub-paths o Optimized path selection and routing 5 References 5.1 Normative References 5.2 Informative References Author's Address Tom Herbert Quantonium Santa Clara, CA USA Email: tom@quantonium.net T. Herbert Expires March 29, 2019 [Page 9]