idnits 2.17.1 draft-irtf-panrg-what-not-to-do-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (May 23, 2019) is 1790 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-04) exists of draft-thomson-use-it-or-lose-it-03 -- Obsolete informational reference (is this intentional?): RFC 793 (Obsoleted by RFC 9293) -- Obsolete informational reference (is this intentional?): RFC 1190 (Obsoleted by RFC 1819) -- Obsolete informational reference (is this intentional?): RFC 2001 (Obsoleted by RFC 2581) -- Obsolete informational reference (is this intentional?): RFC 2581 (Obsoleted by RFC 5681) -- Obsolete informational reference (is this intentional?): RFC 3697 (Obsoleted by RFC 6437) -- Obsolete informational reference (is this intentional?): RFC 5575 (Obsoleted by RFC 8955) Summary: 0 errors (**), 0 flaws (~~), 2 warnings (==), 7 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 PANRG S. Dawkins, Ed. 3 Internet-Draft Wonder Hamster Internetworking 4 Intended status: Informational May 23, 2019 5 Expires: November 24, 2019 7 Path Aware Networking: Obstacles to Deployment (A Bestiary of Roads Not 8 Taken) 9 draft-irtf-panrg-what-not-to-do-03 11 Abstract 13 At the first meeting of the Path Aware Networking Research Group, the 14 research group agreed to catalog and analyze past efforts to develop 15 and deploy Path Aware technologies, most of which were unsuccessful, 16 in order to extract insights and lessons for path-aware networking 17 researchers. 19 This document contains that catalog and analysis. 21 Status of This Memo 23 This Internet-Draft is submitted in full conformance with the 24 provisions of BCP 78 and BCP 79. 26 Internet-Drafts are working documents of the Internet Engineering 27 Task Force (IETF). Note that other groups may also distribute 28 working documents as Internet-Drafts. The list of current Internet- 29 Drafts is at https://datatracker.ietf.org/drafts/current/. 31 Internet-Drafts are draft documents valid for a maximum of six months 32 and may be updated, replaced, or obsoleted by other documents at any 33 time. It is inappropriate to use Internet-Drafts as reference 34 material or to cite them other than as "work in progress." 36 This Internet-Draft will expire on November 24, 2019. 38 Copyright Notice 40 Copyright (c) 2019 IETF Trust and the persons identified as the 41 document authors. All rights reserved. 43 This document is subject to BCP 78 and the IETF Trust's Legal 44 Provisions Relating to IETF Documents 45 (https://trustee.ietf.org/license-info) in effect on the date of 46 publication of this document. Please review these documents 47 carefully, as they describe your rights and restrictions with respect 48 to this document. 50 Table of Contents 52 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 53 1.1. A Note About Path-Aware Technologies Included In This 54 Document . . . . . . . . . . . . . . . . . . . . . . . . 3 55 1.2. Venue for Discussion of this Document . . . . . . . . . . 4 56 1.3. A Note for the Research Group . . . . . . . . . . . . . . 4 57 1.4. A Note for the Editor . . . . . . . . . . . . . . . . . . 4 58 1.5. Architectural Guidance . . . . . . . . . . . . . . . . . 4 59 2. Summary of Lessons Learned . . . . . . . . . . . . . . . . . 5 60 3. Template for Contributions . . . . . . . . . . . . . . . . . 7 61 4. Contributions . . . . . . . . . . . . . . . . . . . . . . . . 7 62 4.1. Stream Transport (ST, ST2, ST2+) . . . . . . . . . . . . 7 63 4.1.1. Reasons for Non-deployment . . . . . . . . . . . . . 8 64 4.1.2. Lessons Learned. . . . . . . . . . . . . . . . . . . 8 65 4.2. Integrated Services (IntServ) . . . . . . . . . . . . . . 9 66 4.2.1. Reasons for Non-deployment . . . . . . . . . . . . . 9 67 4.2.2. Lessons Learned. . . . . . . . . . . . . . . . . . . 10 68 4.3. Quick-Start TCP . . . . . . . . . . . . . . . . . . . . . 10 69 4.3.1. Reasons for Non-deployment . . . . . . . . . . . . . 12 70 4.3.2. Lessons Learned . . . . . . . . . . . . . . . . . . . 12 71 4.4. ICMP Source Quench . . . . . . . . . . . . . . . . . . . 13 72 4.4.1. Reasons for Non-deployment . . . . . . . . . . . . . 13 73 4.4.2. Lessons Learned . . . . . . . . . . . . . . . . . . . 14 74 4.5. Triggers for Transport (TRIGTRAN) . . . . . . . . . . . . 14 75 4.5.1. Reasons for Non-deployment . . . . . . . . . . . . . 15 76 4.5.2. Lessons Learned. . . . . . . . . . . . . . . . . . . 16 77 4.6. Shim6 . . . . . . . . . . . . . . . . . . . . . . . . . . 17 78 4.6.1. Reasons for Non-deployment . . . . . . . . . . . . . 18 79 4.6.2. Lessons Learned . . . . . . . . . . . . . . . . . . . 18 80 4.6.3. Addendum on MultiPath TCP . . . . . . . . . . . . . . 18 81 4.7. Next Steps in Signaling (NSIS) . . . . . . . . . . . . . 19 82 4.7.1. Reasons for Non-deployment . . . . . . . . . . . . . 20 83 4.7.2. Lessons Learned . . . . . . . . . . . . . . . . . . . 21 84 4.8. IPv6 Flow Label . . . . . . . . . . . . . . . . . . . . . 21 85 4.8.1. Reasons for Non-deployment . . . . . . . . . . . . . 22 86 4.8.2. Lessons Learned . . . . . . . . . . . . . . . . . . . 23 87 5. Security Considerations . . . . . . . . . . . . . . . . . . . 24 88 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 24 89 7. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 24 90 8. Informative References . . . . . . . . . . . . . . . . . . . 25 91 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 31 93 1. Introduction 95 At the first meeting of the Path Aware Networking Research Group 96 [PANRG], at IETF 99 [PANRG-99], Oliver Bonaventure led a discussion 97 of "A Decade of Path Awareness" [PATH-Decade], on attempts, which 98 were mostly unsuccessful for a variety of reasons, to exploit Path 99 Aware technologies and achieve a variety of goals over the past 100 decade. At the end of this discussion, two things were abundantly 101 clear. 103 o The Internet community has accumulated considerable experience 104 with many Path Aware technologies over a long period of time, and 106 o Although some Path Aware technologies have been successfully 107 deployed (for example, Differentiated Services, or DiffServ 108 [RFC2475]), most of these technologies haven't seen widespread 109 adoption. The reasons for non-adoption are many, and are worthy 110 of study. 112 The meta-lessons from that experience were 114 o Path Aware Networking has been more Research than Engineering, so 115 establishing an IRTF Research Group for Path Aware Networking is 116 the right thing to do [RFC7418]. 118 o Analyzing a catalog of past experience to learn the reasons for 119 non-adoption would be a great first step for the Research Group. 121 Allison Mankin, as IRTF Chair, officially chartered the Path Aware 122 Networking Research Group in July, 2018. 124 This document contains the analysis performed by that research group 125 (see Section 2), based on that catalog (see Section 4). 127 1.1. A Note About Path-Aware Technologies Included In This Document 129 This document does not catalog every technology about Path Aware 130 Networking that was not implemented and deployed. Instead, we 131 include enough technologies to provide background for the lessons 132 included in Section 2 to guide researchers and protocol engineers in 133 their work. 135 No shame is intended for the technologies included in this document. 136 As shown in Section 2, the quality of specific technologies had 137 little to do with whether they were deployed or not. Based on the 138 technologies cataloged in this document, it is likely that when these 139 technologies were put forward, the proponents were trying to engineer 140 something that could not be engineered without first carrying out 141 research. Actual shame would be failing to learn from experience, 142 and failing to share that experience with other networking 143 researchers and engineers. 145 1.2. Venue for Discussion of this Document 147 (RFC Editor: please remove this section before publication) 149 Discussion of specific contributed experiences and this document in 150 general should take place on the PANRG mailing list. 152 1.3. A Note for the Research Group 154 (RFC Editor: please remove this section before publication) 156 The editor and research group chairs are aware that the current 157 version of this document is tilted toward transport-level Path Aware 158 technologies, and would like to interact with other IETF protocol 159 communities who have experience with Path Aware technologies. 161 It is worth looking at the Lessons Learned in Section 2 to see 162 whether the Internet has changed in ways that would make some lessons 163 less applicable for future protocol design. 165 1.4. A Note for the Editor 167 (Remove after taking these actions) 169 The to-do list for upcoming revisions includes 171 o Confirm that the Summary of Lessons Learned makes sense and is 172 complete, in consultation with the Research Group. 174 o If the Research Group identifies technologies that provided 175 lessons that aren't included in Section 2, solicit contributions 176 for those technologies. 178 o Provide better context for Section 2, to make sure that individual 179 lessons aren't considered in isolation, and to distinguish between 180 impediments to deployment and blockers for deployment. 182 1.5. Architectural Guidance 184 As background for understanding the Lessons Learned contained in this 185 document, the reader is encouraged to become familiar with the 186 Internet Architecture Board's documents on "What Makes for a 187 Successful Protocol?" [RFC5218] and "Planning for Protocol Adoption 188 and Subsequent Transitions" [RFC8170]. 190 Although these two documents do not specifically target path-aware 191 networking protocols, they are helpful resources for readers seeking 192 to improve their understanding of considerations for successful 193 adoption and deployment of any protocol. For example, the Basic 194 Success Factors described in Setion 2.1 of [RFC5218] are helpful for 195 readers of this document. 197 Because there is an economic aspect to decisions about deployment, 198 the IAB Workshop on Internet Technology Adoption and Transition 199 [ITAT] report [RFC7305] also provides food for thought. 201 Most of the Lessons Learned in Section 2 reflect considerations 202 described in [RFC5218], [RFC7305], and [RFC8170]. 204 2. Summary of Lessons Learned 206 This section summarizes the Lessons Learned from the contributed 207 sections in Section 4. 209 Each Lesson Learned is tagged with one or more contributions that 210 encountered this obstacle as a significant impediment to deployment. 211 Other contributed technologies may have also encountered this 212 obstacle, but this obstacle may not have been the biggest impediment 213 to deployment. 215 It is useful to notice that sometimes an obstacle might impede 216 deployment, while at other times, the same obstacle might prevent 217 deployment entirely. The research group discussed distinguishing 218 between obstacles that impede and obstacles that prevent, but it 219 appears that the boundary between "impede" and "prevent" can shift 220 over time - some of the Lessons Learned are based on both Path Aware 221 technologies that were not deployed, and Path Aware technologies that 222 were deployed, but were not deployed widely or quickly. See 223 Section 4.6 and Section 4.6.3 as one example of this shifting 224 boundary. 226 o The benefit of Path Awareness must be great enough to overcome 227 entropy for already-deployed devices. The colloquial American 228 English expression, "If it ain't broke, don't fix it" is a "best 229 current practice" on today's Internet. (See Section 4.3, 230 Section 4.5, and Section 4.4). 232 o Providing benefits for early adopters can be key - if everyone 233 must deploy a technology in order for the technology to provide 234 benefits, or even to work at all, the technology is unlikely to be 235 adopted. (See Section 4.2 and Section 4.3). 237 o Adaptive end-to-end protocol mechanisms may respond to feedback 238 quickly enough that the additional realizable benefit from a new 239 Path Aware mechanism may be much smaller than anticipated (see 240 Section 4.3 and Section 4.5). 242 o "Follow the money." If operators can't charge for a Path Aware 243 technology to recover the costs of deploying it, the benefits to 244 the operator must be really significant. Corollary: If operators 245 charge for a Path Aware technology, the benefits to the user must 246 be significant enough to justify the cost. (See Section 4.5, 247 Section 4.1, and Section 4.2). 249 o Impact of a Path Aware technology requiring changes to operational 250 practices can prevent deployment of promising technology. (See 251 Section 4.6, including Section 4.6.3). 253 o Per-connection state in intermediate devices can be an impediment 254 to adoption and deployment. This is especially true as we move 255 from the edge of the network into the routing core (See 256 Section 4.1 and Section 4.2). 258 o Many modern routers, especially high-end routers, have not been 259 designed to make heavy use of in-band mechanisms such as IPv4 and 260 IPv6 Router Alert Options (RAO), so operators can be reluctant to 261 deploy technologies that rely on these mechanisms. (See 262 Section 4.7). 264 o If the endpoints do not have any trust relationship with the 265 intermediate devices along a path, operators can be reluctant to 266 deploy technologies that rely on endpoints sending unauthenticated 267 control signals to routers. (See Section 4.2 and Section 4.7. We 268 also note this still remains a factor hindering deployment of 269 DiffServ). 271 o If intermediate devices along the path can't be trusted, it's 272 unlikely that endpoints will rely on signals from intermediate 273 devices to drive changes to endpoint behaviors. (See Section 4.5, 274 Section 4.4). The lowest level of trust is sufficient for a 275 device issuing a message to confirm that it has visibility of the 276 packets on the path it is seeking to control [RFC8085] (e.g., an 277 ICMP message included a quoted packet from the source). A higher 278 level of trust can arise when a network device could have a long 279 or short term trust relationship with the sender it controls. 281 o Because the Internet is a distributed system, if the distance that 282 information from distant hosts and routers travels to a Path Aware 283 host or router is sufficiently large, the information may no 284 longer represent the state and situation at the distant host or 285 router when it is received. In this case, the benefit that a Path 286 Aware technology provides likely decreases. (See Section 4.3). 288 o Providing a new feature/signal does not mean that it will be used. 289 Endpoint stacks may not know how to effectively utilize Path-Aware 290 transport protocol technologies, because the technology may 291 require information from applications to permit them to work 292 effectively, but applications may not a-priori know that 293 information. Even if the application does know that information, 294 the de-facto API has no way of signaling the expectations of 295 applications for the network path. Providing this awareness 296 requires an API that signals more than the packets to be sent. 297 TAPS is exploring such an API [TAPS-WG], yet even with such an 298 API, policy is needed to bind the application expectations to the 299 network characteristics. (See Section 4.1 and Section 4.2). 301 3. Template for Contributions 303 There are many things that could be said about the Path Aware 304 networking technologies that have been developed. For the purposes 305 of this document, contributors are requested to provide 307 o the name of a technology, including an abbreviation if one was 308 used 310 o if available, a long-term pointer to the best reference describing 311 the technology 313 o a short description of the problem the technology was intended to 314 solve 316 o a short description of the reasons why the technology wasn't 317 adopted 319 o a short statement of the lessons that researchers can learn from 320 our experience with this technology. 322 This document is being built collaboratively. To contribute your 323 experience, please send a Github pull request to 324 https://github.com/panrg/draft-dawkins-panrg-what-not-to-do. 326 4. Contributions 328 Additional contributions that provide Lessons Learned beyond those 329 already captured in Section 2 are welcomed. 331 4.1. Stream Transport (ST, ST2, ST2+) 333 The suggested references for IntServ are: 335 o ST - A Proposed Internet Stream Protocol [IEN-119] 337 o Experimental Internet Stream Protocol, Version 2 (ST-II) [RFC1190] 338 o Internet Stream Protocol Version 2 (ST2) Protocol Specification - 339 Version ST2+ [RFC1819] 341 The first version of Stream Transport, ST [IEN-119], was published in 342 the late 1970's and was implemented and deployed on the ARPANET at 343 small scale. It was used throughout the 1980's for experimental 344 transmission of voice, video, and distributed simulation. 346 The second version of the ST specification (ST2) [RFC1190] [RFC1819] 347 was an experimental connection-oriented internetworking protocol that 348 operated at the same layer as connectionless IP. ST2 packets could 349 be distinguished by their IP header protocol numbers (IP, at that 350 time, used protocol number 4, while ST2 used protocol number 5). 352 ST2 used a control plane layered over IP to select routes and reserve 353 capacity for real-time streams across a network path, based on a flow 354 specification communicated by a separate protocol. The flow 355 specification could be associated with QoS state in routers, 356 producing an experimental resource reservation protocol. This 357 allowed ST2 routers along a path to offer end-to-end guarantees, 358 primarily to satisfy the QoS requirements for realtime services over 359 the Internet. 361 4.1.1. Reasons for Non-deployment 363 Although implemented in a range of equipment, ST2 was not widely used 364 after completion of the experiments. It did not offer the 365 scalability and fate-sharing properties that have come to be desired 366 by the Internet community. 368 The ST2 protocol is no longer in use. 370 4.1.2. Lessons Learned. 372 As time passed, the trade-off between router processing and link 373 capacity changed. Links became faster and the cost of router 374 processing became comparatively more expensive. 376 The ST2 control protocol used "hard state" - once a route was 377 established, and resources were reserved, routes and resources 378 existing until they were explicitly released via signaling. A soft- 379 state approach was thought superior to this hard-state approach, and 380 led to development of the IntServ model described in Section 4.2. 382 4.2. Integrated Services (IntServ) 384 The suggested references for IntServ are: 386 o RFC 1633 Integrated Services in the Internet Architecture: an 387 Overview [RFC1633] 389 o RFC 2211 Specification of the Controlled-Load Network Element 390 Service [RFC2211] 392 o RFC 2212 Specification of Guaranteed Quality of Service [RFC2212] 394 o RFC 2215 General Characterization Parameters for Integrated 395 Service Network Elements [RFC2215] 397 o RFC 2205 Resource ReSerVation Protocol (RSVP) [RFC2205] 399 In 1994, when the IntServ architecture document [RFC1633] was 400 published, real-time traffic was first appearing on the Internet. At 401 that time, bandwidth was still a scarce commodity. Internet Service 402 Providers built networks over DS3 (45 Mbps) infrastructure, and sub- 403 rate (< 1 Mpbs) access was common. Therefore, the IETF anticipated a 404 need for a fine-grained QoS mechanism. 406 In the IntServ architecture, some applications can require service 407 guarantees. Therefore, those applications use the Resource 408 Reservation Protocol (RSVP) [RFC2205] to signal QoS reservations 409 across network paths. Every router in the network maintains per-flow 410 soft-state to a) perform call admission control and b) deliver 411 guaranteed service. 413 Applications use Flow Specification (Flow Specs) [RFC2210] to 414 describe the traffic that they emit. RSVP reserves capacity for 415 traffic on a per Flow Spec basis. 417 4.2.1. Reasons for Non-deployment 419 Although IntServ has been used in enterprise and government networks, 420 IntServ was never widely deployed on the Internet because of its 421 cost. The following factors contributed to operational cost: 423 o IntServ must be deployed on every router that is on a path where 424 IntServ is to be used 426 o IntServ maintained per flow state 428 As IntServ was being discussed, the following occurred: 430 o For many expected uses, it became more cost effective to solve the 431 QoS problem by adding bandwidth. Between 1994 and 2000, Internet 432 Service Providers upgraded their infrastructures from DS3 (45 433 Mbps) to OC-48 (2.4 Gbps). This meant that even if an endpoint 434 was using IntServ in an IntServ-enabled network, its requests 435 would never be denied, so endpoints and Internet Service Providers 436 had little reason to enable IntServ. 438 o DiffServ [RFC2475] offered a more cost-effective, albeit less 439 fine-grained, solution to the QoS problem. 441 4.2.2. Lessons Learned. 443 The following lessons were learned: 445 o Any mechanism that requires a router to maintain per-flow state is 446 not likely to succeed, unless the additional cost for offering the 447 feature can be recovered from the user. 449 o Any mechanism that requires an operator to upgrade all of its 450 routers is not likely to succeed, unless the additional cost for 451 offering the feature can be recovered from the user. 453 In environments where IntServ has been deployed, trust relationships 454 with endpoints are very different from trust relationships on the 455 Internet itself, and there are often clearly-defined hierarchies in 456 Service Level Agreements (SLAs), and well-defined transport flows 457 operating with pre-determined capacity and latency requirements over 458 paths where capacity or other attributes are constrained. 460 IntServ was never widely deployed to manage capacity across the 461 Internet. However, the technology that it produced was deployed for 462 reasons other than bandwidth management. RSVP is widely deployed as 463 an MPLS signaling mechanism. BGP reuses the RSVP concept of Filter 464 Specs to distribute firewall filters, although they are called Flow 465 Spec Component Types in BGP [RFC5575]. 467 4.3. Quick-Start TCP 469 The suggested references for Quick-Start TCP are: 471 o RFC 4782 Quick-Start for TCP and IP [RFC4782] 473 o Determining an appropriate initial sending rate over an 474 underutilized network path [SAF07] 476 o Fast Startup Internet Congestion Control for Broadband Interactive 477 Applications [Sch11] 479 o RFC 5634 Quick-Start for the Datagram Congestion Control Protocol 480 (DCCP) [RFC5634] 482 o Using Quick-Start to enhance TCP-friendly rate control performance 483 in bidirectional satellite networks [QS-SAT] 485 Quick-Start [RFC4782] is an Experimental TCP extension that leverages 486 support from the routers on the path to determine an allowed initial 487 sending rate for a path through the Internet, either at the start of 488 data transfers or after idle periods. A corresponding mechanism was 489 also specified for other congestion controllers (e.g., "Quick-Start 490 for the Datagram Congestion Control Protocol (DCCP)" [RFC5634]). In 491 these cases, a sender cannot easily determine an appropriate initial 492 sending rate, given the lack of information about the path. The 493 default TCP congestion control therefore uses the time-consuming 494 slow-start algorithm. With Quick-Start, connections are allowed to 495 use higher initial sending rates if there is significant unused 496 bandwidth along the path, and if the sender and all of the routers 497 along the path approve the request. 499 By examining the Time To Live (TTL) field in Quick-Start packets, a 500 sender can determine if routers on the path have approved the Quick- 501 Start request. However, this method is unable to take into account 502 the routers hidden by tunnels or other network devices invisible at 503 the IP layer. 505 The protocol also includes a nonce that provides protection against 506 cheating routers and receivers. If the Quick-Start request is 507 explicitly approved by all routers along the path, the TCP host can 508 send at up to the approved rate; otherwise TCP would use the default 509 congestion control. Quick-Start requires modifications in the 510 involved end-systems as well in routers. Due to the resulting 511 deployment challenges, Quick-Start was only proposed in [RFC4782] for 512 controlled environments. 514 The Quick-Start mechanism is a lightweight, coarse-grained, in-band, 515 network-assisted fast startup mechanism. The benefits are studied by 516 simulation in a research paper [SAF07] that complements the protocol 517 specification. The study confirms that Quick-Start can significantly 518 speed up mid-sized data transfers. That paper also presents router 519 algorithms that do not require keeping per-flow state. Later studies 520 [Sch11] comprehensively analyzes Quick-Start with a full Linux 521 implementation and with a router fast path prototype using a network 522 processor. In both cases, Quick-Start could be implemented with 523 limited additional complexity. 525 4.3.1. Reasons for Non-deployment 527 However, experiments with Quick-Start in [Sch11] revealed several 528 challenges: 530 o Having information from the routers along the path can reduce the 531 risk of congestion, but cannot avoid it entirely. Determining 532 whether there is unused capacity is not trivial in actual router 533 and host implementations. Data about available capacity visible 534 at the IP layer may be imprecise, and due to the propagation 535 delay, information can already be outdated when it reaches a 536 sender. There is a trade-off between the speedup of data 537 transfers and the risk of congestion even with Quick-Start. This 538 could be mitigated by only allowing Quick-Start to access a 539 proportion of the unused capacity along a path. 541 o For scalable router fast path implementation, it is important to 542 enable parallel processing of packets, as this is a widely used 543 method e.g. in network processors. One challenge is 544 synchronization of information between different packets, which 545 should be avoided as much as possible. 547 o Only some types of application traffic can benefit from Quick- 548 Start. Capacity needs to be requested and discovered. The 549 discovered capacity needs to be utilized by the flow, or it 550 implicitly becomes available for other flows. Failing to use the 551 requested capacity may have already reduced the pool of Quick- 552 Start capacity that was made available to other competing Quick- 553 Start requests. The benefit is greatest when senders use this 554 only for bulk flows and avoid sending unnecessary Quick-Start 555 requests, e.g. for flows that only send a small amount of data. 556 Choosing an appropriate request size requires application-internal 557 knowledge that is not commonly expressed by the transport API. 558 How a sender can determine the rate for an initial Quick-Start 559 request is still a largely unsolved problem. 561 There is no known deployment of Quick-Start for TCP or other IETF 562 transports. 564 4.3.2. Lessons Learned 566 Some lessons can be learned from Quick-Start. Despite being a very 567 light-weight protocol, Quick-Start suffers from poor incremental 568 deployment properties, both regarding the required modifications in 569 network infrastructure as well as its interactions with applications. 570 Except for corner cases, congestion control can be quite efficiently 571 performed end-to-end in the Internet, and in modern stacks there is 572 not much room for significant improvement by additional network 573 support. 575 After publication of the Quick-Start specification, there have been 576 large-scale experiments with an initial window of up to 10 MSS 577 [RFC6928]. This alternative "IW10" approach can also ramp-up data 578 transfers faster than the standard congestion control, but it only 579 requires sender-side modifications. As a result, this approach can 580 be easier and incrementally deployed in the Internet. While 581 theoretically Quick-Start can outperform "IW10", the improvement in 582 completion time for data transfer times can, in many cases, be small. 583 After publication of [RFC6928], most modern TCP stacks have increased 584 their default initial window. 586 4.4. ICMP Source Quench 588 The suggested references for ICMP Source Quench are: 590 o INTERNET CONTROL MESSAGE PROTOCOL [RFC0792] 592 The ICMP Source Quench message [RFC0792] allowed an on-path router to 593 request the source of a flow to reduce its sending rate. This method 594 allowed a router to provide an early indication of impending 595 congestion on a path to the sources that contribute to that 596 congestion. 598 4.4.1. Reasons for Non-deployment 600 This method was deployed in Internet routers over a period of time, 601 the reaction of endpoints to receiving this signal has varied. For 602 low speed links, with low multiplexing of flows the method could be 603 used to regulate (momentarily reduce) the transmission rate. 604 However, the simple signal does not scale with link speed, or the 605 number of flows sharing a link. 607 The approach was overtaken by the evolution of congestion control 608 methods in TCP [RFC2001], and later also by other IETF transports. 609 Because these methods were based upon measurement of the end-to-end 610 path and an algorithm in the endpoint, they were able to evolve and 611 mature more rapidly than methods relying on interactions between 612 operational routers and endpoint stacks. 614 After ICMP Source Quench was specified, the IETF began to recommend 615 that transports provide end-to-end congestion control [RFC2001]. The 616 Source Quench method has been obsoleted by the IETF [RFC6633], and 617 both hosts and routers must now silently discard this message. 619 4.4.2. Lessons Learned 621 This method had several problems: 623 First, [RFC0792] did not sufficiently specify how the sender would 624 react to the ICMP Source Quench signal from the path (e.g., 625 [RFC1016]). There was ambiguity in how the sender should utilize 626 this additional information. This could lead to unfairness in the 627 way that receivers (or routers) responded to this message. 629 Second, while the message did provide additional information, the 630 Explicit Congestion Notification (ECN) mechanism [RFC3168] provided a 631 more robust and informative signal for network devices to provide 632 early indication that a path has become congested. 634 The mechanism originated at a time when the Internet trust model was 635 very different. Most endpoint implementations did not attempt to 636 verify that the message originated from an on-path device before they 637 utilized the message. This made it vulnerable to denial of service 638 attacks. In theory, routers might have chosen to use the quoted 639 packet contained in the ICMP payload to validate that the message 640 originated from an on-path device, but this would have increased per- 641 packet processing overhead for each router along the path, would have 642 required transport functionality in the router to verify whether the 643 quoted packet header corresponded to a packet the router had sent. 644 In addition, section 5.2 of [RFC4443] noted ICMPv6-based attacks on 645 hosts that would also have threatened routers processing ICMPv6 646 Source Quench payloads. As time passed, it became increasingly 647 obvious that the lack of validation of the messages exposed receivers 648 to a security vulnerability where the messages could be forged to 649 create a tangible denial of service opportunity. 651 4.5. Triggers for Transport (TRIGTRAN) 653 The suggested references for TRIGTRAN are: 655 o TRIGTRAN BOF at IETF 55 [TRIGTRAN-55] 657 o TRIGTRAN BOF at IETF 56 [TRIGTRAN-56] 659 TCP [RFC0793] has a well-known weakness - the end-to-end flow control 660 mechanism has only a single signal, the loss of a segment, and TCP 661 implementations since the late 1980s have interpreted the loss of a 662 segment as evidence that the path between two endpoints may have 663 become congested enough to exhaust buffers on intermediate hops, so 664 that the TCP sender should "back off" - reduce its sending rate until 665 it knows that its segments are now being delivered without loss 666 [RFC2581]. More modern TCP stacks have added a growing array of 667 strategies about how to establish the sending rate [RFC5681], but 668 when a path is no longer operational, TCP would continue to retry 669 transmissions, which would fail, again, and double their 670 Retransmission Time Out (RTO) timers with each failed transmission, 671 with the result that TCP would wait many seconds before retrying a 672 segment, even if the path becomes operational while the sender is 673 waiting for its next retry. 675 The thinking behind TRIGTRAN was that if a path completely stopped 676 working because a link along the path was "down", somehow TCP could 677 be signaled when that link returned to service, and the sending TCP 678 could retry immediately, without waiting for a full retransmission 679 timeout (RTO) period. 681 4.5.1. Reasons for Non-deployment 683 The early dreams for TRIGTRAN were dashed because of an assumption 684 that TRIGTRAN triggers would be unauthenticated. This meant that any 685 "safe" TRIGTRAN mechanism would have relied on a mechanism such as 686 setting the IPv4 TTL or IPv6 Hop Count to 255 at a sender and testing 687 that it was 254 upon receipt, so that a receiver could verify that a 688 signal was generated by an adjacent sender known to be on the path 689 being used, and not some unknown sender which might not even be on 690 the path (e.g., "The Generalized TTL Security Mechanism (GTSM)" 691 [RFC5082]). This situation is very similar to the case for ICMP 692 Source Quench messages as described in Section 4.4, which were also 693 unauthenticated, and could be sent by an off-path attacker, resulting 694 in deprecation of ICMP Source Quench message processing [RFC6633]. 696 TRIGTRAN's scope shrunk from "the path is down" to "the first-hop 697 link is down". 699 But things got worse. 701 Because TRIGTRAN triggers would only be provided when the first-hop 702 link was "down", TRIGTRAN triggers couldn't replace normal TCP 703 retransmission behavior if the path failed because some link further 704 along the network path was "down". So TRIGTRAN triggers added 705 complexity to an already complex TCP state machine, and did not allow 706 any existing complexity to be removed. 708 There was also an issue that the TRIGTRAN signal was not sent in 709 response to a specific host that had been sending packets, and was 710 instead a signal that stimulated a response by any sender on the 711 link. This needs to scale when there are multiple flows trying to 712 use the same resource, yet the sender of a trigger has no 713 understanding how many of the potential traffic sources will respond 714 by sending packets - if recipients of the signal back-off their 715 responses to a trigger to improve scaling, then that immediately 716 mitigates the benefit of the signal. 718 Finally, intermediate forwarding devices required modification to 719 provide TRIGTRAN triggers, but operators couldn't charge for TRIGTRAN 720 triggers, so there was no way to recover the cost of modifying, 721 testing, and deploying updated intermediate devices. 723 Two TRIGTRAN BOFs were held, at IETF 55 [TRIGTRAN-55] and IETF 56 724 [TRIGTRAN-56], but this work was not chartered, and there was no 725 interest in deploying TRIGTRAN unless it was chartered and 726 standardized in the IETF. 728 4.5.2. Lessons Learned. 730 The reasons why this work was not chartered, much less deployed, 731 provide several useful lessons for researchers. 733 o TRIGTRAN started with a plausible value proposition, but 734 networking realities in the early 2000s forced reductions in scope 735 that led directly to reductions in potential benefits, but no 736 corresponding reductions in costs and complexity. 738 o These reductions in scope were the direct result of an inability 739 for hosts to trust or authenticate TRIGTRAN signals they received 740 from the network. 742 o Operators did not believe they could charge for TRIGTRAN 743 signaling, because first-hop links didn't fail frequently, and 744 TRIGTRAN provided no reduction in operating expenses, so there was 745 little incentive to purchase and deploy TRIGTRAN-capable network 746 equipment. 748 It is also worth noting that the targeted environment for TRIGTRAN in 749 the late 1990s contained links with a relatively small number of 750 directly-connected hosts - for instance, cellular or satellite links. 751 The transport community was well aware of the dangers of sender 752 synchronization based on multiple senders receiving the same stimulus 753 at the same time, but the working assumption for TRIGTRAN was that 754 there wouldn't be enough senders for this to be a meaningful problem. 755 In the 2010s, it is common for a single "link" to support many 756 senders and receivers on a single link, likely requiring TRIGTRAN 757 senders to wait some random amount of time before sending after 758 receiving a TRIGTRAN signal, which would have reduced the benefits of 759 TRIGTRAN even more. 761 4.6. Shim6 763 The suggested references for Shim6 are: 765 o RFC5533 Shim6: Level 3 Multihoming Shim Protocol for IPv6 766 [RFC5533] 768 The IPv6 routing architecture [RFC1887] assumed that most sites on 769 the Internet would be identified by Provider Assigned IPv6 prefixes, 770 so that Default-Free Zone routers only contained routes to other 771 providers, resulting in a very small routing table. 773 For a single-homed site, this could work well. A multihomed site 774 with only one upstream provider could also work well, although BGP 775 multihoming from a single upstream provider was often a premium 776 service (costing more than twice as much as two single-homed sites), 777 and if the single upstream provider went out of service, all of the 778 multihomed paths could fail simultaneously. 780 IPv4 sites often multihomed by obtaining Provider Independent 781 prefixes, and advertising these prefixes through multiple upstream 782 providers. With the assumption that any multihomed IPv4 site would 783 also multihome in IPv6, it seemed likely that IPv6 routing would be 784 subject to the same pressures to announce Provider Independent 785 prefixes, resulting in a global IPv6 routing table that exhibited the 786 same problems as the global IPv4 routing table. During the early 787 2000s, work began on a protocol that would provide the same benefits 788 for multihomed IPv6 sites without requiring sites to advertise 789 Provider Independent prefixes into the global routing table. 791 This protocol, called Shim6, allowed two endpoints to exchange 792 multiple addresses ("Locators") that all mapped to the same endpoint 793 ("Identity"). After an endpoint learned multiple Locators for the 794 other endpoint, it could send to any of those Locators with the 795 expectation that those packets would all be delivered to the endpoint 796 with the same Identity. Shim6 was an example of an "Identity/Locator 797 Split" protocol. 799 Shim6, as defined in [RFC5533] and related RFCs, provided a workable 800 solution for IPv6 multihoming using Provider Assigned prefixes, 801 including capability discovery and negotiation, and allowing end-to- 802 end application communication to continue even in the face of path 803 failure, because applications don't see Locator failures, and 804 continue to communicate with the same Identity using a different 805 Locator. 807 4.6.1. Reasons for Non-deployment 809 Note that the problem being addressed was "site multihoming", but 810 Shim6 was providing "host multihoming". That meant that the decision 811 about what path would be used was under host control, not under 812 router control. 814 Although more work could have been done to provide a better technical 815 solution, the biggest impediments to Shim6 deployment were 816 operational and business considerations. These impediments were 817 discussed at multiple network operator group meetings, including 818 [Shim6-35] at [NANOG-35]. 820 The technology issues centered around concerns that Shim6 relied on 821 the host to track all the connections, while also tracking Identity/ 822 Locator mappings in the kernel, and tracking failures to recognize 823 that a backup path has failed. 825 The operator issues centered around concerns that operators were 826 performing traffic engineering, but would have no visibility or 827 control over hosts when they chose to begin using another path, and 828 relying on hosts to engineer traffic exposed their networks to 829 oscillation based on feedback loops, as hosts move from path to path. 830 At a minimum, traffic engineering policies must be pushed down to 831 individual hosts. In addition, the usual concerns about firewalls 832 that expected to find a transport-level protocol header in the IP 833 payload, and won't be able to perform firewalling functions because 834 its processing logic would have to look past the Identity header. 836 The business issues centered removing or reducing the ability to sell 837 BGP multihoming service, which is often more expensive than single- 838 homed connectivity. 840 4.6.2. Lessons Learned 842 It is extremely important to take operational concerns into account 843 when a path-aware protocol is making decisions about path selection 844 that may conflict with existing operational practices and business 845 considerations. 847 4.6.3. Addendum on MultiPath TCP 849 During discussions in the PANRG session at IETF 103 [PANRG-103-Min], 850 Lars Eggert, past Transport Area Director, pointed out that during 851 charter discussions for the Multipath TCP working group [MP-TCP], 852 operators expressed concerns that customers could use Multipath TCP 853 to loadshare TCP connections across operators simultaneously and 854 compare passive performance measurements across network paths in real 855 time, changing the balance of power in those business relationships. 856 Although the Multipath TCP working group was chartered, this concern 857 could have acted as an obstacle to deployment. 859 Operator objections to Shim6 were focused on technical concerns, but 860 this concern could have also been an obstacle to Shim6 deployment if 861 the technical concerns had been overcome. 863 4.7. Next Steps in Signaling (NSIS) 865 The suggested references for NSIS are: 867 o the concluded working group charter [NSIS-CHARTER-2001] 869 o RFC 5971 GIST: General Internet Signalling Transport [RFC5971] 871 o RFC 5973 NAT/Firewall NSIS Signaling Layer Protocol (NSLP) 872 [RFC5973] 874 o RFC 5974 NSIS Signaling Layer Protocol (NSLP) for Quality-of- 875 Service Signaling [RFC5974] 877 o RFC 5981 "Authorization for NSIS Signaling Layer Protocols 878 [RFC5981] 880 The Next Steps in Signaling (NSIS) Working Group worked on signaling 881 technologies for network layer resources (e.g., QoS resource 882 reservations, Firewall and NAT traversal). 884 When RSVP [RFC2205] was used in deployments, a number of questions 885 came up about its perceived limitations and potential missing 886 features. The issues noted in the NSIS Working Group charter 887 [NSIS-CHARTER-2001] include interworking between domains with 888 different QoS architectures, mobility and roaming for IP interfaces, 889 and complexity. Later, the lack of security in RSVP was also 890 recognized ([RFC4094]). 892 The NSIS Working Group was chartered to tackle those issues and 893 initially focused on QoS signaling as its primary use case. However, 894 over time a new approach evolved that introduced a modular 895 architecture using application-specific signaling protocols (the NSIS 896 Signaling Layer Protocol (NSLP)) on top of a generic signaling 897 transport protocol (the NSIS Transport Layer Protocol (NTLP)). 899 The NTLP is defined in [RFC5971]. Two NSLPs are defined: the NSIS 900 Signaling Layer Protocol (NSLP) for Quality-of-Service Signaling 901 [RFC5974] as well as the NAT/Firewall NSIS Signaling Layer Protocol 902 (NSLP) [RFC5973]. 904 4.7.1. Reasons for Non-deployment 906 The obstacles for deployment can be grouped into implementation- 907 related aspects and operational aspects. 909 o Implementation-related aspects: 911 Although NSIS provides benefits with respect to flexibility, 912 mobility, and security compared to other network signaling 913 technologies, hardware vendors were reluctant to deploy this 914 solution, because it would require additional implementation effort 915 and would result in additional complexity for router implementations. 917 The NTLP mainly operates as path-coupled signaling protocol, i.e, its 918 messages are processed at the intermediate node's control plane that 919 are also forwarding the data flows. This requires a mechanism to 920 intercept signaling packets while they are forwarded in the same 921 manner (especially along the same path) as data packets. One reason 922 for the non-deployment of NSIS is the usage of the IPv4 and IPv6 923 Router Alert Option (RAO) to allow for an efficient interception of 924 those path-coupled signaling messages: This option requires router 925 implementations to correctly understand and implement the handling of 926 RAOs, e.g., to only process packet with RAOs of interest and to leave 927 packets with irrelevant RAOs in the fast forwarding processing path 928 (a comprehensive discussion of these issues can be found in 929 [RFC6398]). The latter was an issue with some router implementations 930 at the time of standardization. 932 Another reason is that path-coupled signaling protocols that interact 933 with routers and request manipulation of state at these routers (or 934 any other network element in general) are under scrutiny: a packet 935 (or sequence of packets) out of the mainly untrusted data path is 936 requesting creation and manipulation of network state. This is seen 937 as potentially dangerous (e.g., opens up a Denial of Service (DoS) 938 threat to a router's control plane) and difficult for an operator to 939 control. End-to-end signaling approaches were considered problematic 940 (see also section 3 of [RFC6398]). There are recommendations on how 941 to secure NSIS nodes and deployments (e.g., [RFC5981]). 943 o Operational Aspects: 945 End-to-end signaling technologies not only require trust between 946 customers and their provider, but also among different providers. 947 Especially, QoS signaling technologies would require some kind of 948 dynamic service level agreement support that would imply (potentially 949 quite complex) bilateral negotiations between different Internet 950 service providers. This complexity was not considered to be 951 justified and increasing the bandwidth (and thus avoiding 952 bottlenecks) was cheaper than actively managing network resource 953 bottlenecks by using path-coupled QoS signaling technologies. 954 Furthermore, an end-to-end path typically involves several provider 955 domains and these providers need to closely cooperate in cases of 956 failures. 958 4.7.2. Lessons Learned 960 One goal of NSIS was to decrease the complexity of the signaling 961 protocol, but a path-coupled signaling protocol comes with the 962 intrinsic complexity of IP-based networks, beyond the complexity of 963 the signaling protocol itself. Sources of intrinsic complexity 964 include: 966 o the presence of asymmetric routes between endpoints and routers 968 o the lack of security and trust at large in the Internet 969 infrastructure 971 o the presence of different trust boundaries 973 o the effects of best-effort networks (e.g., robustness to packet 974 loss) 976 o divergence from the fate sharing principle (e.g., state within the 977 network). 979 Any path-coupled signaling protocol has to deal with these realities. 981 Operators view the use of IPv4 and IPv6 Router Alert Option (RAO) to 982 signal routers along the path from end systems with suspicion, 983 because these end systems are usually not authenticated and heavy use 984 of RAOs can easily increase the CPU load on routers that are designed 985 to process most packets using a hardware "fast path". 987 4.8. IPv6 Flow Label 989 The suggested references for IPv6 Flow Label are: 991 o IPv6 Flow Label Specification [RFC6437] 993 IPv6 specifies a 20-bit field Flow Label field [RFC6437], included in 994 the fixed part of the IPv6 header and hence present in every IPv6 995 packet. An endpoint sets the value in this field to one of a set of 996 pseudo-randomly assigned values. If a packet is not part of any 997 flow, the flow label value is set to zero [RFC3697]. A number of 998 Standards Track and Best Current Practice RFCs (e.g., [RFC8085], 999 [RFC6437], [RFC6438]) encourage IPv6 endpoints to set a non-zero 1000 value in this field. A multiplexing transport could choose to use 1001 multiple flow labels to allow the network to independently forward 1002 its subflows, or to use one common value for the traffic aggregate. 1003 The flow label is present in all fragments. IPsec was originally put 1004 forward as one important use-case for this mechanism and does encrypt 1005 the field [RFC6438]. 1007 Once set, the flow label can provide information that can help inform 1008 network devices about subflows present at the transport layer, 1009 without needing to interpret the setting of upper layer protocol 1010 fields [RFC6294]. This information can also be used to coordinate 1011 how aggregates of transport subflows are grouped when queued in the 1012 network and to select appropriate per-flow forwarding when choosing 1013 between alternate paths [RFC6438] (e.g. for Equal Cost Multipath 1014 Routing (ECMP) and Link Aggregation (LAG)). 1016 4.8.1. Reasons for Non-deployment 1018 Despite the field being present in every IPv6 packet, the mechanism 1019 did not receive as much use as originally envisioned. One reason is 1020 that to be useful it requires engagement by two different 1021 stakeholders: 1023 o Endpoint Implementation: 1025 For network devices along a path to utilize the flow label there 1026 needs to be a non-zero value value inserted in the field [RFC6437] at 1027 the sending endpoint. There needs to be an incentive for an endpoint 1028 to set an appropriate non-zero value. The value should appropriately 1029 reflect the level of aggregation the traffic expects to be provided 1030 by the network. However, this requires the stack to know granularity 1031 at which flows should be identified (or conversely which flows should 1032 receive aggregated treatment), i.e., which packets carry the same 1033 flow label. Therefore, setting a non-zero value may result in 1034 additional choices that need to be made by an application developer. 1036 Although the standard [RFC3697] forbids any encoding of meaning into 1037 the flow label value, the opportunity to use the flow label as a 1038 covert channel or to signal other meta-information may have raised 1039 concerns about setting a non-zero value [RFC6437]. 1041 Before methods are widely deployed to use this method, there could be 1042 no incentive for an endpoint to set the field. 1044 o Operational support in network devices: 1046 A benefit can only be realized when a network device along the path 1047 also uses this information to inform its decisions. Network 1048 equipment (routers and/or middleboxes) need to include appropriate 1049 support so they can utilize the field when making decisions about how 1050 to classify flows, or to inform forwarding choices. Use of any 1051 optional feature in a network device also requires corresponding 1052 updates to operational procedures, and therefore is normally only 1053 introduced when the cost can be justified. 1055 A benefit from utilizing the flow label is expected to be increased 1056 quality of experience for applications - but this comes at some 1057 operational cost to an operator, and requires endpoints to set the 1058 field. 1060 4.8.2. Lessons Learned 1062 The flow label is a general purpose header field for use by the path. 1063 Multiple uses have been proposed. One candidate use was to reduce 1064 the complexity of forwarding decisions. However, modern routers can 1065 use a "fast path", often taking advantage of hardware to accelerate 1066 processing. The method can assist in more complex forwarding, such 1067 as ECMP and load balancing. 1069 Although [RFC6437] recommended that endpoints should by default 1070 choose uniformly-distributed labels for their traffic, the 1071 specification permitted an endpoint to choose to set a zero value. 1072 This ability of endpoints to choose to set a flow label of zero has 1073 had consequences on deployability: 1075 o Before wide-scale support by endpoints, it would be impossible to 1076 rely on a non-zero flow label being set. Network devices 1077 therefore would need to also employ other techniques to realize 1078 equivalent functions. An example of a method is one assuming 1079 semantics of the source port field to provide entropy input to a 1080 network-layer hash. This use of a 5-tuple to classify a packet 1081 represents a layering violation [RFC6294]. When other methods 1082 have been deployed, they increase the cost of deploying standards- 1083 based methods, even though they may offer less control to 1084 endpoints and result in potential interaction with other uses/ 1085 interpretation of the field. 1087 o Even though the flow label is specified as an end-to-end field, 1088 some network paths have been observed to not transparently forward 1089 the flow label. This could result from non-conformant equipment, 1090 or could indicate that some operational networks have chosen to 1091 re-use the protocol field for other (e.g. internal purposes). 1092 This results in lack of transparency, and a deployment hurdle to 1093 endpoints expecting that they can set a flow label that is 1094 utilized by the network. The more recent practice of "greasing" 1095 [GREASE] would suggest that a different outcome could have been 1096 achieved if endpoints were always required to set a non-zero 1097 value. 1099 o [RFC1809] noted that setting the choice of the flow label value 1100 can depend on the expectations of the traffic generated by an 1101 application, which suggests an API should be presented to control 1102 the setting or policy that is used. However, many currently 1103 available APIs do not have this support. 1105 A growth in the use of encrypted transports, (e.g. QUIC [QUIC-WG]) 1106 seems likely to raise similar issues to those discussed above and 1107 could motivate renewed interest in utilizing the flow label. 1109 5. Security Considerations 1111 This document describes Path Aware technologies that were not adopted 1112 and widely deployed on the Internet, so it doesn't affect the 1113 security of the Internet. 1115 If this document meets its goals, we may develop new technologies for 1116 Path Aware Networking that would affect the security of the Internet, 1117 but security considerations for those technologies will be described 1118 in the corresponding RFCs that specify them. 1120 6. IANA Considerations 1122 This document makes no requests of IANA. 1124 7. Acknowledgments 1126 Initial material for Section 4.1 on ST2 was provided by Gorry 1127 Fairhurst. 1129 Initial material for Section 4.2 on IntServ was provided by Ron 1130 Bonica. 1132 Initial material for Section 4.3 on Quick-Start TCP was provided by 1133 Michael Scharf. 1135 Initial material for Section 4.4 on ICMP Source Quench was provided 1136 by Gorry Fairhurst. 1138 Initial material for Section 4.5 on Triggers for Transport (TRIGTRAN) 1139 was provided by Spencer Dawkins. 1141 Section 4.6 on Shim6 builds on initial material describing obstacles 1142 provided by Erik Nordmark, with background added by Spencer Dawkins. 1144 Initial material for Section 4.7 on Next Steps In Signaling (NSIS) 1145 was provided by Roland Bless and Martin Stiemerling. 1147 Initial material for Section 4.8 on IPv6 Flow Labels was provided by 1148 Gorry Fairhurst. 1150 Our thanks to C.M. Heard, Gorry Fairhurst, Joe Touch, Joeri de 1151 Ruiter, Roland Bless, Ruediger Geib, and Wes Eddy, who provided 1152 review comments on previous versions. 1154 Special thanks to Adrian Farrel for helping Spencer navigate the 1155 twisty little passages of Flow Specs and Filter Specs in IntServ, 1156 RSVP, MPLS, and BGP. They are all alike, except for the differences 1157 [Colossal-Cave]. 1159 8. Informative References 1161 [Colossal-Cave] 1162 "Wikipedia Page for Colossal Cave Adventure", January 1163 2019, 1164 . 1166 [GREASE] Thomson, M., "Long-term Viability of Protocol Extension 1167 Mechanisms", January 2019, . 1170 [IEN-119] Forgie, J., "ST - A Proposed Internet Stream Protocol", 1171 September 1979, 1172 . 1174 [ITAT] "IAB Workshop on Internet Technology Adoption and 1175 Transition (ITAT)", December 2013, 1176 . 1178 [MP-TCP] "Multipath TCP Working Group Home Page", n.d., 1179 . 1181 [NANOG-35] 1182 "North American Network Operators Group NANOG-35 Agenda", 1183 October 2005, 1184 . 1186 [NSIS-CHARTER-2001] 1187 "Next Steps In Signaling Working Group Charter", March 1188 2011, 1189 . 1191 [PANRG] "Path Aware Networking Research Group (Home Page)", n.d., 1192 . 1194 [PANRG-103-Min] 1195 "Path Aware Networking Research Group - IETF-103 Minutes", 1196 November 2018, 1197 . 1199 [PANRG-99] 1200 "Path Aware Networking Research Group - IETF-99", July 1201 2017, 1202 . 1204 [PATH-Decade] 1205 Bonaventure, O., "A Decade of Path Awareness", July 2017, 1206 . 1209 [QS-SAT] Secchi, R., Sathiaseelan, A., Potorti, F., Gotta, A., and 1210 G. Fairhurst, "Using Quick-Start to enhance TCP-friendly 1211 rate control performance in bidirectional satellite 1212 networks", 2009, 1213 . 1215 [QUIC-WG] "QUIC Working Group Home Page", n.d., 1216 . 1218 [RFC0792] Postel, J., "Internet Control Message Protocol", STD 5, 1219 RFC 792, DOI 10.17487/RFC0792, September 1981, 1220 . 1222 [RFC0793] Postel, J., "Transmission Control Protocol", STD 7, 1223 RFC 793, DOI 10.17487/RFC0793, September 1981, 1224 . 1226 [RFC1016] Prue, W. and J. Postel, "Something a Host Could Do with 1227 Source Quench: The Source Quench Introduced Delay 1228 (SQuID)", RFC 1016, DOI 10.17487/RFC1016, July 1987, 1229 . 1231 [RFC1190] Topolcic, C., "Experimental Internet Stream Protocol: 1232 Version 2 (ST-II)", RFC 1190, DOI 10.17487/RFC1190, 1233 October 1990, . 1235 [RFC1633] Braden, R., Clark, D., and S. Shenker, "Integrated 1236 Services in the Internet Architecture: an Overview", 1237 RFC 1633, DOI 10.17487/RFC1633, June 1994, 1238 . 1240 [RFC1809] Partridge, C., "Using the Flow Label Field in IPv6", 1241 RFC 1809, DOI 10.17487/RFC1809, June 1995, 1242 . 1244 [RFC1819] Delgrossi, L., Ed. and L. Berger, Ed., "Internet Stream 1245 Protocol Version 2 (ST2) Protocol Specification - Version 1246 ST2+", RFC 1819, DOI 10.17487/RFC1819, August 1995, 1247 . 1249 [RFC1887] Rekhter, Y., Ed. and T. Li, Ed., "An Architecture for IPv6 1250 Unicast Address Allocation", RFC 1887, 1251 DOI 10.17487/RFC1887, December 1995, 1252 . 1254 [RFC2001] Stevens, W., "TCP Slow Start, Congestion Avoidance, Fast 1255 Retransmit, and Fast Recovery Algorithms", RFC 2001, 1256 DOI 10.17487/RFC2001, January 1997, 1257 . 1259 [RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S. 1260 Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1 1261 Functional Specification", RFC 2205, DOI 10.17487/RFC2205, 1262 September 1997, . 1264 [RFC2210] Wroclawski, J., "The Use of RSVP with IETF Integrated 1265 Services", RFC 2210, DOI 10.17487/RFC2210, September 1997, 1266 . 1268 [RFC2211] Wroclawski, J., "Specification of the Controlled-Load 1269 Network Element Service", RFC 2211, DOI 10.17487/RFC2211, 1270 September 1997, . 1272 [RFC2212] Shenker, S., Partridge, C., and R. Guerin, "Specification 1273 of Guaranteed Quality of Service", RFC 2212, 1274 DOI 10.17487/RFC2212, September 1997, 1275 . 1277 [RFC2215] Shenker, S. and J. Wroclawski, "General Characterization 1278 Parameters for Integrated Service Network Elements", 1279 RFC 2215, DOI 10.17487/RFC2215, September 1997, 1280 . 1282 [RFC2475] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z., 1283 and W. Weiss, "An Architecture for Differentiated 1284 Services", RFC 2475, DOI 10.17487/RFC2475, December 1998, 1285 . 1287 [RFC2581] Allman, M., Paxson, V., and W. Stevens, "TCP Congestion 1288 Control", RFC 2581, DOI 10.17487/RFC2581, April 1999, 1289 . 1291 [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition 1292 of Explicit Congestion Notification (ECN) to IP", 1293 RFC 3168, DOI 10.17487/RFC3168, September 2001, 1294 . 1296 [RFC3697] Rajahalme, J., Conta, A., Carpenter, B., and S. Deering, 1297 "IPv6 Flow Label Specification", RFC 3697, 1298 DOI 10.17487/RFC3697, March 2004, 1299 . 1301 [RFC4094] Manner, J. and X. Fu, "Analysis of Existing Quality-of- 1302 Service Signaling Protocols", RFC 4094, 1303 DOI 10.17487/RFC4094, May 2005, 1304 . 1306 [RFC4443] Conta, A., Deering, S., and M. Gupta, Ed., "Internet 1307 Control Message Protocol (ICMPv6) for the Internet 1308 Protocol Version 6 (IPv6) Specification", STD 89, 1309 RFC 4443, DOI 10.17487/RFC4443, March 2006, 1310 . 1312 [RFC4782] Floyd, S., Allman, M., Jain, A., and P. Sarolahti, "Quick- 1313 Start for TCP and IP", RFC 4782, DOI 10.17487/RFC4782, 1314 January 2007, . 1316 [RFC5082] Gill, V., Heasley, J., Meyer, D., Savola, P., Ed., and C. 1317 Pignataro, "The Generalized TTL Security Mechanism 1318 (GTSM)", RFC 5082, DOI 10.17487/RFC5082, October 2007, 1319 . 1321 [RFC5218] Thaler, D. and B. Aboba, "What Makes for a Successful 1322 Protocol?", RFC 5218, DOI 10.17487/RFC5218, July 2008, 1323 . 1325 [RFC5533] Nordmark, E. and M. Bagnulo, "Shim6: Level 3 Multihoming 1326 Shim Protocol for IPv6", RFC 5533, DOI 10.17487/RFC5533, 1327 June 2009, . 1329 [RFC5575] Marques, P., Sheth, N., Raszuk, R., Greene, B., Mauch, J., 1330 and D. McPherson, "Dissemination of Flow Specification 1331 Rules", RFC 5575, DOI 10.17487/RFC5575, August 2009, 1332 . 1334 [RFC5634] Fairhurst, G. and A. Sathiaseelan, "Quick-Start for the 1335 Datagram Congestion Control Protocol (DCCP)", RFC 5634, 1336 DOI 10.17487/RFC5634, August 2009, 1337 . 1339 [RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion 1340 Control", RFC 5681, DOI 10.17487/RFC5681, September 2009, 1341 . 1343 [RFC5971] Schulzrinne, H. and R. Hancock, "GIST: General Internet 1344 Signalling Transport", RFC 5971, DOI 10.17487/RFC5971, 1345 October 2010, . 1347 [RFC5973] Stiemerling, M., Tschofenig, H., Aoun, C., and E. Davies, 1348 "NAT/Firewall NSIS Signaling Layer Protocol (NSLP)", 1349 RFC 5973, DOI 10.17487/RFC5973, October 2010, 1350 . 1352 [RFC5974] Manner, J., Karagiannis, G., and A. McDonald, "NSIS 1353 Signaling Layer Protocol (NSLP) for Quality-of-Service 1354 Signaling", RFC 5974, DOI 10.17487/RFC5974, October 2010, 1355 . 1357 [RFC5981] Manner, J., Stiemerling, M., Tschofenig, H., and R. Bless, 1358 Ed., "Authorization for NSIS Signaling Layer Protocols", 1359 RFC 5981, DOI 10.17487/RFC5981, February 2011, 1360 . 1362 [RFC6294] Hu, Q. and B. Carpenter, "Survey of Proposed Use Cases for 1363 the IPv6 Flow Label", RFC 6294, DOI 10.17487/RFC6294, June 1364 2011, . 1366 [RFC6398] Le Faucheur, F., Ed., "IP Router Alert Considerations and 1367 Usage", BCP 168, RFC 6398, DOI 10.17487/RFC6398, October 1368 2011, . 1370 [RFC6437] Amante, S., Carpenter, B., Jiang, S., and J. Rajahalme, 1371 "IPv6 Flow Label Specification", RFC 6437, 1372 DOI 10.17487/RFC6437, November 2011, 1373 . 1375 [RFC6438] Carpenter, B. and S. Amante, "Using the IPv6 Flow Label 1376 for Equal Cost Multipath Routing and Link Aggregation in 1377 Tunnels", RFC 6438, DOI 10.17487/RFC6438, November 2011, 1378 . 1380 [RFC6633] Gont, F., "Deprecation of ICMP Source Quench Messages", 1381 RFC 6633, DOI 10.17487/RFC6633, May 2012, 1382 . 1384 [RFC6928] Chu, J., Dukkipati, N., Cheng, Y., and M. Mathis, 1385 "Increasing TCP's Initial Window", RFC 6928, 1386 DOI 10.17487/RFC6928, April 2013, 1387 . 1389 [RFC7305] Lear, E., Ed., "Report from the IAB Workshop on Internet 1390 Technology Adoption and Transition (ITAT)", RFC 7305, 1391 DOI 10.17487/RFC7305, July 2014, 1392 . 1394 [RFC7418] Dawkins, S., Ed., "An IRTF Primer for IETF Participants", 1395 RFC 7418, DOI 10.17487/RFC7418, December 2014, 1396 . 1398 [RFC8085] Eggert, L., Fairhurst, G., and G. Shepherd, "UDP Usage 1399 Guidelines", BCP 145, RFC 8085, DOI 10.17487/RFC8085, 1400 March 2017, . 1402 [RFC8170] Thaler, D., Ed., "Planning for Protocol Adoption and 1403 Subsequent Transitions", RFC 8170, DOI 10.17487/RFC8170, 1404 May 2017, . 1406 [SAF07] Sarolahti, P., Allman, M., and S. Floyd, "Determining an 1407 appropriate sending rate over an underutilized network 1408 path", Computer Networking Volume 51, Number 7, May 2007. 1410 [Sch11] Scharf, M., "Fast Startup Internet Congestion Control for 1411 Broadband Interactive Applications", Ph.D. Thesis, 1412 University of Stuttgart, April 2011. 1414 [Shim6-35] 1415 Meyer, D., Huston, G., Schiller, J., and V. Gill, "IAB 1416 IPv6 Multihoming Panel at NANOG 35", NANOG North American 1417 Network Operator Group, October 2005, 1418 . 1420 [TAPS-WG] "Transport Services Working Group Home Page", n.d., 1421 . 1423 [TRIGTRAN-55] 1424 "Triggers for Transport BOF at IETF 55", July 2003, 1425 . 1427 [TRIGTRAN-56] 1428 "Triggers for Transport BOF at IETF 56", November 2003, 1429 . 1431 Author's Address 1433 Spencer Dawkins (editor) 1434 Wonder Hamster Internetworking 1436 Email: spencerdawkins.ietf@gmail.com