idnits 2.17.1 draft-bryant-arch-fwd-layer-ps-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 13, 2020) is 1381 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'I-D.bryant-arch-fwd-layer-uc' is mentioned on line 593, but not defined == Outdated reference: A later version (-31) exists of draft-bonica-6man-comp-rtg-hdr-22 == Outdated reference: A later version (-04) exists of draft-bonica-6man-crh-helper-opt-01 == Outdated reference: A later version (-04) exists of draft-bonica-spring-sr-mapped-six-01 == Outdated reference: A later version (-02) exists of draft-cheng-spring-shorter-srv6-sid-requirement-01 == Outdated reference: A later version (-07) exists of draft-decraene-spring-srv6-vlsid-03 == Outdated reference: A later version (-16) exists of draft-filsfils-spring-net-pgm-extension-srv6-usid-07 == Outdated reference: A later version (-03) exists of draft-filsfilscheng-spring-srv6-srh-comp-sl-enc-01 == Outdated reference: A later version (-03) exists of draft-herbert-6man-eh-attrib-01 == Outdated reference: A later version (-13) exists of draft-ietf-detnet-mpls-09 == Outdated reference: A later version (-16) exists of draft-ietf-detnet-security-10 == Outdated reference: A later version (-17) exists of draft-ietf-ippm-ioam-data-09 == Outdated reference: A later version (-28) exists of draft-ietf-spring-srv6-network-programming-16 == Outdated reference: A later version (-03) exists of draft-lc-6man-generalized-srh-00 -- Obsolete informational reference (is this intentional?): RFC 2460 (Obsoleted by RFC 8200) Summary: 0 errors (**), 0 flaws (~~), 15 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 INTAREA Working Group S. Bryant 3 Internet-Draft U. Chunduri 4 Intended status: Informational T. Eckert 5 Expires: January 14, 2021 A. Clemm 6 Futurewei Technologies Inc. 7 July 13, 2020 9 Forwarding Layer Problem Statement 10 draft-bryant-arch-fwd-layer-ps-01 12 Abstract 14 This document considers the problems that need to addressed in IP in 15 order to address the use cases and new network services described in 16 draft-bryant-arch-fwd-layer-uc-00. 18 Status of This Memo 20 This Internet-Draft is submitted in full conformance with the 21 provisions of BCP 78 and BCP 79. 23 Internet-Drafts are working documents of the Internet Engineering 24 Task Force (IETF). Note that other groups may also distribute 25 working documents as Internet-Drafts. The list of current Internet- 26 Drafts is at https://datatracker.ietf.org/drafts/current/. 28 Internet-Drafts are draft documents valid for a maximum of six months 29 and may be updated, replaced, or obsoleted by other documents at any 30 time. It is inappropriate to use Internet-Drafts as reference 31 material or to cite them other than as "work in progress." 33 This Internet-Draft will expire on January 14, 2021. 35 Copyright Notice 37 Copyright (c) 2020 IETF Trust and the persons identified as the 38 document authors. All rights reserved. 40 This document is subject to BCP 78 and the IETF Trust's Legal 41 Provisions Relating to IETF Documents 42 (https://trustee.ietf.org/license-info) in effect on the date of 43 publication of this document. Please review these documents 44 carefully, as they describe your rights and restrictions with respect 45 to this document. Code Components extracted from this document must 46 include Simplified BSD License text as described in Section 4.e of 47 the Trust Legal Provisions and are provided without warranty as 48 described in the Simplified BSD License. 50 Table of Contents 52 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 53 2. Forwarding Layer . . . . . . . . . . . . . . . . . . . . . . 4 54 3. Underlying New Requirements . . . . . . . . . . . . . . . . . 4 55 3.1. Better than Best Effort . . . . . . . . . . . . . . . . . 4 56 3.2. Efficient Packet Design . . . . . . . . . . . . . . . . . 5 57 3.3. Forwarding Identifiers . . . . . . . . . . . . . . . . . 6 58 3.4. Operational visibility . . . . . . . . . . . . . . . . . 6 59 3.5. Holistic Solution . . . . . . . . . . . . . . . . . . . . 7 60 4. Existing Protocol, Layering Challenges and Gaps . . . . . . . 7 61 4.1. Challenges with IPv6 . . . . . . . . . . . . . . . . . . 8 62 4.1.1. The End-to-End Model . . . . . . . . . . . . . . . . 8 63 4.1.2. Fixed Address Length . . . . . . . . . . . . . . . . 11 64 4.2. Better Than Best Effort E2E Network Services . . . . . . 13 65 4.3. Adaptive Bit-rate Video streaming . . . . . . . . . . . . 14 66 4.4. Limited Domain Opportunities . . . . . . . . . . . . . . 15 67 4.5. DetNet and Higher Precision Networking Service . . . . . 15 68 4.6. Forwarding Plane vs. Control Plane . . . . . . . . . . . 16 69 4.7. User-Network/Network-User Interface Signaling . . . . . . 17 70 5. Candidate Solution Directions . . . . . . . . . . . . . . . . 18 71 5.1. Variable Length Addresses . . . . . . . . . . . . . . . . 18 72 5.2. Address Semantics . . . . . . . . . . . . . . . . . . . . 19 73 5.3. Multiple Instructions . . . . . . . . . . . . . . . . . . 19 74 5.4. Node and Path Specific Processing Instructions . . . . . 19 75 5.5. Integrated Assurance and Verification . . . . . . . . . . 20 76 5.6. For Consideration in a Future Version . . . . . . . . . . 20 77 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20 78 7. Security Considerations . . . . . . . . . . . . . . . . . . . 21 79 8. References . . . . . . . . . . . . . . . . . . . . . . . . . 21 80 8.1. Normative References . . . . . . . . . . . . . . . . . . 21 81 8.2. Informative References . . . . . . . . . . . . . . . . . 21 82 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 26 84 1. Introduction 86 There is an emerging set of new requirements that exceed the network 87 and transport services of the current Internet, which only delivers 88 "best effort" service. While many controlled or private networks 89 include further services, such as other DiffServ QoS in addition to 90 best effort and traffic engineering with bandwidth guarantees, the 91 solutions used today only support walled gardens and are thus not 92 available to application service providers and consumers across the 93 Internet. 95 The uses cases and service needs that are foreseen as necessary for 96 deployment in the medium future are described in [I-D.bryant-arch- 97 fwd-layer-uc]. 99 The purpose of this document is to examine the shortcomings that the 100 existing network and transport layer protocols a well as their 101 associated control plane need to overcome to meet these needs. 103 The IETF is the body responsible for the long term evolution of the 104 IP protocol suit, but is missing a work track to discuss the long- 105 term Internet network architecture evolution. In particular it lacks 106 a programme for the long term evolution of IP itself. 108 Approximately 30 years ago, the IETF started a process to 109 revolutionize the IPv4 [RFC0791] Internet Protocol. In this process, 110 researchers, industry, and service providers got together, and 111 brought up a number of new proposals, and worked toward a successor 112 to IPv4, which became IPv6 [RFC2460] and later [RFC8200]. 114 30 years later, there is heavy resistance to anything more than minor 115 incremental evolutions to IPv6. There are a number of reasons for 116 this ranging from opinions that all future IP needs can be met 117 through minor incremental evolutions to fears that major proposals 118 for innovation at the IP would be an unwelcome disrupter to the 119 current business of the vendors or the service providers. 121 The authors take no position on the scale of the problem or the 122 difficulties of deploying any solutions at scale in the Internet. 123 What we seek to do is to establish the scope and nature of the 124 problem. A decision on which aspects of the problem are economically 125 tractable is out of scope of this text, but technologies to support 126 monetization are not. 128 As a problem statement, this documents goal is to not propose or 129 promote specific solutions to the problems raised. Instead it uses 130 references to not Internet adopted, but proposed or existing 131 solutions only as example evidence that the described problem can 132 actually be solved. 134 Because the document does not propose specific solutions, it also 135 does not attempt to structure the problem description in a way that 136 would identify sub-set of problems to be resolved by specific 137 solution components. 139 The purpose of this text is thus to stimulate discussion on the 140 emerging needs of the forwarding layer and to start the process of 141 determining how they are best satisfied within the IETF protocol 142 suite. 144 2. Forwarding Layer 146 The term "forwarding layer" is used in this work because none of the 147 standard terms encompass the parts of the network stack that need 148 attention to address the needs of the applications that are foreseen. 150 It is possible that development work will need to reach down to layer 151 2.5 in order to ensure that packets are handled correctly down to the 152 physical layer. The MAC layer is quite sophisticated and includes 153 its own switching function so we need to be sure that the good work 154 done in the network layer is not undone lower down the stack. 155 Equally it is possible that development work will need to reach into 156 the transport layer to address new approaches to congestion, and to 157 ensure that the network layer understands the requirements placed on 158 it by the application. An open mind is needed on the boundaries of 159 the layers as they exist today when analyzing the consequential 160 network changes needed to support the evolving application space. 162 In the network layer itself, this document is only concerned with the 163 forwarding component, not path selection or the other components of 164 routing. 166 Thus, we use the term forwarding layer to describe the scope of the 167 stack that this document addresses. 169 3. Underlying New Requirements 171 3.1. Better than Best Effort 173 The current Internet is essentially of best-effort system, but future 174 applications require high-precision KPIs on throughput, latency and 175 packet loss for industrial manufacturing, control, automation, and 176 machine-to-machine communications. 178 The emerging use cases for networks require deployment of 179 capabilities that are beyond best effort. Best effort networks can 180 do remarkably well by simply throwing bandwidth at the problem and 181 lightly loading the network. For the case where a greater capability 182 is needed the IETF has invested effort in deterministic networking 183 (DN) [RFC8655]. Whilst DN is an improvement over best effort it is 184 still fundamentally a best effort service with enhancement to 185 improved the probability of a packet not being delayed or lost due to 186 congestion. It is an after the fact enhancement to the method of 187 operation of what is a largely unmodified data plane. I the case of 188 MPLS [I-D.ietf-detnet-mpls] there is some assistance from the PREOF 189 function, but IP runs the standard data plane and relies entirely on 190 special case packet selection queue management. It is thus an after- 191 the-fact enhancement to a minimally changed data plane restricted to 192 a single network domain. 194 With upcoming Cellular technologies (5G/B5G) there is a need for 195 Service Providers to expand the type of customers for metropolitan 196 size networks to address their better than best-effort traffic needs. 198 DetNet has been proposed to support this, however: 200 o Only some aspects of DetNet currently only run on top of current 201 IP/IPv6. 203 o DetNet service is constrained: It only supports constant bit rate 204 (CBR), reserved bandwidth. It does not support flexible 205 bandwidth. The notion of contracts in a future development of the 206 forwarding layer will support more flexible managed bandwidth and 207 managed latency contracts for traffic. 209 3.2. Efficient Packet Design 211 The ratio of useful data in the payload to overhead has a direct 212 financial impact on communication links; these links are of finite 213 capacity and hence have a finite cost-per-unit-data that can be 214 calculated. The capacity used to transport information as compared 215 to the overhead which is unavailable for use by a customer, but 216 required to transmit is often expresses as a good-put efficiency and 217 can be related to cost to transmit payload data. 219 o There is a need to support large number of low power user 220 equipment (UE) devices (low-power IoTs) connecting through various 221 radio networks (LTE/5G/B5G) where spectral efficiency is needed. 222 This needs to be achieved without header compression techniques 223 like as [RFC6282] since, compression can result in additional 224 processing and energy consumption overhead. 226 o The handling network protocol headers, requires that portions of 227 each packet be held in memory or buffer structures; the more 228 levels of information which need to be held for processing by 229 network nodes, the more memory space will be required, and this 230 directly effects the cost of operation and cost of manufacture/ 231 provision of such equipment. 233 On the other hand, in various non-constrained environments where 234 various network layer functionalities are desired, there are 235 different set of requirements. For example: 237 o Segment Routing over IPv6 (SRv6) parameter encoding 238 [I-D.ietf-spring-srv6-network-programming] in the SRv6 SID 239 [RFC8754] is limited by the prefix portion of the IPv6 address. 241 o In Identifier Locator Addressing (ILA), the identifier (ID) 242 portion of the address length is limited because of 128 bits 243 limit. 245 3.3. Forwarding Identifiers 247 Developments in IPv6 [I-D.ietf-spring-srv6-network-programming] 248 formalize a trend that has been happening for a long time: the 249 morphing of network layer addresses into forwarding identifiers (FI). 250 However, constraining FIs to a fixed size ill serves the development 251 of the forwarding layer. There are clear cases as illustrated above 252 where it would be useful to have shorter network layer addresses. 253 Equally we can see that there will be future cases where 128 bits may 254 be insufficient to specify a forwarding operation. The requirement 255 is thus to formally introduce the concept of forwarding identifiers 256 in place of network layer addresses, and use a forwarding identifier 257 construct that supports multiple semantics and multiple, possibly 258 fully variable, lengths. 260 There is further discussion on this point in Section 4.1.2. 262 3.4. Operational visibility 264 Network operators require facilities that let them better understand 265 and fine tune detailed network behavior. These features are hard to 266 retrofit with current IP/IPv6. 268 The rise of machine learning has led to the expectation of being able 269 to better optimize networks This in turn leads to the increase of 270 network telemetry as a source of data to base these systems on. In- 271 Situ OAM (IOAM) [I-D.ietf-ippm-ioam-data] represents one of the 272 latest developments in that space, allowing the data plane to piggy- 273 back telemetry data onto individual packets in order to diagnose and 274 fine-tune service levels such as latency or jitter. However, there 275 are several issues with this approach: 277 o MTU issues limit amount of data that can be obtained. With IOAM 278 packet size increases with number of data items and number of 279 hops. 281 o The data that can be obtained is very limited. 283 o The OAM data volume can easily exceeds that of production traffic 284 which is wasteful 286 o There is no ability to aggregate OAM data, or make context 287 dependent OAM collection. 289 o Integration with other solutions such as DetNet is unclear. 291 While useful, IOAM exposes the limits of what add-on solutions can 292 provide. Solutions that provide visibility at the level of flows or 293 that provide automatic verification of Service Level Objectives are 294 missing entirely. 296 3.5. Holistic Solution 298 It needs to be recognized that it will not be sufficient for 299 solutions to support new services and capabilities one at a time and 300 independently from one another. For example, better-than-best- 301 effort, operational visibility, and efficient packet design should go 302 together, without leading to additional integration problems ore 303 requiring users to make a choice. 305 A piecemeal approach, in which solutions for any one particular 306 problem are developed and emerge one at a time, results in a 307 fragmented solution which gets progressively more difficult to 308 integrate with components previously designed. Thus it is better if 309 solutions are holistic and be able to support new services and 310 capabilities in integrated fashion and simultaneously with each 311 other. 313 We therefore need to identify an elegant approach that is simple and 314 naturally extensible to address problems that we do not yet conceive 315 as requiring addressing. 317 Any such solution needs to be intrinsically secure and yet be able to 318 support security without privacy and privacy without security. 320 4. Existing Protocol, Layering Challenges and Gaps 322 Despite IPv4 still having a large user base, and having a number of 323 useful properties the IETF has abandoned future development of IPv4 324 as a way to force the deployment of IPv6. For example, in terms of 325 traffic steering the segment routing could have usefully been applied 326 to IPv4 to support network operators that wished to retain IPv4 as 327 their preferred internal protocol. 329 Given the gaps in each of the existing network layer protocols the 330 IETF may wish to look at the design of a protocol that both fills the 331 gaps and unifies its three existing network layer protocols: IPv4, 332 IPv6 and MPLS. 334 Additionally there is a clear need for a more sophisticated approach 335 to indicating the required quality of service that a packet, or flow, 336 needs in an IP network. 338 4.1. Challenges with IPv6 340 4.1.1. The End-to-End Model 342 IPv6 and specifically [RFC8200] was designed to fit within an 343 Internet architecture centered around the end-to-end model with 344 "Internet Paths" potentially passing through one or more networks 345 without any relationship to the endpoints of a communication such as 346 most so-called transit-AS. As history already from IPv4 had shown, 347 anything more than the most simple per-hop processing options can 348 cause interoperability issues. In result, [RFC8200] has drastically 349 limited such per-hop processing options. 351 Two core restrictions of RFC8200 are the following: 353 o Restrictions on extension headers (EH): EHs must never be deleted 354 or changed in size by any node on the path the packet takes. 355 Intermediate nodes are only expected to examine these headers (if 356 they are configured to do so). Implementations cannot expect 357 intermediate nodes to examine, or act on, except for hop-by-hop 358 header (section 4.8 of [RFC8200]). 360 At the time of writing this is an area of considerable active 361 discussion in the IETF 6MAN and SPRING working groups. The issues 362 that arise from allowing unrestricted insertion, deletion or 363 modification of EHs are for example: 365 o Breakage of path MTU discovery 367 o Impact on the Authentication Header protocol 369 o Inability to return ICMP error messages to the correct node. 371 See Section 4.1.1.1 for further discussion. 373 o No new hop-by-hop headers (HBH) in IPV6: No new EHs that require 374 hop-by-hop behavior should be defined (section 4 of [RFC8200]) - 375 the only EH that has hop-by-hop behavior is the Hop-by-Hop Options 376 header. The only alternative available to the designer is instead 377 to use destination headers (section 6.8 of [RFC8200]). 379 4.1.1.1. IPv6 For Controlled Networks 381 While [RFC8200] is a conservative set of requirements to enable 382 proliferation of the target use case of "Internet Paths", the same 383 set of requirements limit the flexibility of IPv6 unnecessarily when 384 it is used in controlled networks where the constraints and 385 interoperability issues for "Internet Paths" do not equally apply, 386 for example the deployment scenarios described in Sections "Embedded 387 Service" and "Embedded Global Service" of [I-D.bryant-arch-fwd-layer- 388 uc]. 390 One typical type of controlled networks are service providers (SP) 391 where SRv6 is used as the architecture within the SP network. 393 o IPv6 extension headers can not be added on a midpoint. Any 394 addition/change requires an encapsulation where another IPv6 395 header with optional SRH extension header is prepended to the 396 carried IPv6 packet. This is expensive in terms of packet MTU, 397 and in terms of packet buffer requirements at the ends of the 398 provider path which can be an economic issue in cost sensitive 399 network segments. 401 o The requirement to encapsulate instead of being allowed to add an 402 EH along the path stems from the desire to isolate any header 403 changes from Path MTU Discovery (PMTUD). This is a necessary 404 complexity when traversing uncontrolled hops across the Internet, 405 but it is unnecessary overhead when only passing through 406 controlled hops. In MPLS and SR-MPLS, the MPLS header size is not 407 included in the MTU available to the MPLS payload, instead the 408 network is managed such that the maximum MPLS header size plus the 409 available payload MTU is always smaller that the encapsulating L2 410 frame MTU. In IPv6 instead, the encapsulating and decapsulating 411 would logically have to perform signaling for PMTUD 412 (unnecessarily). 414 o Because of the authorization header (AH) [RFC4302] and OAM 415 concerns, [RFC8200] likewise prohibits removing extension headers 416 or fields thereof on hops along the path, requiring for example 417 more complex packet parsers. In SR-MPLS it is possible to simply 418 remove the top SID on a node that has processed it, in SRv6 it is 419 instead necessary to look up an offset field in the SRH and, read 420 the appropriate SID (which may be deep in the packet), and then 421 increment the offset field. 423 o Even though the number of identifiers required within a controlled 424 network is often less than 16 bit, and almost always 32 bits, 425 carrying the overhead of 128 bits per SID in SRv6 can be seen as a 426 significant unnecessary overhead, and workarounds such a proposed 427 micro programs [I-D.bonica-6man-comp-rtg-hdr], 428 [I-D.bonica-spring-srv6-plus], 429 [I-D.filsfils-spring-net-pgm-extension-srv6-usid] require complex 430 forwarding plane processing and SRv6 programmability in the lower 431 64 bit is not required in the majority of use-cases for SIDs on 432 midpoints. 434 For use-cases like this, it would be a lot easier to innovate IPv6 by 435 clone & modify: E.g.: defining (say) IPv7 to be similar to IPv6, but 436 without the constraints that are not useful for the controlled 437 network use-case. A better alternative would be to create different 438 profiles of IPv6 with [RFC8200] being one. However, there is, as 439 yet, no concept of "profiles" in IPv6. 441 The issue of IP protocol operation in limited domains is discussed in 442 [I-D.carpenter-limited-domains]. 444 Some possible solutions are described in 445 [I-D.herbert-6man-eh-attrib]. This will be considered further in a 446 future version of this text. 448 4.1.1.2. IPv6 for Edge-Compute 450 Today, the majority of end-to-end connections already do not pass via 451 the traditional "Internet-Path" but instead toward a server in data 452 center co-located with the access service provider Edge-2-Edge-EP 453 [DOT]. In this case, there is no transit service provider, but there 454 is a well-established commercial relationship between either end of 455 the communications and the access service provider. 457 Today, the majority of traffic consists of video-streaming/TV 458 services, but in the future, Edge-Compute will enable ever more 459 applications to operate in such a controlled environment. 461 The difference between the aforementioned use-case of IPv6 within an 462 service provider, and this use-case is that enhanced services in this 463 would naturally operate end-to-end between a Data Center application 464 server and the subscriber endpoints. 466 In the case of SRv6, it is not necessary to incur the overhead of an 467 IPv6 in IPv6 encapsulation, the SRH can be inserted by the endpoint 468 and removed by the endpoint on the other side. Nevertheless, the 469 [RFC8200] limitations of not being able to add/remove or freely 470 change the content of the SRH payload or any other EH on a midpoint 471 router still exists. This seriously limits the usage and evolution 472 of IPv6 to the edge-to-edge model. 474 4.1.1.3. Hop-by-Hop Extension Header processing 476 Hop-by-hop IPv6 extension headers caused interoperability and 477 performance issues and as a result caused resistance to further 478 leverage and extend them except for SRv6-SRH RPL-SRH [RFC6554]. In 479 the authors opinions, this regression on hop-by-hop extension headers 480 is because of a combination of insufficient specifications and 481 resulting implementation issues. Both could be solved in future work 482 with new hop-by-hop processing specifications. 484 For example, router alert (RA) was (and still maybe) implemented in 485 routers so that all router alert packets are punted from the fast- 486 path to the slow-path even when the "value" field identifies a 487 protocol that the router can not process. As a result, protocols 488 that rely on RA such as RSVP [RFC2205] or even more so Pragmatic 489 General Multicast (PGM) [RFC3208] where filtered in networks because 490 they caused high control plane load on routers that did not support 491 either protocols but still unnecessarily punted their packets with 492 RA. 494 There are no normative statements about the need that fast-path 495 forwarding planes "MUST" be able to ignore unsupported/not-enabled EH 496 features at a speed such that such a packet can be forward at the 497 same speed as the same packet without the EH. For example, for RA, 498 there is only a "SHOULD" requirement to do this in [RFC6398], a BCP 499 published a decade after IPv6 router alert [RFC2711]. With such a 500 gap in time between the specification and the BCP, it is impossible 501 to rely on the existing RA and expect safe deployment across the 502 Internet without still running into performance issues. 504 4.1.1.4. Segment Routing Header Constraints 506 The same design paradigm could have been used for the Segment Routing 507 Header (SRH) [RFC8754], but there is no distinction possible for IPv6 508 instances running in such a controlled network or running as an 509 Internetwork instance to form the Internet. This is particularly 510 unfortunate as we are evolving to a model where, as noted earlier in 511 this document, in most cases the packet will only travel through two 512 well-known networks: the hosts network and the service provider 513 network hosting the server to which the client is interacting. 515 4.1.2. Fixed Address Length 517 When IPv6 was designed, the key focus was on solving the problem of 518 growth of the Internet and resulting growth of global Internet 519 address space. Variable length and a heterogeneous address approach 520 were proposed [RFC1347] however, these were rejected partially for 521 political reasons and partially out of a concern over the difficulty 522 of parsing the packet and doing a fast address lookup. 524 There was seemingly no focus on better supporting the now millions of 525 often network-layer isolated TCP/IP networks in industrial, defense, 526 research, embedded, industrial or other commercial environments. 528 One key problems with with 128 bit addresses is the overhead on low- 529 speed radio/IoT-wire networks. This is especially the case when 530 using source-routing, where multiple of these addresses have to be 531 included in the header. Current solutions are only able to resolve 532 these issues with CPU expensive IETF standardized header compression 533 techniques [RFC2507], [RFC3095], [RFC5795]. Even though these 534 approaches are feasible in many of todays IoT networks, there is a 535 strong desire to reduce power consumption in such devices. This is 536 particularly the case where they are powered by a single-for-life- 537 battery, or are self-powering through automatic replenished energy 538 sources. As a result of this CPU performance in future IoT network 539 should not be expected to increase but whenever feasible is more 540 likely to decrease. 542 Another, often overlooked, problem of the 128 bit IPv6 addresses is 543 that global address prefix allocation is a a big up-front burden on 544 many IoT networks, but also isolated networks (industrial, defense, 545 research, industrial). Often, this leads to the use of Unique Local 546 Addresses (ULA) [RFC4193], which have the risk of conflicts when 547 those previously isolated networks need to interconnect with other 548 networks. 550 A further insight into the issues of IPv6 address lengths of 128 bits 551 can be seen in the tussle over how to compress the address lengths in 552 Segment Routing and network programming (in no particular order): 554 [I-D.bonica-6man-comp-rtg-hdr], [I-D.bonica-6man-crh-helper-opt], 555 [I-D.bonica-spring-sr-mapped-six], 556 [I-D.cheng-spring-shorter-srv6-sid-requirement], 557 [I-D.decraene-spring-srv6-vlsid], 558 [I-D.filsfilscheng-spring-srv6-srh-comp-sl-enc], 559 [I-D.lc-6man-generalized-srh], [I-D.li-spring-compressed-srv6-np], 560 [I-D.mirsky-spring-unified-id-network-programming], 561 [I-D.steinberg-6man-crh-vs-sr-mpls], [I-D.templin-6man-crh-variable]. 563 The root cause of this debate is the inflexibility of IPv6 in terms 564 of its address length and semantics. 566 While solutions to these problems may look easier enough, it should 567 be noted that in the time when IPv6 was designed, variable length 568 addresses in the fastest forwarding planes were not seen as feasible, 569 and there was also a lack of experience with the impact of 570 interconnecting heterogeneous address spaces other than as ships-in- 571 the-night parallel operation of protocols. A lot of that experience 572 came later through 14++ IPv4/IPv6 transition solutions designed in 573 the past 20 years and respective work on address discovery in IETF 574 frameworks such as SIP/STUN/ICE. 576 Another issue with the fixed length homogeneous address approach is 577 the constraints this places on the current practice of overloading 578 addresses with other functionality for example 579 [I-D.ietf-spring-srv6-network-programming]. 581 Since the original decision to only support fixed length packet 582 addresses was taken there has been a significant improvement in the 583 packet lookup capability of hardware. This is has been driven by the 584 need to perform complex ACL lookup for security reasons and the 585 interest in flow based techniques such as OpenFlow. It is thus worth 586 revisiting the decision to only allow a single fixed address length 587 and format. 589 4.2. Better Than Best Effort E2E Network Services 591 Some of the fastest growing network segments where new services are 592 being introduced in an End-2-End manner belong to deployment models 593 as described in [I-D.bryant-arch-fwd-layer-uc]. The requirements 594 here for service delivery involves stringent E2E latency with no 595 retransmission and no packet loss. Not all scenarios need "lower" 596 latency but bounded to a particular value/range. Example use cases 597 involving an user equipment (UE) consuming service from the provider 598 cloud network or another UE (e.g. Vehicular device, IIoT) in the 599 same network. Here the service endpoints could be connected over 600 wire or wireless (LTE/NR) and the service termination happens in the 601 provider network either close to the access network or provider core 602 network. The existing network layer and best-effort model simply 603 cannot guarantee needed service level objectives in these scenarios. 605 Some specific needs and requirements from cellular fixed transport 606 networks are: 608 o Need for determinism on E2E throughput and latency. The current 609 TCP/IP is hence not-suitable for Mission-critical and real-time 610 E2E applications. 612 o Need for E2E QoS for ultra-reliable-low-latency communications 613 (uRLLC). 615 o Efficient use of protocols in the network by minimizing tunnels 616 over tunnels and duplicate header fields. 618 o Efficient deployment of network slicing 620 4.3. Adaptive Bit-rate Video streaming 622 Even without going to future application requirements as described 623 elsewhere in this document, even the majority of existing Internet 624 traffic is lacking competitively usable and standardized service to 625 support quality of service. 627 The majority of traffic today is Adaptive Bit-rate (ABR) based audio/ 628 video streaming. The primary benefit of this approach is that it can 629 adjust itself to much lower bandwidth than the bandwidth to offer the 630 ideal/target experience quality to the user. It therefore enabled 631 Over The Top (OTT) services to offer streaming media. Nevertheless, 632 ABR itself does not provide any actual quality guarantees. 634 Service providers that use ABR streaming to their subscribers do 635 therefore combine ABR with IP developments, some non-published, which 636 are often out-of-band bandwidth reservation schemes. These allow ABR 637 video streams to have their ideal/target experience bandwidth within 638 the SP's network and only need to degrade if there was bandwidth 639 contention in the subscribers (home) network. 641 If a subscriber, or a content provider which is not the access 642 service provider wanted to get the same type of bandwidth guarantees 643 for other content across the access providers network, they could do 644 so with existing IETF standards via RSVP [RFC2205] which is widely 645 implemented, or NSIS [RFC4080], which was to the knowledge of the 646 authors never implemented in widely used router products (because it 647 does not offer sufficient benefits over RSVP). In either case, the 648 per-flow control-plane based signaling architecture including the 649 aforementioned router-alert issues make these protocols a difficult, 650 likely not future-proof solution. 652 Even more fundamentally, ABR has shown that media streaming can 653 easily support elastic adjustment between a range of bandwidth limits 654 in which the quality is between acceptable and ideal, but there is as 655 of today no standardized mechanisms by which to express relative 656 bandwidth allocations when streams compete against each other that 657 goes beyond the very loosely defined "internet fairness". For 658 example, more intelligent congestion management could defend 659 bandwidth the more the bandwidth approaches the minimum acceptable 660 bandwidth, or admission control of bandwidth could be elastic. Some 661 work in these direction exists in [RFC8698] with its ability for 662 weighted congestion control or 663 [I-D.ietf-tsvwg-intserv-multiple-tspec] for (limited) elastic 664 admission control management. 666 4.4. Limited Domain Opportunities 668 Strictly of course this refers to the opportunities that the 669 acceptance of limited domains [I-D.carpenter-limited-domains] 670 provides to the network operator in terms of the flexibility to 671 enhance packet delivery in cases of high value traffic. 673 The removal of the constraint of a globally uniform protocol, such as 674 unenhanced IPv6 would allow a best in class, domain specific 675 forwarding layer to be deployed without the constant of the 676 requirement that the protocol needed to serve all purposed, for all 677 applications in all parts of the global network. 679 These opportunities are are further enhanced by noting that the 680 delivery protocol to the application server, which as noted elsewhere 681 in this text is moving closer to the edge, does not need to be the 682 same as the host to application protocol since this is increasingly 683 being opaquely tunneled over the delivery protocol. Furthermore, any 684 distributed set of application servers maybe in their own domain, and 685 this is not constrained to the same protocol that is used between the 686 client and the server. 688 Clearly their are costs and complexities associated with moving from 689 a globally heterogeneous protocol to a domain specific protocol, but 690 the deciding factors are whether the application is deliverable over 691 a globally general purpose forwarding layer, and whether there 692 application and delivery system are economically attractive. 694 4.5. DetNet and Higher Precision Networking Service 696 Time Critical (TC), Ultra-Reliable, Low Latency (URLLC), Internet-of- 697 Things is another important use case scenario-set that highlights 698 requirements that are difficult to satisfy with existing Internet 699 connectivity paths where a part of that path includes a radio access 700 link. These kind of close-loop control systems borne over 701 heterogeneous communications networks have very precision and bounded 702 latency requirements for the E2E network connecting the sensor and 703 actuator. 705 Deterministic networking within the IETF is focused on only one 706 dimension of the URLLC problem. 708 DetNet is also far from attempting to identify currently if/how the 709 services it plans to introduce could be made to operate over the 710 Internet in general, instead, it focuses mostly on the shorter term 711 goal to enable them in controlled networks within a limited domains. 713 Currently, the requirements for a DetNet forwarding plane have been 714 reasonably mapped out for an MPLS based forwarding layer. 715 Nevertheless, in addressing these needs within an IP network 716 [I-D.ietf-detnet-ip] the solution has of necessity been limited to 717 the capabilities of the IP as it exists today. It has not, for 718 example, been possible to add the packet replication elimination and 719 reordering function (PREOF)which allows multiple concurrent packet 720 delivery attempts in an MPLS network [I-D.ietf-detnet-mpls]. The 721 DETNET body of requirements needs to be revisited in the light of any 722 development to network forwarding capabilities. 724 4.6. Forwarding Plane vs. Control Plane 726 High-end hardware with accelerated forwarding plane devices, can 727 support a significant number of forwarding states including 728 destination entries (IP destination/mask, MPLS label, SR SID) as well 729 as 2, 3 or 5 tuple IP/IPv6 "flow" entries. Nevertheless, the control 730 plane that builds and changes these entries often limits their 731 usability because the control plane does not even scale to the number 732 of hardware accelerated forwarding entries possible, or because the 733 supported rate of changes is slow. 735 The root of this problem is that with the increase of speed and scale 736 of hardware accelerated forwarding hardware, control plane had 737 challenges to keep up in performance. The performance of 738 appropriately priced control plane CPUs (relative to the cost of the 739 forwarding plane) has not grown at the same speed as that of hardware 740 accelerated forwarding plane chips. 742 One of the directions to overcome these challenges is invisible 743 outside these forwarder devices and it is to optimize the control- 744 plane to forwarding plane interactions, such as programming the 745 building of forwarding state directly on the accelerated forwarding 746 infrastructure (e.g. NPU), but using otherwise existing control 747 plane protocols. 749 A more fundamental approach is to redesign control plane protocols 750 such that they are lighter weight in their signaling and state 751 machinery, and can therefore be completely implemented in the 752 hardware accelerated forwarding plane. Effectively turning a control 753 plane protocol into an advanced forwarding plane protocol function. 755 This approach is logically most easily applicable to on-path per-flow 756 signaling mechanisms such as RSVP or RSVP-TE, both of which are quite 757 complex with their signaling messaging and state keeping and 758 therefore directly infeasible to become hardware accelerated 759 forwarding implementations. An example approach to provide similar 760 functionality to RSVP with signaling light-weight enough to allow 761 hardware accelerated implementation are the in-band signaling 762 mechanisms (e.g. for TCP or UDP) described in [DIP1] 763 [I-D.han-tsvwg-ip-transport-qos] [I-D.han-tsvwg-enhanced-diffserv]. 765 Signaling that is feasible to become part of a complete in- 766 forwarding-plane signaling solution is not limited to in-band on-path 767 flow signaling, but would likely also be applied to other signaling 768 options. Of the aforementioned existing signaling protocols, IGPs 769 are likely the ones whose signaling could most easily be processed in 770 an NPU compute elements except that the SPF calculation itself 771 introduces a complexity that would make this very complex. One 772 example of a solution that solves this problem by signaling the 773 actual per-hop adjacencies in IGP and therefore eases NPU 774 implementation can be found in 775 [I-D.chunduri-isis-preferred-path-routing]. 777 In summary: The scope of what should be considered forwarding plane 778 today is defined by decade historic architectures, but should for the 779 future be scoped by the realities of the new, different "layers" of 780 hardware and their capabilities. Hence also the use of the term 781 forwarding plane, because it can span not only across classical 782 bridging (L2), label/tag/SIG switching (L2.5), network/internetwork 783 (L3) and transport (L4) layers, but also across the classical "data 784 plane" and "control plane" components of each such layer. 786 4.7. User-Network/Network-User Interface Signaling 788 Some of the deployment models as described in [I-D.bryant-arch-fwd- 789 layer-uc], needs specific signaling mechanism from user/applications. 790 These are needed for E2E service offering for better than best effort 791 Section 4.2 or high-precision networking Section 4.5. These may 792 involve new transport mechanisms at hosts, middle-boxes and routers 793 to meet the E2E service requirements in these limited domain 794 deployments. 796 Here one of the functional requirements is to signal the service 797 level objectives (SLOs) dynamically for a particular service from the 798 network. This signaling includes the service description, the 799 service negotiation with the network, the service setup or 800 modification, or the need to execute some functions at network device 801 and send the results back to the sender. However, the current IP was 802 not designed for this. For example, the result of SLO negotiation at 803 any hop needs to be updated in the IP packet at the router and 804 returned back to the sender (originating host or gateway device for a 805 Service Provider). 807 There are some attempts to achieve the above as described in 808 [I-D.han-tsvwg-ip-transport-qos], which describes general in-band 809 signaling for QoS control with IPv6 protocol and 810 [I-D.han-tsvwg-enhanced-diffserv], which proposes a backward 811 compatible class-based queuing and scheduling schema for hybrid 812 service to support guaranteed service from the network (e.g. for 813 latency and bandwidth). 815 In summary, it is difficult to do better than best effort or High 816 Precision Services described in Section Section 4.5, in closed 817 domains with current IP given the best effort congestion control 818 (TCP/QUIC) and explicit congestion notification (ECN) framework. A 819 comprehensive mechanism needs to be explored as the limitations in 820 silicon technologies or deployment models 30 years ago are not 821 relevant with respect to security, scalability, packet size change, 822 MSS or FCS recalculation, etc. 824 5. Candidate Solution Directions 826 This section is an incomplete list of solution considerations, but is 827 not prescriptive about any specific approach or technical solution, 828 and is provided to stimulate thought on the subject. 830 5.1. Variable Length Addresses 832 When private networks are set up, they only need to use an address 833 length that allows the construction of networks sufficiently large to 834 meet the expected service requirements. If a future network layer 835 protocol could support address length of e.g.: 16, 24, 32, 48, 64 and 836 128 bits (or maybe more), it would be easy for such networks to pick 837 a right size. This would allow them to have as efficient packets 838 without compression as possible, and it would also avoid for them to 839 have to think about allocation procedures for "global" addresses. 841 Whenever networks with a smaller address size would later on have to 842 interconnect to other networks, the shorter length address would have 843 to be interpreted as the suffix of a sufficiently larger address 844 space through which those connecting networks could achieve unique, 845 non-overlapping addresses. At the border between these networks, 846 high speed forwarding planes could easily perform per-packet 847 stateless prefix addition/deletion transformations of addresses in 848 the packet header when the interconnection should be free of further 849 policy. When such an interconnection is desired to employ specific 850 traffic control policies, mapping of addresses in a stateful manner 851 is a convenient way to enforce and support such policies through the 852 forwarding plane. 854 5.2. Address Semantics 856 Classically IP unicast addresses identify an interface. There is the 857 special case of a loop-back address, but this is normally modeled as 858 an internal interface. Addresses are often silently mapped to 859 include other semantics and this is most developed in the IP network 860 programming concept [I-D.ietf-spring-srv6-network-programming]. 862 MPLS is more general. It defines the concept of a Forwarding 863 Equivalence Class in which a Label which can be visualized as an 864 offset into a specific table with up to 2^20 entries, with the table 865 containing the instruction to be executed. Thus a single identifier 866 is able to specify: forward towards an egress, forward along a 867 specific path, decapsulate and sent to an interface, decapsulate and 868 forward via an IP lookup in a label specific address table etc. 870 The semantics of the MPLS label and the size of the label are such 871 that it is not possible to include any instruction parameters in the 872 label and very inefficient to include those parameters in one or more 873 further labels. The only example of doing this is the Entropy Label 874 indicator [RFC6790] which uses two Label Stack Entries (LSEs). Any 875 future development along these lines will need at least three LSEs. 877 Whilst an IPv6 is larger there is still limited space to add 878 parameters within the address. In the current work on this the size 879 is limited to 16 bits, and there is a fundamental limit of 64 bits. 881 It is clear that move is towards a multiplicity of semantic for the 882 network layer address, and indeed a formal recognition that the 883 address is in reality an instruction with a specific scope. 885 5.3. Multiple Instructions 887 What we have learned from MPLS and then from SRv6 is that it is often 888 desirable for a node (be that the originating host or a router) to 889 impose on a packet a set of instructions to be executed in sequence 890 by one or more entities in the network. An development of IP or any 891 successor needs to recognize this and provide a simple and efficient 892 way to incorporate a list (or stack) of instructions within the 893 packet header. 895 5.4. Node and Path Specific Processing Instructions 897 There is an established need to do node specific instructions as is 898 indicated by the design of MPLS and Segment Routing (SR). Any 899 development of the forwarding system needs to retain this feature and 900 ideally develop a method that is simultaneously both general and 901 efficient. 903 References to efficiency include efficiency in packet size and 904 efficiency decoding and and executing the instruction. The 905 efficiency of encoding is not simply a matter of on the wire 906 bandwidth, but is also a matter of the size of the forwarder packet 907 header cache. This cache has to operate at wire speed can be an 908 expensive silicon element. 910 There is also a need to do path specific operations as are done in 911 RSVP-TE. However RSVP has a significant path set-up and path 912 maintenance cost. Clearly a per path instruction can be specified as 913 a set of N per node instructions where N is the number of hops along 914 the path, for example by using SR, but that is not an efficient 915 encoding where N is large. It is thus a useful optimization to 916 include the ability to include per path instructions, and this is the 917 subject of further study. 919 5.5. Integrated Assurance and Verification 921 Being best effort in nature, assurance for services provided using IP 922 is left to add-on solutions built after the fact. How to perform 923 tasks such as verifying of service levels is left as an exercise for 924 network providers, often approached using statistical approaches that 925 are themselves "best effort" in nature. This will be no longer 926 sufficient for mission-critical services such as tele-driving or 927 tele-operations that demand guarantees, where failure to meet those 928 guarantees may expose providers and users exposed to liability 929 demands and call the feasibility of applications relying on those 930 services into question. 932 Moving forward, network protocols suitable to deliver high-precision 933 services for mission critical applications need to address assurance 934 as an intrinsic property, not left to afterthoughts. 936 5.6. For Consideration in a Future Version 938 A future version of this document will consider E2E communication 939 beyond-best-effort, high precision services, high precision 940 telemetry, E2E Volumetric data transfer and high precision congestion 941 control beyond that provided by the diffserv QoS bits. 943 6. IANA Considerations 945 This document does not request any allocations from IANA. 947 7. Security Considerations 949 Security is likely to be more significant with the applications being 950 considered in this work. With interest in tightly controlled access 951 and latency, and contractual terms of business it is going to be 952 necessary to have provable right of access to network resources. 953 However heavyweight security is a contra-requirement to the light- 954 weight process needed for power efficiency, fast forwarding and low 955 latency. Addressing this will require new insights into network 956 security. 958 Further information on the issue of providing security in latency 959 sensitive environments can be found in [I-D.ietf-detnet-security] 960 which are a sub-set of the considerations applicable to the new use 961 cases considered in this text. 963 8. References 965 8.1. Normative References 967 [RFC0791] Postel, J., "Internet Protocol", STD 5, RFC 791, 968 DOI 10.17487/RFC0791, September 1981, 969 . 971 [RFC8200] Deering, S. and R. Hinden, "Internet Protocol, Version 6 972 (IPv6) Specification", STD 86, RFC 8200, 973 DOI 10.17487/RFC8200, July 2017, 974 . 976 8.2. Informative References 978 [DIP1] ETSI, "Recommendation for New Transport Technologies, GR 979 NGP 010", September 2018, 980 . 983 [DOT] Huston, G., "The Death of Transit and Beyond", n.d., 984 . 987 [I-D.bonica-6man-comp-rtg-hdr] 988 Bonica, R., Kamite, Y., Niwa, T., Alston, A., and L. 989 Jalil, "The IPv6 Compact Routing Header (CRH)", draft- 990 bonica-6man-comp-rtg-hdr-22 (work in progress), May 2020. 992 [I-D.bonica-6man-crh-helper-opt] 993 Li, X., Bao, C., Ruan, E., and R. Bonica, "Compressed 994 Routing Header (CRH) Helper Option", draft-bonica-6man- 995 crh-helper-opt-01 (work in progress), May 2020. 997 [I-D.bonica-spring-sr-mapped-six] 998 Bonica, R., Hegde, S., Kamite, Y., Alston, A., Henriques, 999 D., Jalil, L., Halpern, J., Linkova, J., and G. Chen, 1000 "Segment Routing Mapped To IPv6 (SRm6)", draft-bonica- 1001 spring-sr-mapped-six-01 (work in progress), April 2020. 1003 [I-D.bonica-spring-srv6-plus] 1004 Bonica, R., Hegde, S., Kamite, Y., Alston, A., Henriques, 1005 D., Jalil, L., Halpern, J., Linkova, J., and G. Chen, 1006 "Segment Routing Mapped To IPv6 (SRm6)", draft-bonica- 1007 spring-srv6-plus-06 (work in progress), October 2019. 1009 [I-D.carpenter-limited-domains] 1010 Carpenter, B. and B. Liu, "Limited Domains and Internet 1011 Protocols", draft-carpenter-limited-domains-13 (work in 1012 progress), February 2020. 1014 [I-D.cheng-spring-shorter-srv6-sid-requirement] 1015 Cheng, W., Xie, C., Pang, R., Li, Z., Chen, R., Lijun, L., 1016 Duan, X., and G. Mirsky, "Shorter SRv6 SID Requirements", 1017 draft-cheng-spring-shorter-srv6-sid-requirement-01 (work 1018 in progress), March 2020. 1020 [I-D.chunduri-isis-preferred-path-routing] 1021 Chunduri, U., Li, R., White, R., Tantsura, J., Contreras, 1022 L., and Y. Qu, "Preferred Path Routing (PPR) in IS-IS", 1023 draft-chunduri-isis-preferred-path-routing-00 (work in 1024 progress), June 2018. 1026 [I-D.decraene-spring-srv6-vlsid] 1027 Decraene, B., Raszuk, R., Li, Z., and C. Li, "SRv6 vSID: 1028 Network Programming extension for variable length SIDs", 1029 draft-decraene-spring-srv6-vlsid-03 (work in progress), 1030 March 2020. 1032 [I-D.filsfils-spring-net-pgm-extension-srv6-usid] 1033 Filsfils, C., Camarillo, P., Cai, D., Voyer, D., Meilik, 1034 I., Patel, K., Henderickx, W., Jonnalagadda, P., Melman, 1035 D., Liu, Y., and J. Guichard, "Network Programming 1036 extension: SRv6 uSID instruction", draft-filsfils-spring- 1037 net-pgm-extension-srv6-usid-07 (work in progress), May 1038 2020. 1040 [I-D.filsfilscheng-spring-srv6-srh-comp-sl-enc] 1041 Cheng, W., Filsfils, C., Li, Z., Cai, D., Voyer, D., Clad, 1042 F., Shay, S., Guichard, J., and L. Aihua, "Compressed SRv6 1043 Segment List Encoding in SRH", draft-filsfilscheng-spring- 1044 srv6-srh-comp-sl-enc-01 (work in progress), May 2020. 1046 [I-D.han-tsvwg-enhanced-diffserv] 1047 Han, L., Qu, Y., and R. Li, "Enhanced DiffServ by In-band 1048 Signaling", draft-han-tsvwg-enhanced-diffserv-00 (work in 1049 progress), November 2019. 1051 [I-D.han-tsvwg-ip-transport-qos] 1052 Han, L., Qu, Y., Dong, L., Li, R., Nadeau, T., Smith, K., 1053 and J. Tantsura, "Resource Reservation Protocol for IP 1054 Transport QoS", draft-han-tsvwg-ip-transport-qos-03 (work 1055 in progress), October 2019. 1057 [I-D.herbert-6man-eh-attrib] 1058 Herbert, T., "Attribution Option for Extension Header 1059 Insertion", draft-herbert-6man-eh-attrib-01 (work in 1060 progress), July 2020. 1062 [I-D.ietf-detnet-ip] 1063 Varga, B., Farkas, J., Berger, L., Fedyk, D., and S. 1064 Bryant, "DetNet Data Plane: IP", draft-ietf-detnet-ip-07 1065 (work in progress), July 2020. 1067 [I-D.ietf-detnet-mpls] 1068 Varga, B., Farkas, J., Berger, L., Malis, A., Bryant, S., 1069 and J. Korhonen, "DetNet Data Plane: MPLS", draft-ietf- 1070 detnet-mpls-09 (work in progress), July 2020. 1072 [I-D.ietf-detnet-security] 1073 Mizrahi, T. and E. Grossman, "Deterministic Networking 1074 (DetNet) Security Considerations", draft-ietf-detnet- 1075 security-10 (work in progress), May 2020. 1077 [I-D.ietf-ippm-ioam-data] 1078 Brockners, F., Bhandari, S., Pignataro, C., Gredler, H., 1079 Leddy, J., Youell, S., Mizrahi, T., Mozes, D., Lapukhov, 1080 P., remy@barefootnetworks.com, r., daniel.bernier@bell.ca, 1081 d., and J. Lemon, "Data Fields for In-situ OAM", draft- 1082 ietf-ippm-ioam-data-09 (work in progress), March 2020. 1084 [I-D.ietf-spring-srv6-network-programming] 1085 Filsfils, C., Camarillo, P., Leddy, J., Voyer, D., 1086 Matsushima, S., and Z. Li, "SRv6 Network Programming", 1087 draft-ietf-spring-srv6-network-programming-16 (work in 1088 progress), June 2020. 1090 [I-D.ietf-tsvwg-intserv-multiple-tspec] 1091 Polk, J. and S. Dhesikan, "Integrated Services (IntServ) 1092 Extension to Allow Signaling of Multiple Traffic 1093 Specifications and Multiple Flow Specifications in 1094 RSVPv1", draft-ietf-tsvwg-intserv-multiple-tspec-02 (work 1095 in progress), February 2013. 1097 [I-D.lc-6man-generalized-srh] 1098 Li, Z., Li, C., Cheng, W., Xie, C., Cong, L., Tian, H., 1099 and F. Zhao, "Generalized Segment Routing Header", draft- 1100 lc-6man-generalized-srh-00 (work in progress), February 1101 2020. 1103 [I-D.li-spring-compressed-srv6-np] 1104 Li, Z., Li, C., Xie, C., LEE, K., Tian, H., Zhao, F., 1105 Guichard, J., Cong, L., and S. Peng, "Compressed SRv6 1106 Network Programming", draft-li-spring-compressed- 1107 srv6-np-02 (work in progress), February 2020. 1109 [I-D.mirsky-spring-unified-id-network-programming] 1110 Cheng, W., Mirsky, G., Aihua, L., and S. Peng, "SRv6 1111 network programming using Unified Identifier", draft- 1112 mirsky-spring-unified-id-network-programming-00 (work in 1113 progress), March 2020. 1115 [I-D.steinberg-6man-crh-vs-sr-mpls] 1116 Steinberg, D., Henderickx, W., Li, Z., Cheng, W., and D. 1117 Voyer, "SR-MPLS over IPv6 satisfies CRH requirements", 1118 draft-steinberg-6man-crh-vs-sr-mpls-00 (work in progress), 1119 June 2020. 1121 [I-D.templin-6man-crh-variable] 1122 Templin, F., "IPv6 Compressed Routing Header with Variable 1123 Length Addresses", draft-templin-6man-crh-variable-00 1124 (work in progress), May 2020. 1126 [RFC1347] Callon, R., "TCP and UDP with Bigger Addresses (TUBA), A 1127 Simple Proposal for Internet Addressing and Routing", 1128 RFC 1347, DOI 10.17487/RFC1347, June 1992, 1129 . 1131 [RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S. 1132 Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1 1133 Functional Specification", RFC 2205, DOI 10.17487/RFC2205, 1134 September 1997, . 1136 [RFC2460] Deering, S. and R. Hinden, "Internet Protocol, Version 6 1137 (IPv6) Specification", RFC 2460, DOI 10.17487/RFC2460, 1138 December 1998, . 1140 [RFC2507] Degermark, M., Nordgren, B., and S. Pink, "IP Header 1141 Compression", RFC 2507, DOI 10.17487/RFC2507, February 1142 1999, . 1144 [RFC2711] Partridge, C. and A. Jackson, "IPv6 Router Alert Option", 1145 RFC 2711, DOI 10.17487/RFC2711, October 1999, 1146 . 1148 [RFC3095] Bormann, C., Burmeister, C., Degermark, M., Fukushima, H., 1149 Hannu, H., Jonsson, L-E., Hakenberg, R., Koren, T., Le, 1150 K., Liu, Z., Martensson, A., Miyazaki, A., Svanbro, K., 1151 Wiebke, T., Yoshimura, T., and H. Zheng, "RObust Header 1152 Compression (ROHC): Framework and four profiles: RTP, UDP, 1153 ESP, and uncompressed", RFC 3095, DOI 10.17487/RFC3095, 1154 July 2001, . 1156 [RFC3208] Speakman, T., Crowcroft, J., Gemmell, J., Farinacci, D., 1157 Lin, S., Leshchiner, D., Luby, M., Montgomery, T., Rizzo, 1158 L., Tweedly, A., Bhaskar, N., Edmonstone, R., 1159 Sumanasekera, R., and L. Vicisano, "PGM Reliable Transport 1160 Protocol Specification", RFC 3208, DOI 10.17487/RFC3208, 1161 December 2001, . 1163 [RFC4080] Hancock, R., Karagiannis, G., Loughney, J., and S. Van den 1164 Bosch, "Next Steps in Signaling (NSIS): Framework", 1165 RFC 4080, DOI 10.17487/RFC4080, June 2005, 1166 . 1168 [RFC4193] Hinden, R. and B. Haberman, "Unique Local IPv6 Unicast 1169 Addresses", RFC 4193, DOI 10.17487/RFC4193, October 2005, 1170 . 1172 [RFC4302] Kent, S., "IP Authentication Header", RFC 4302, 1173 DOI 10.17487/RFC4302, December 2005, 1174 . 1176 [RFC5795] Sandlund, K., Pelletier, G., and L-E. Jonsson, "The RObust 1177 Header Compression (ROHC) Framework", RFC 5795, 1178 DOI 10.17487/RFC5795, March 2010, 1179 . 1181 [RFC6282] Hui, J., Ed. and P. Thubert, "Compression Format for IPv6 1182 Datagrams over IEEE 802.15.4-Based Networks", RFC 6282, 1183 DOI 10.17487/RFC6282, September 2011, 1184 . 1186 [RFC6398] Le Faucheur, F., Ed., "IP Router Alert Considerations and 1187 Usage", BCP 168, RFC 6398, DOI 10.17487/RFC6398, October 1188 2011, . 1190 [RFC6554] Hui, J., Vasseur, JP., Culler, D., and V. Manral, "An IPv6 1191 Routing Header for Source Routes with the Routing Protocol 1192 for Low-Power and Lossy Networks (RPL)", RFC 6554, 1193 DOI 10.17487/RFC6554, March 2012, 1194 . 1196 [RFC6790] Kompella, K., Drake, J., Amante, S., Henderickx, W., and 1197 L. Yong, "The Use of Entropy Labels in MPLS Forwarding", 1198 RFC 6790, DOI 10.17487/RFC6790, November 2012, 1199 . 1201 [RFC8655] Finn, N., Thubert, P., Varga, B., and J. Farkas, 1202 "Deterministic Networking Architecture", RFC 8655, 1203 DOI 10.17487/RFC8655, October 2019, 1204 . 1206 [RFC8698] Zhu, X., Pan, R., Ramalho, M., and S. Mena, "Network- 1207 Assisted Dynamic Adaptation (NADA): A Unified Congestion 1208 Control Scheme for Real-Time Media", RFC 8698, 1209 DOI 10.17487/RFC8698, February 2020, 1210 . 1212 [RFC8754] Filsfils, C., Ed., Dukes, D., Ed., Previdi, S., Leddy, J., 1213 Matsushima, S., and D. Voyer, "IPv6 Segment Routing Header 1214 (SRH)", RFC 8754, DOI 10.17487/RFC8754, March 2020, 1215 . 1217 Authors' Addresses 1219 Stewart Bryant 1220 Futurewei Technologies Inc. 1222 Email: sb@stewartbryant.com 1223 Uma Chunduri 1224 Futurewei Technologies Inc. 1226 Email: uma.chunduri@futurewei.com 1228 Toerless Eckert 1229 Futurewei Technologies Inc. 1231 Email: tte@cs.fau.de 1233 Alexander Clemm 1234 Futurewei Technologies Inc. 1236 Email: ludwig@clemm.org