idnits 2.17.1 draft-han-tsvwg-ip-transport-qos-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet has text resembling RFC 2119 boilerplate text. -- The document date (October 15, 2019) is 1648 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'RFC2581' is defined on line 1152, but no explicit reference was found in the text == Unused Reference: 'I-D.falk-xcp-spec' is defined on line 1178, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-aqm-codel' is defined on line 1199, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-aqm-fq-codel' is defined on line 1204, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-aqm-pie' is defined on line 1211, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-tcpm-dctcp' is defined on line 1228, but no explicit reference was found in the text == Unused Reference: 'I-D.sridharan-tcpm-ctcp' is defined on line 1239, but no explicit reference was found in the text == Unused Reference: 'TCP-vegas' is defined on line 1285, but no explicit reference was found in the text ** Obsolete normative reference: RFC 2581 (Obsoleted by RFC 5681) == Outdated reference: A later version (-10) exists of draft-ietf-aqm-codel-06 == Outdated reference: A later version (-10) exists of draft-ietf-tcpm-dctcp-03 Summary: 1 error (**), 0 flaws (~~), 12 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 TSVWG Working Group L. Han 3 Internet-Draft Y. Qu 4 Intended status: Informational L. Dong 5 Expires: April 17, 2020 R. Li 6 Futurewei Technologies 7 T. Nadeau 8 Lucid Vision 9 K. Smith 10 Vodafone 11 J. Tantsura 12 Apstra 13 October 15, 2019 15 Resource Reservation Protocol for IP Transport QoS 16 draft-han-tsvwg-ip-transport-qos-03 18 Abstract 20 IP is designed for use in Best Effort Networks, which are networks 21 that provide no guarantee that data is delivered, or that delivery 22 meets any specified quality of service parameters. However there are 23 new applications requiring IP to provide deterministic services in 24 terms of bandwidth and latency, such as network based AR/VR 25 (Augmented Reality and Virtual Reality), industrial internet. This 26 document proposes a solution in IPv6 that can be used by transport 27 layer protocols to guarantee certain level of service quality. This 28 new service is fined-grained and could apply to individual or 29 aggregated TCP/UDP flow(s). 31 Status of This Memo 33 This Internet-Draft is submitted in full conformance with the 34 provisions of BCP 78 and BCP 79. 36 Internet-Drafts are working documents of the Internet Engineering 37 Task Force (IETF). Note that other groups may also distribute 38 working documents as Internet-Drafts. The list of current Internet- 39 Drafts is at https://datatracker.ietf.org/drafts/current/. 41 Internet-Drafts are draft documents valid for a maximum of six months 42 and may be updated, replaced, or obsoleted by other documents at any 43 time. It is inappropriate to use Internet-Drafts as reference 44 material or to cite them other than as "work in progress." 46 This Internet-Draft will expire on April 17, 2020. 48 Copyright Notice 50 Copyright (c) 2019 IETF Trust and the persons identified as the 51 document authors. All rights reserved. 53 This document is subject to BCP 78 and the IETF Trust's Legal 54 Provisions Relating to IETF Documents 55 (https://trustee.ietf.org/license-info) in effect on the date of 56 publication of this document. Please review these documents 57 carefully, as they describe your rights and restrictions with respect 58 to this document. Code Components extracted from this document must 59 include Simplified BSD License text as described in Section 4.e of 60 the Trust Legal Provisions and are provided without warranty as 61 described in the Simplified BSD License. 63 Table of Contents 65 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 66 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 67 3. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 6 68 3.1. Design Targets . . . . . . . . . . . . . . . . . . . . . 6 69 3.2. Scope and Assumptions . . . . . . . . . . . . . . . . . . 7 70 3.3. Sub-layer in IP for Transport Control . . . . . . . . . . 7 71 3.4. IP In-band signaling . . . . . . . . . . . . . . . . . . 8 72 3.5. IPv6 Approach . . . . . . . . . . . . . . . . . . . . . . 10 73 4. Key Messages and Parameters . . . . . . . . . . . . . . . . . 11 74 4.1. Setup and Setup State Report messages . . . . . . . . . . 11 75 4.2. Forwarding State and Forwarding State Report messages 12 76 4.3. Hop Number . . . . . . . . . . . . . . . . . . . . . . . 13 77 4.4. Flow Identifying Method and Service ID . . . . . . . . . 13 78 4.5. QoS State and life of Time . . . . . . . . . . . . . . . 14 79 4.6. Authentication . . . . . . . . . . . . . . . . . . . . . 14 80 5. Packet Forwarding . . . . . . . . . . . . . . . . . . . . . . 15 81 5.1. Basic Hardware Capability . . . . . . . . . . . . . . . . 15 82 5.2. Flow Identification in Packet Forwarding . . . . . . . . 16 83 5.3. QoS Forwarding State Detection and Failure Handling . . . 16 84 6. Details of Working with Transport Layer . . . . . . . . . . . 17 85 6.1. Working with TCP . . . . . . . . . . . . . . . . . . . . 17 86 6.2. Working with UDP and other Protocols . . . . . . . . . . 20 87 7. Additional Considerations . . . . . . . . . . . . . . . . . . 20 88 7.1. User and Application driven . . . . . . . . . . . . . . . 20 89 7.2. Traffic Management in Host . . . . . . . . . . . . . . . 21 90 7.3. Heterogeneous Network . . . . . . . . . . . . . . . . . . 22 91 7.4. Proxy Control . . . . . . . . . . . . . . . . . . . . . . 22 92 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 22 93 9. Security Considerations . . . . . . . . . . . . . . . . . . . 24 94 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 25 95 10.1. Normative References . . . . . . . . . . . . . . . . . . 25 96 10.2. Informative References . . . . . . . . . . . . . . . . . 26 97 Appendix A. Acknowledgements . . . . . . . . . . . . . . . . . . 28 98 Appendix B. Message Objects . . . . . . . . . . . . . . . . . . 29 99 B.1. Setup State Object . . . . . . . . . . . . . . . . . . . 29 100 B.2. Bandwidth Object . . . . . . . . . . . . . . . . . . . . 31 101 B.3. Burst Msg . . . . . . . . . . . . . . . . . . . . . . . . 31 102 B.4. Latency Object . . . . . . . . . . . . . . . . . . . . . 32 103 B.5. Authentication Object . . . . . . . . . . . . . . . . . . 32 104 B.6. OAM Object . . . . . . . . . . . . . . . . . . . . . . . 33 105 B.7. Forwarding State Object . . . . . . . . . . . . . . . . . 34 106 B.8. Setup State Report Object . . . . . . . . . . . . . . . . 34 107 B.9. Forward State Report Object . . . . . . . . . . . . . . . 35 108 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 36 110 1. Introduction 112 Recently, more and more new applications for The Internet are 113 emerging. These applications have a number of key requirements that 114 are common to all such as that their required bandwidth is very high 115 and/or latency is very low compared with traditional applications 116 like most of web and video applications. 118 For example, network based Augmented Reality (AR) or Virtual Reality 119 (VR) applications may need hundreds of Mbps bandwidth (throughput) 120 and a low single digit millisecond latency. Moreover, the difference 121 between mean bit rate and peak bit rate may be significant due to the 122 choice of compression algorithm 123 [I-D.han-iccrg-arvr-transport-problem]. This may result in large 124 bursts, and make traffic management more difficult. 126 Some future applications may expect networks to provide a bounded 127 latency service. One such example is tactile network [Tactile]. 129 With the technology development in 5G [HU5G][QU2016] and beyond, the 130 wireless access network is also increasing the demand for the Ultra- 131 Reliable and Low-Latency Communications (URLLC). This also leads to 132 the question of whether IP can provide such service in an Evolved 133 Packet Core (EPC)[EPC] network. IP is becoming more and more 134 important in the EPC when the Multi-access Edge Computing (MEC)[MEC] 135 for 5G requires the cloud and data service to move closer to eNodeB 136 [eNodeB]. 138 [I-D.ietf-detnet-use-cases] identifies some use cases from different 139 industries which have a common need for "deterministic flows". Such 140 flows require guaranteed bandwidth and bounded latency. 142 Traditionally, an IP network provides an unreliable or best-effort 143 datagram service over a collection of underlying networks (i.e.: 145 ethernet, ATM, etc...). Integrated services(IntServ) [RFC3175] 146 specifies a fine-grained QoS system, which requires all routers along 147 the traffic path to support it and maintain the states for resource 148 reserved IP flow(s), so it is difficult to scale up to keep track of 149 all the reservations. Differentiated services (DiffServ) [RFC2475] 150 specifies a simple and scalable mechanism to classify traffic and 151 provide more coarse QoS, however because it can only specify per-hop 152 behaviors (PHBs), and how individual routers deal with the DS 153 [RFC2474] field is configuration specific. It is difficult to 154 provide consistent resource reservation for specified class of 155 traffic, thus hard to support the end-to-end bandwidth or latency 156 guarantee. 158 The transport layer (TCP/UDP) on top of IP is based on the best- 159 effort-only service, which has influenced the transport layer 160 evolution for quite long time, and results in some widely accepted 161 assumptions and solutions, such as: 163 1. The IP layer can only provide basic P2P (point to point) or P2MP 164 (point to multi-point) end-to-end connectivity in the Internet, 165 but the connectivity is not reliable and does not guarantee any 166 quality of service to end-user or application, such as bandwidth, 167 packet loss, latency etc. Due to this assumption, the transport 168 layer or application must have its own control mechanism in 169 congestion and flow to obtain the reliable and satisfactory 170 service to cooperate with the under layer network quality. 172 2. The transport layer assumes that the IP layer can only process 173 all IP flows equally in the hardware since the best effort 174 service is actually an un-differentiated service. The process 175 includes scheduling, queuing and forwarding. Thus, the transport 176 layer must behave nicely and friendly to make sure all flows will 177 only obtain its own faired share of resource, and no one could 178 consume more and no one could be starved. 180 This document proposes a new IP transport service that guarantees 181 bandwidth and latency for new applications. The scope and criteria 182 for the new technology will also be discussed. This new IP transport 183 service is designed to be supplementary to regular IP transport 184 services, only meant to be used for special applications that are 185 bandwidth and/or latency sensitive. 187 2. Terminology 189 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 190 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 191 "OPTIONAL" in this document are to be interpreted as described in BCP 192 14 RFC 2119 [RFC2119] RFC 8174 [RFC8174] when, and only when, they 193 appear in all capitals, as shown here. 195 Abbreviations used in this documents: 197 E2E 198 End-to-end. 200 EH 201 IPv6 Extension Header or Extension Option. 203 QoS 204 Quality of Service. 206 OAM 207 Operation and Management. 209 In-band Signaling 210 In telecommunications, in-band signaling is sending control 211 information within the same band or channel used for voice or 212 video. 214 Out-of-band Signaling 215 out-of-band signaling is that the control information sent over 216 a different channel, or even over a separate network. 218 IP flow 219 For non-IPSec, an IP flow is identified by the source, 220 destination IP address, the protocol number, the source and 221 destination port number. 223 IP path 224 An IP path is the route that IP flow will traverse. It could 225 be the shortest path determined by routing protocols (IGP or 226 BPG), or the explicit path such as segment routing 227 [I-D.ietf-spring-segment-routing]. 229 QoS channel 230 A forwarding channel that is QoS guaranteed. It provides 231 additional QoS service to IP forwarding. A QoS channel can be 232 used for one or multiple IP flows depending on the granularity 233 of in-band signaling. 235 CIR 236 Committed Information Rate. 238 PIR 239 Peak Information Rate. 241 HbH-EH 242 IPv6 Hop-by-Hop Extension Header. 244 Dst-EH 245 IPv6 Destination Extension Header. 247 HbH-EH-aware node 248 Network nodes that are configured to process the IPv6 Hop-by- 249 Hop Extension Header. 251 3. Overview 253 Semiconductor chip technology has advanced significantly in the last 254 decade, and as such the widely used network processing and forwarding 255 process can now not only forward packets at line speed, but also 256 easily support other feature processing such as QoS for DiffServ/ 257 MPLS, Access Control List (ACL), fire wall, and Deep Packet 258 Inspection (DPI). 260 This advancement enables network processors to do the general process 261 to handle simple control messages for traffic management, such as 262 signaling for hardware programming, congestion state report, OAM, 263 etc. So now it's possible to treat some TCP/IP flows differently 264 from others and give them specified resource are feasible now by 265 using network processor. 267 This document proposes a deterministic IP transport service, which 268 can provide guaranteed bandwidth and latency. The solution is based 269 on the QoS implemented in network processor through in-band 270 signaling. 272 3.1. Design Targets 274 The proposed transport service is expected to satisfy the following 275 criteria: 277 o End user or application may directly use the new service. 279 o The new service can coexist with the current transport service and 280 is backward compatible. 282 o Service providers can manage the new service. 284 o Performance and scalability targets of this new service are 285 practical for vendors to achieve. 287 o The new service is transport agnostic. TCP, UDP and other 288 transport protocols on top of IP can use it. 290 3.2. Scope and Assumptions 292 The initial aim is to propose a solution for IPv6. To limit the 293 scope of the document and simplify the design and solution, the 294 following constraints are given: 296 1. The new service with QoS is aimed to be supplementary to regular 297 IP service. It is targeted for the applications that are 298 bandwidth and/or latency sensitive. It is not intended to 299 replace the TCP/IP variants that have been proved to be efficient 300 and successful for current applications. 302 2. The new service is limited within one administrative domain, even 303 it does not exclude the possibilities of extending the mechanism 304 for inter-domain scenarios. Currently only inter-domain security 305 is considered, and the inter-domain SLA, accounting and other 306 issues are not discussed. 308 3. Due to high bandwidth requirement of new service for individual 309 flow, the total number of the flows with the new service cannot 310 be high for a port, or a system. From another point of view, the 311 new service is targeted for applications that really need it, the 312 number of supported applications/users should be controlled and 313 cannot be unlimited. Hence the scalability requirement for the 314 new service is limited. 316 4. The new service must be able to coexist with the regular 317 transport service in the same hardware, and be backward 318 compatible. Also, a transport flow can switch between regular 319 transport and new service without service interruption. 321 3.3. Sub-layer in IP for Transport Control 323 In order to provide some new features for the layer above IP, it is 324 very useful to introduce an additional sub-layer, Transport Control, 325 between layer 3 (IP) and layer 4 (TCP/UDP). The new layer belongs to 326 IP, and is present only when the system needs to provide extra 327 control for the upper layer, in addition to the normal IP forwarding. 328 Fig 1. illustrates a new stack with the sub-layer. 330 +=========================+ 331 | APP | 332 +=========================+ 333 | TCP/UDP | 334 +=========================+ 335 | Transport Ctl | 336 +-+-+-+-+-+-+-+-+-+-+-+-+-+ 337 | IP | 338 +=========================+ 339 | Network Access | 340 +=========================+ 342 Figure 1: The new stack with a sub-layer in Layer 3 344 The new sub-layer is always bound with IP layer and can provide 345 support of the features for upper layer, such as: 347 In-band Signaling 348 The IP header with the new sub-layer can carry the signaling 349 information for the devices on the IP path. The information may 350 include all QoS related parameters used for hardware programming. 352 Congestion control 353 The congestion state in each device on the path can be detected 354 and notified to the source of flows by the sub-layer; The dynamic 355 congestion control instruction can also be carried by the sub- 356 layer and examined by network devices on the IP path. 358 IP Path OAM 359 The OAM instruction can be carried in the sub-layer, and the OAM 360 state can be notified to the source of flows by the sub-layer. 361 The OAM includes the path and device property detection, QoS 362 forwarding diagnosis and report. 364 IPv4 could use the IP option for the purpose of the sub-layer. But 365 due to the limit size of the IP option, the functionalities, 366 scalability of the layer is restricted. 368 IPv6 can realize the sub-layer easily using IPv6 extension header 369 [RFC8200]. The document will focus on the solution for IPv6 using 370 different IPv6 extension headers. 372 3.4. IP In-band signaling 374 In-band signaling messages are carried along with the payload. It is 375 guaranteed that the signaling follows the same path as the data flow, 376 and this can bring up some advantages that other methods can hardly 377 provide: 379 Diagnosis 380 The in-band signaling message takes the same path, same hops, same 381 processing at each hop as the data packet, this will make the 382 diagnosis for both signaling and data path easier. 384 Simplicity 385 The in-band signaling message is forwarded with the normal data 386 packet, it does not need to run a separate protocol. This will 387 dramatically reduce the complexity of the control. 389 Performance and scalability 390 Due to the simplicity of in-band signaling for control, it is 391 easier to provide a better performance and scalability for a new 392 future. 394 There have been similar works done or proposed in the industry for 395 quite some time. The in-band QoS signaling for IPv6 was discussed by 396 Lawrence Roberts in 2005 [I-D.roberts-inband-qos-ipv6]. The 397 requirements of IP in-band signaling was proposed by Jon Haper in 398 2007 [I-D.harper-inband-signalling-requirements]. Telecommunications 399 Industry Association (TIA) published a standard for "QoS Signaling 400 for IP QoS Support and Sender Authentication" in 2006 [TIA]. 402 This document proposes an optimized solution for QoS service using 403 in-band signaling, and it also tries to address issues raised by 404 previous proposals, such as security, scalability and performance. 406 The major differences from the previous works are: 408 1. Focus on IPv6 only. 410 2. The proposed solution could be driven by end-user operating 411 system's protocol stack such as TCP, UDP or other protocols, or 412 by network device working as a proxy. 414 3. Simplified signaling process with minimal information carried, 415 reduced QoS state maintenance at network devices. 417 4. Use different IPv6 options for signaling and signaling state 418 report. 420 5. Support both bandwidth reservation and latency expectation at 421 each hop. 423 6. Support dynamic resource reservation. 425 7. Support dynamic QoS forwarding state monitoring. 427 3.5. IPv6 Approach 429 IPv6 extension header is used for signaling. There are two types of 430 extension header used for the purpose of transport QoS control, one 431 is the hop-by-hop EH (HbH-EH) and another is the destination EH (Dst- 432 EH). 434 The HbH-EH may be examined and processed by the nodes that are 435 explicitly configured to do so [RFC8200], and these nodes are called 436 HbH-EH-aware nodes. Note, not all nodes along a patch need to HbH- 437 EH-aware. HbH-EH is used to carry the QoS requirement for dedicated 438 flow(s) and then the information is intercepted by HbH-EH-aware nodes 439 on the path to program hardware accordingly. 441 The destination EH will only be examined and processed by the 442 destination device that is associated with the destination IPv6 443 address in the IPv6 header. This EH is used to send the QoS related 444 report information directly to the source of the signaling at other 445 end. 447 The following figure illustrates the path setup process: 449 Setup Message (bandwidth, latency, setup state) 451 +------------------------HbH-EH------ -----------------+ 452 | | | | 453 | | | | 454 +------+ +------------+ +-----+ +------------+ +------+ 455 | | | | | | | | | | 456 | host |---| R1 |---| R2 |---| R3 |---|server| 457 | | |HbH-EH-aware| | | |HbH-EH-aware| | | 458 | | | | | | | | | | 459 +------+ +------------+ +-----+ +------------+ +------+ 460 | | 461 | | 462 +----------------------- Dst-EH ------------------------+ 464 Setup State Report Message (setup state report) 466 Figure 2: Path Setup 468 Using the figure.2 for illustration, to set up a path with resource 469 reservation, a setup message including QoS requirements, such as max/ 470 min bandwidth, burst size, the latency, and the setup state is sent 471 from the host to the server. After each HbH-EH-aware node along the 472 path receives the message, it reads the QoS information and programs 473 the hardware for resource reservation, queuing management etc. The 474 setup state object is updated at each HbH-EH-aware node to include 475 the QoS programming and provisioning result and the necessary 476 hardware reference information for IP forwarding with QoS. After the 477 setup message reaches the server, the server will send a setup state 478 report message encoded as Dst-EH to the host. The setup state report 479 message carries the path setup results from the setup state object. 481 4. Key Messages and Parameters 483 4.1. Setup and Setup State Report messages 485 Setup message is intended to program the hardware for QoS channel on 486 the IP path from the source to the destination expressed in IPv6 487 header. It is embedded as the HbH-EH in an IPv6 packet and will be 488 processed at each HbH-EH-aware node. For the simplicity, performance 489 and scalability purpose, not all routers along the path need do the 490 processing or be HbH-EH-aware. For different QoS requirements and 491 scenarios, different criteria can be used to configure HbH-EH-aware 492 nodes. 494 A throttle router is the device that an interested TCP/UDP session 495 cannot get the enough bandwidth to support its application, and it 496 will also contribute more to latency than non-throttle routers. The 497 regular throttle routers include the BRAS (broadband remote access 498 server) in broadband access network, the PGW (PDN Gateway) in LTE 499 network etc. In more general case, any routers which aggregated 500 traffic may become as a throttle router. Throttle routers should be 501 configured to process HbH-EH when: 503 o Reserved bandwidth is required: The throttle router is the 504 critical point to be configured to process the hop-by-hop EH for 505 the bandwidth reservation. Moreover, the direction of congestion 506 must be considered. 508 o Bounded latency is required: In theory, each router and switch 509 could contribute some delay to the end-to-end latency, but the 510 throttle router will contribute more than non-throttle routers, 511 and slow device will contribute more than fast device. We can use 512 OAM to detect the latency contribution in a network, and configure 513 those worst-cast devices to process the HbH-EH. 515 Setup State Report message is the message sent from the destination 516 host to the source host (from the point of view of the Setup 517 message). The message is embedded into the Dst-EH in any data 518 packet. The Setup State Report in the message is just a copy from 519 the Setup message received at the destination host for a typical TCP 520 session. The message is used at the source host to forward the 521 packet later and to do the congestion control. 523 ::= [ ] 525 [ ] [ ] 527 [ ] [ ] 529 ::= 531 [ ] 533 4.2. Forwarding State and Forwarding State Report messages 535 After the QoS is programmed by the in-band signaling, the specified 536 IP flows can be processed and forwarded for the QoS requirement. 537 There are two ways for host to use the QoS channel for associated TCP 538 session: 540 1. Host directly send the IP packet without any changes to the 541 packet, this is for the following cases: 543 * The hardware was programmed to use the tuples in IP header as 544 identification for QoS process (SIS = 0), and 546 * The packet does not function to collect the QoS forwarding 547 state on the path. 549 2. Host add the Forward State message into a data packet's IP header 550 as HbH-EH and send the packet, this is for the cases: 552 * The hardware was programmed to use the Service ID as 553 identification for QoS process (SIS != 0). 555 * The hardware was programmed to use the tuples in IP header as 556 identification for QoS process (SIS = 0), and the data packet 557 functions to collect the QoS forwarding state on the path. 558 This is the situation that host wants to detect the QoS 559 forwarding state for the purpose of failure handling (See 560 section 4.3). 562 Forwarding State message format is shown in the Section 6.7. It is 563 used to notify the service ID and also update QoS forwarding state 564 for the hops that are HbH-EH-aware nodes. 566 After Forwarding State message is reaching the destination host, the 567 host is supposed to retrieve it and form a Forwarding State Report 568 message, and carry it in any data packet as the Dst-EH, then send it 569 to the host in the reverse direction. 571 ::= [ 572 ] 573 [ ] [ ] 575 ::= 577 [ ] 579 4.3. Hop Number 581 This is the parameter for total number of HbH-EH-aware nodes on the 582 path. It is the field "Hop_num" in Setup message, and is used to 583 locate the bit position for "Setup State" and the "Service ID" in 584 "Service ID List". The value of "Hop_num" must be decremented at 585 each HbH-EH-aware node. At the receiving host of the in-band 586 signaling, the Hop_num must be zero. 588 The source host must know the exact hop number, and setup the initial 589 value in the Setup message. The exact hop number can be detected 590 using OAM message. 592 4.4. Flow Identifying Method and Service ID 594 A QoS channel might be enforced for a group of flows or a delicate 595 flow, and flow identifying method means the way of identifying a flow 596 or a group of flows that can use a HW programmed QoS channel. 597 Different levels of flow granularities to support QoS are defined as 598 below: 600 Flow level 601 The flow identification could be 5 tuples for non IPSec IPv6 602 packet: the source, destination IP address, protocol number, 603 source and destination port number, and also could be 3 tuples for 604 IPSec IPv6 packet: the source, destination IP address and the flow 605 label. 607 Address level In-band Signaling 608 A flow of packets share the same source, destination IP address, 609 but with different protocol number. This is the scenario that the 610 signaling is for the aggregated flows which have the same source, 611 destination address. i.e, All TCP/UDP flows between the same 612 client and same server (only one address for client and one for 613 server) 615 Transport level In-band Signaling 616 Packets share the same source, destination IP address, protocol 617 number, but with different source or destination port number (non- 618 IPSec) or different flow label (IPSec). This could be for the 619 aggregated TCP or UDP flows that started and terminated at the 620 same IP addresses. 622 DiffServ level In-band Signaling 623 Packets share the same DSCP value. This means aggregated 624 differentiated service flows that have the same DSCP value. The 625 DSCP value is determined by the 6 most-significant bits in 8-bits 626 DiffServ field for IPv4 or 8-bits Traffic Class field for IPv6. 628 There are two ways for flow identifying. One is by tuple or DSCP 629 value in IP header, another is by a local significant number, called 630 service ID, generated and maintained in a router. When "Service ID 631 Size" (SIS) is zero, it means the "Flow identification method" (FI) 632 is used for both control plane and data plane. When "SIS" is not 633 zero, it means "FI" is only used in signaling of setting up the QoS 634 channel, and the data plane will only use the "Service ID". The use 635 of local generated number to identify flow is to speed up the flow 636 lookup and QoS process for data plane. 638 The "Service ID List" is a list of "Service ID" for all hops that are 639 HbH-EH-aware nodes on the IP path. When a router receives a HbH-EH, 640 it may generate a service ID for the flow(s) that is defined by the 641 Flow Identifying Method in "FI". Then the router must attach the 642 service ID value to the end of the Service ID List. After the packet 643 reaches the destination host, the Service ID List will be that the 644 1st router's service ID as the list header, and the last router's 645 service ID as the list tail. 647 4.5. QoS State and life of Time 649 After a router is programmed for a QoS, a QoS state is created. The 650 QoS state life is determined by the "Time" in the Setup message. 651 Whenever there is a packet processed by a QoS state, the associated 652 timer for the QoS state is reset. If the timer of a QoS state is 653 expired, the QoS state will be erased and the associated resource 654 will be released. 656 In order to keep the QoS state active, a application at source host 657 can send some zero size of data to refresh the QoS state. 659 When the Time is set to zero, it means the life of the QoS State will 660 be kept until the de-programming message is received. 662 4.6. Authentication 664 The in-band signaling is designed to have a basic security mechanism 665 to protect the integrity of a signaling message. The Authentication 666 message is to attach to a signaling message, the source host 667 calculates the harsh value of a key and all invariable part of a 668 signaling message (Setup message: ver, FI, R, SIS, P, Time; Bandwidth 669 message, Latency message, Burst message). The key is only known to 670 the hosts and all HbH-EH-aware nodes. The securely distribution of 671 the key is out the scope of the document. 673 5. Packet Forwarding 675 To achieve the required QoS, after the path setup with guaranteed 676 bandwidth there are some requirements to be met during data 677 forwarding. These include the hardware capability, the scheme for 678 the data forwarding, QoS processing, state report, etc. 680 5.1. Basic Hardware Capability 682 Section 4 explains how QoS guaranteed path can be set up and the 683 corresponding messages used, however different implementations may 684 vary in details. To achieve the satisfactory targets for performance 685 and scalability, the protocol must be cooperated with capable 686 hardware to provide the desired fine-grained QoS for different 687 transport. 689 In our experiment to implement the feature for TCP, we used a network 690 processor with traffic management feature. The traffic management 691 can provide the fine-grained QoS for any configured flow(s). 693 The following capabilities are RECOMMENDED: 695 1. The in-banding signaling is processed in network processor 696 without punting to controller CPU for help 698 2. The QoS forwarding state is kept and maintained in network 699 processor without the involvement from controller CPU. 701 3. The QoS state has a life of a pre-configured time and will be 702 automatically deleted if there is no data packet processed by 703 that QoS state. The timer can be changed on the fly. 705 4. The data forwarding does not need to be done at the controller 706 CPU, or so called slow path. It is at the same hardware as the 707 normal IP forwarding. For any IP packet, the QoS forwarding is 708 executed first. Normal forwarding will be executed if there is 709 no QoS state associated with the identification of the flow. 711 5. The QoS forwarding and normal forwarding can be switched on the 712 fly. 714 The details of data plane and hardware related implementations, such 715 as traffic classification, shaping, queuing and scheduling, are out 716 of scope of this document. The report of [NGP] has given some 717 experiments and results by using commercial hardwares. 719 5.2. Flow Identification in Packet Forwarding 721 Flow identification in Packet Forwarding is same as the QoS channel 722 establishment by Setup message. It is to forward a packet with a 723 specified QoS process if the packet is identified to be belonging to 724 specified flow(s). 726 There are two method used in data forwarding to identify flows: 728 1. Hardware was programmed to use tuples in IP header implicitly. 729 This is indicated by that the "SIS" is zero or the Service ID is 730 not used. When a packet is received, its tuples are looked up 731 according to the value of "FI". If there is a QoS table has 732 match for the packet, the packet will be processed by the QoS 733 state found in the QoS table. This method does not need any EH 734 added into the data packet unless the data packet function to 735 collect the QoS forwarding state on the path. 737 2. Hardware was programmed to use service ID to identify flows. 738 This is indicated by that the "SIS" is not zero. When a packet 739 is received, the service ID associated with the hop is retrieved 740 and looked up for the QoS table. If it has match for the packet, 741 the packet will be processed by the QoS state entry found in the 742 QoS table. 744 5.3. QoS Forwarding State Detection and Failure Handling 746 QoS forwarding may fail due to different reasons: 748 1. Hardware failure in HbH-EH-aware node. 750 2. IP path change due to link failure, node failure or routing 751 changes; And the IP path change has impact to the HbH-EH-aware 752 node. 754 3. Network topology change; and the change leads to the changes of 755 HbH-EH-aware nodes. 757 Application may need to be aware of the service status of QoS 758 guarantee when the application is using a TCP session with QoS. In 759 order to provide such feature, the TCP stack in the source host can 760 detect the QoS forwarding state by sending TCP data packet with 761 Forwarding State message coded as HbH-EH. After the TCP data packet 762 reaches the destination host, the host will copy the forwarding state 763 into a Forwarding State Report message, and send it with another TCP 764 packet (for example, TCP-ACK) in reverse direction to the source 765 host. Thereafter, the source host can obtain the QoS forwarding 766 state on all HbH-EH-aware nodes. 768 A host can do the QoS forwarding state detection by three ways: on 769 demand, periodically or constantly. 771 After a host detects that there is QoS forwarding state failure, it 772 can repair such failure by sending another Setup message embedded 773 into a HbH-EH of any TCP packet. This repairing can handle all 774 failure case mentioned above. 776 If a failure cannot be repaired, host will be notified, and 777 appropriate action can be taken, see section 7.1 779 6. Details of Working with Transport Layer 781 The proposed new IP service is transport agnostic, which means any 782 transport layer protocol can use it. 784 6.1. Working with TCP 786 Considering TCP as the most widely used transport layer protocol, 787 this document uses TCP as an example of transport protocol to show 788 how it works with the proposed IP service. 790 The following is the list of messages for signaling and associated 791 data forwarding. 793 o Setup: This is for the setup of QoS channel through the IP path. 795 o Bandwidth: This is the required bandwidth for the QoS channel. It 796 has minimum (CIR) and maximum bandwidth (PIR). 798 o Latency: This is the required latency for the QoS channel, it is 799 the bounded latency for each hop on the path. This is not the end 800 to end latency. 802 o Burst: This is the required burst for the QoS channel, it is the 803 maximum burst size. 805 o Authentication: This is the security message for a in-band 806 signaling. 808 o OAM: This is the Operation and Management message for the QoS 809 channel. 811 o Setup State Report: This is the state report of a setup message. 813 o Forwarding State: This is the forwarding state message used for 814 data packet. 816 o Forwarding State Report: This is the forwarding state report of a 817 QoS channel. 819 There are three scenarios of QoS signaling for TCP session setup with 820 QoS 822 1. Upstream: This is for the direction of client to server. A 823 application decides to open a TCP session with upstream QoS (for 824 uploading), it will call TCP API to open a socket and connect to 825 a server. The client host will form a TCP SYN packet with the 826 HbH-EH in the IPv6 header. The EH includes Setup message and 827 Bandwidth message, and optionally Latency, Burst, Authentication 828 and OAM messages. The packet is forwarded at each hop. Each 829 HbH-EH-aware nodes will process the signaling message to finish 830 the following tasks before forwarding the packet to next hop: 832 * Retrieve the QoS parameters to program the Hardware, it 833 includes: FL, Time, Bandwidth, Latency, Burst 835 * Update the field in the EH, it includes: Hop_number, 836 Total_latency, and possibly Service ID List 838 When the server receives the TCP SYN, the Host kernel will also 839 check the HbH-EH while punting the TCP packet to the TCP stack 840 for processing. If the HbH-EH is present and the Report bit is 841 set, the Host kernel must form a new Setup State Report message, 842 all fields in the message must be copied from the Setup message 843 in the HbH-EH. When the TCP stack is sending the TCP-SYNACK to 844 the client, the kernel must add the Setup State Report message as 845 a Dst-EH in the IPv6 header. After this, the IPv6 packet is 846 complete and can be sent to wire; When the client receives the 847 TCP-SYNACK, the Host kernel will check the Dst-EH while punting 848 the TCP packet to the TCP stack for processing. If the Dst-EH is 849 present and the Setup State Report message is valid, the kernel 850 must read the Setup State Report message. Depending on the setup 851 state, the client will operate according to description in 852 section 7.1 854 2. Downstream: This is for the direction of server to client. A 855 application decides to open a TCP session with downstream QoS 856 (for downloading), it will call TCP API to open a socket and 857 connect to a server. The client host will form a TCP SYN packet 858 with the Dst-EH in the IPv6 header. The EH includes Bandwidth 859 message, and optionally Latency, Burst messages. The packet is 860 forwarded at each hop. Each hop will not process the Dst-EH. 861 When the server receives the TCP SYN, the Host kernel will check 862 the Dst-EH while punting the TCP packet to the TCP stack for 863 processing. If the Dst-EH is present, the Host kernel will 864 retrieve the QoS requirement information from Bandwidth, Latency 865 and Burst message, and check the QoS policy for the user. If the 866 user is allowed to get the service with the expected QoS, the 867 server will form a Setup message similar to the case of client to 868 server, and add it as the HbH-EH in the IPv6 header, and send the 869 TCP-SYNACK to client. Each HbH-EH-aware nodes on the path from 870 server to client will process the message similar to the case of 871 client to server. After the client receives the TCP-SYNACK, The 872 client will send the Setup State Report message to server as the 873 Dst-EH in the TCP-ACK. Finally the server receives the TC-ACK 874 and Setup State Report message, it can send the data to the 875 established session according to the pre-negotiated QoS 876 requirements. 878 3. Bi-direction: This is the case that the client wants to setup a 879 session with bi-direction QoS guarantee. The detailed operations 880 are actually a combination of Upstream and Downstream described 881 above. 883 After a QoS channel is setup, the in-band signaling message can still 884 be exchanged between two hosts, there are two scenarios for this. 886 1. Modify QoS on the fly: When the pre-set QoS parameters need to be 887 adjusted, the application at source host can re-send a new in- 888 band signaling message, the message can be embedded into any TCP 889 packet as a IPv6 HbH-EH. The QoS modification should not impact 890 the established TCP session and programmed QoS service. Thus, 891 there is no service impacted during the QoS modification. 892 Depending on the hardware performance, the signaling message can 893 be sent with TCP packet with different data size. If the 894 performance is high, the signaling message can be sent with any 895 TCP packet; otherwise, the signaling message should be sent with 896 small size TCP packet or zero-size TCP packet (such as TCP ACK). 897 Modification of QoS on the fly is a very critical feature for the 898 so called "Application adaptive QoS transport service". With 899 this service, an application (or the proxy from a service 900 provider) could setup an optimized CIR for different stage of 901 application for the economical and efficient purpose. For 902 example, in the transport of compressed video, the I-frame has 903 big size and cannot be lost, but P-frame and B-frame both have 904 smaller size and can tolerate some loss. There are much more 905 P-frame and B-frame than I-frame in videos with smooth changes 906 and variations in images [I-D.han-iccrg-arvr-transport-problem]. 908 Based on this characteristics, application can request a 909 relatively small CIR for the time of P-frame and P-frame, and 910 request a big CIR for the time of I-frame. 912 2. Repairing of the QoS channel: This is the case the QoS channel 913 was broken and need to be repaired, see section 5.3. 915 6.2. Working with UDP and other Protocols 917 There are other transport layer protocols, such as UDP, QUICK and 918 SCTP, and for these protocols similar strategy as TCP can be applied. 919 The to establish a closed-loop for the transport control. 921 For protocols with natively bi-directional control mechanism such as 922 SCTP, only some QoS control functionalities for the protocol need to 923 be added. The mechanism for TCP can be borrowed for such job. There 924 will be the QoS setup for one directional data stream, and QoS setup 925 state report for another directional data stream. The protocol may 926 also have functionalities in the stack to handle the adjustment of 927 the behaviour for different QoS setup and setup states. 929 For protocols that natively lack the feed-back control mechanism to 930 form a closed-loop such as UDP, this mechanism needs to be added into 931 the streams. There are two options to realize this: 933 1. Modify the protocol itself to have some state machine to 934 establish the closed-loop for the protocol. This can be done in 935 the kernel of the OS by modifying the protocol stack. 937 2. Modify the user data stream to introduce the closed-loop scheme, 938 this becomes as application work. It is up to application to add 939 or modify codes for the state machine of the closed-loop control. 941 7. Additional Considerations 943 This document only covers the details of setting up a path with QoS 944 using IPv6, and TCP is used as an example of transport layer protocol 945 to achieve flow level service. Only basic scenarios are covered, and 946 there are lots of open issues to be researched. The following is a 947 non-comprehensive list, and they can be addressed in separate drafts. 949 7.1. User and Application driven 951 The QoS transport service is initiated and controlled by end user's 952 application. Following tasks are done in host: 954 1. The detailed QoS parameters in signaling message are set by end 955 user application. New socket option must be added, the option is 956 a place holder for QoS parameters (Setup, Bandwidth, etc.), Setup 957 State Report and Forwarding State Report messages. 959 2. The Setup State Report and Forwarding State Report message 960 received at host are processed by transport service in kernel. 961 The Setup State Report message processed at host can result in 962 the notification to the application whether the setup is 963 successful. If the setup is successful, the application can 964 start to use the socket having the QoS support; If the setup is 965 failed, the application may have three choices: 967 * Lower the QoS requirement and re-setup a new QoS channel with 968 new in-band signaling message. 970 * Use the TCP session as traditional transport without any QoS 971 support. 973 * Lookup the service provider for help to locate the problem in 974 network. 976 7.2. Traffic Management in Host 978 In order to better accommodate this new IP in-band service, the OS on 979 a host may be changed in traffic management related areas. There are 980 two parts for traffic management to be changed: one is to manage 981 traffic going out a host's shared links, and the other is congestion 982 control for TCP flows. 984 1. For current traffic management in a host, all TCP/UDP sessions 985 will share the bandwidth for all egress links. For the purpose 986 to work with the differentiated service provided by under layer 987 network in bandwidth and latency, the kernel may allocate 988 expected resource to applications that are using the QoS 989 transport service. For example, kernel can queue different 990 packets from different applications or users to different queue 991 and schedule them in different priority. Only after this change, 992 some application can use more bandwidth and get less queuing 993 delay for a link than others. 995 2. The congestion control in a host manages the behavior of TCP 996 flow(s). This includes important features like slow start, AIMD, 997 fast retransmit, selective ACK, etc. To accommodate the benefit 998 of the QoS guaranteed transport service, the congestion control 999 can be much simpler [I-D.han-tsvwg-cc]. The new congestion 1000 control is related to the implementation of QoS guarantee. 1001 Following is a simple congestion control algorithm assuming that 1002 the CIR is guaranteed and PIR is shared between flows: 1004 * There is no slow start, the TCP can start sending traffic at 1005 the rate of CIR. 1007 * The AIMD is kept, but the range of the sawtooth pattern should 1008 be maintained between CIR and PIR. 1010 * Other congestion control features can be kept. 1012 7.3. Heterogeneous Network 1014 When an IP network is connected with a non-IP network, such as MPLS 1015 or Ethernet network, the in-band signaling should also work in that 1016 network to achieve an end-to-end connection. The behavior, protocol 1017 and rules in the interworking with non-IP network is out of the scope 1018 of this draft, and further research needs to be done to solve the 1019 problem. 1021 7.4. Proxy Control 1023 It is expected that in a real service provider network, the in-band 1024 signaling will be checked, filtered and managed at proxy routers. It 1025 serves the following purposes: 1027 1. A proxy can check if the in-band signaling from an end user meets 1028 the SLA compliance. This adds extra security and DOS attack 1029 prevention. 1031 2. A proxy can collect the statistics for user's TCP flows and check 1032 the in-band signaling for accounting and charging. 1034 3. A proxy can insert and process appropriate in-band signaling for 1035 TCP flows if the host does not support this new feature. This 1036 can provide backward compatibility, also enable the host to use 1037 the new feature. 1039 8. IANA Considerations 1041 This document defines a new option type for the Hop-by-Hop Options 1042 header and the Destination Options header. According to [RFC8200], 1043 the detailed value are: 1045 +-----------+----------------+---------------------+---------------+ 1046 | | Binary Value | | | 1047 | Hex Value +----+---+-------+ Description | Reference | 1048 | | act|chg| rest | | | 1049 +-----------+----+---+-------+---------------------+---------------+ 1050 | 0x0 | 00 | 0 | 10000 | In-band Signaling | Section 4 | 1051 | | | | | | in this doc | 1052 +-----------+----+---+-------+---------------------+---------------+ 1054 Figure 3: The New Option Type 1056 1. The highest-order 2 bits: 00, indicating if the processing IPv6 1057 node does not recognize the Option type, skip over this option and 1058 continue processing the header. 1060 2. The third-highest-order bit: 0, indicating the Option Data does 1061 not change en route. 1063 3. The low-order 5 bits: 10000, assigned by IANA. 1065 This document also defines a 4-bit subtype field, for which IANA will 1066 create and will maintain a new sub-registry entitled "In-band 1067 signaling Subtypes" under the "Internet Protocol Version 6 (IPv6) 1068 Parameters" [IPv6_Parameters] registry. Initial values for the 1069 subtype registry are given below 1070 +-------+------------+-----------------------------+---------------+ 1071 | Type | Mnemonic | Description | Reference | 1072 +-------+------------+-----------------------------+---------------+ 1073 | 0 | SETUP | Setup object | Appendix B | 1074 +-------+------------+-----------------------------+---------------+ 1075 | 1 | BANDWIDTH | Bandwidth object | Appendix B | 1076 +-------+------------+-----------------------------+---------------+ 1077 | 2 | BURST | Burst object | Appendix B | 1078 +-------+------------+-----------------------------+---------------+ 1079 | 3 | LATENCY | Latency object | Appendix B | 1080 +-------+------------+-----------------------------+---------------+ 1081 | 4 | AUTH | Authentication object | Appendix B | 1082 +-------+------------+-----------------------------+---------------+ 1083 | 5 | OAM | OAM object | Appendix B | 1084 +-------+------------+-----------------------------+---------------+ 1085 | 6 | FWD STATE | Forward state | Appendix B | 1086 +-------+------------+-----------------------------+---------------+ 1087 | 7 |SETUP REPORT| Setup state report | Appendix B | 1088 +-------+------------+-----------------------------+---------------+ 1089 | 8 | FWD REPORT | Forwarding state report | Appendix B | 1090 +-------+------------+-----------------------------+---------------+ 1092 Figure 4: The In-band Signaling Sub Type 1094 9. Security Considerations 1096 It is important to guarantee that the resource reservation is used by 1097 authenticated users, and false signaling should not be accepted or 1098 processed. The following aspects may be considered: 1100 Authentication of user 1102 If an user is interested in using this new service, the user 1103 should sign up to a service provider. Service provider should 1104 do the proper authentication check for a new user, and 1105 establish account for the user. 1107 After the sign up, a user should provide a security key to the 1108 service provider through a secured channel (https, registered 1109 mail, etc.), or the key could be generated and given to user by 1110 the service provider. Service provider should distribute the 1111 security key of the user to different network device. More 1112 specifically, the security key should be distributed securely 1113 to all HbH-EH-aware nodes for an open network, or the proxy for 1114 a closed network. 1116 Proxy 1117 Proxy or gateway is the 1st network device connecting to 1118 customer's devices (Host, phone, etc.) that can generate the 1119 signaling for resource reservation. The functionality of the 1120 Proxy is to check if the signaling is allowed to go through 1121 SP's network. This can be done by checking the signaling 1122 integrity and other info associated with the user, such as the 1123 source/destination IP address, the account balance, the user's 1124 privilege, etc. 1126 Authentication of signaling message 1128 The signaling for resource reservation should be checked at 1129 each HbH-EH-aware nodes or a proxy node. 1131 Service ID is originally used for performance improvement of 1132 forwarding with QoS, and it can also provide additional security 1133 protection of forwarding resource in data plane. Service ID in each 1134 HbH-EH-aware node is to represent an IP flow with programmed QoS 1135 service, and it is a local significant number generated by a router 1136 to identify a flow that was offered QoS service. So, the router can 1137 periodically change the number for the same flow to protect any 1138 middle box sniffing for DOS attacking. It can be done by host 1139 periodical send out in-band signaling with the same QoS parameters 1140 and obtain the new Service ID and Service ID List for the use of next 1141 data forwarding. 1143 10. References 1145 10.1. Normative References 1147 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1148 Requirement Levels", BCP 14, RFC 2119, 1149 DOI 10.17487/RFC2119, March 1997, 1150 . 1152 [RFC2581] Allman, M., Paxson, V., and W. Stevens, "TCP Congestion 1153 Control", RFC 2581, DOI 10.17487/RFC2581, April 1999, 1154 . 1156 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 1157 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 1158 May 2017, . 1160 [RFC8200] Deering, S. and R. Hinden, "Internet Protocol, Version 6 1161 (IPv6) Specification", STD 86, RFC 8200, 1162 DOI 10.17487/RFC8200, July 2017, 1163 . 1165 10.2. Informative References 1167 [eNodeB] wikipedia, "eNodeB", 2018, 1168 . 1170 [EPC] 3GPP, "The Evolved Packet Core", 2018, 1171 . 1174 [HU5G] Huawei, "5G Vision: 100 Billion connections, 1 ms Latency, 1175 and 10 Gbps Throughput", 2015, 1176 . 1178 [I-D.falk-xcp-spec] 1179 Falk, A., "Specification for the Explicit Control Protocol 1180 (XCP)", draft-falk-xcp-spec-03 (work in progress), July 1181 2007. 1183 [I-D.han-iccrg-arvr-transport-problem] 1184 Han, L. and K. Smith, "Problem Statement: Transport 1185 Support for Augmented and Virtual Reality Applications", 1186 draft-han-iccrg-arvr-transport-problem-01 (work in 1187 progress), March 2017. 1189 [I-D.han-tsvwg-cc] 1190 Han, L., Qu, Y., and T. Nadeau, "A New Congestion Control 1191 in Bandwidth Guaranteed Network", draft-han-tsvwg-cc-00 1192 (work in progress), March 2018. 1194 [I-D.harper-inband-signalling-requirements] 1195 Harper, J., "Requirements for In-Band QoS Signalling", 1196 draft-harper-inband-signalling-requirements-00 (work in 1197 progress), January 2007. 1199 [I-D.ietf-aqm-codel] 1200 Nichols, K., Jacobson, V., McGregor, A., and J. Iyengar, 1201 "Controlled Delay Active Queue Management", draft-ietf- 1202 aqm-codel-06 (work in progress), December 2016. 1204 [I-D.ietf-aqm-fq-codel] 1205 Hoeiland-Joergensen, T., McKenney, P., 1206 dave.taht@gmail.com, d., Gettys, J., and E. Dumazet, "The 1207 FlowQueue-CoDel Packet Scheduler and Active Queue 1208 Management Algorithm", draft-ietf-aqm-fq-codel-06 (work in 1209 progress), March 2016. 1211 [I-D.ietf-aqm-pie] 1212 Pan, R., Natarajan, P., Baker, F., and G. White, "PIE: A 1213 Lightweight Control Scheme To Address the Bufferbloat 1214 Problem", draft-ietf-aqm-pie-10 (work in progress), 1215 September 2016. 1217 [I-D.ietf-detnet-use-cases] 1218 Grossman, E., "Deterministic Networking Use Cases", draft- 1219 ietf-detnet-use-cases-20 (work in progress), December 1220 2018. 1222 [I-D.ietf-spring-segment-routing] 1223 Filsfils, C., Previdi, S., Ginsberg, L., Decraene, B., 1224 Litkowski, S., and R. Shakir, "Segment Routing 1225 Architecture", draft-ietf-spring-segment-routing-15 (work 1226 in progress), January 2018. 1228 [I-D.ietf-tcpm-dctcp] 1229 Bensley, S., Eggert, L., Thaler, D., Balasubramanian, P., 1230 and G. Judd, "Datacenter TCP (DCTCP): TCP Congestion 1231 Control for Datacenters", draft-ietf-tcpm-dctcp-03 (work 1232 in progress), November 2016. 1234 [I-D.roberts-inband-qos-ipv6] 1235 Roberts, L. and J. Harford, "In-Band QoS Signaling for 1236 IPv6", draft-roberts-inband-qos-ipv6-00 (work in 1237 progress), July 2005. 1239 [I-D.sridharan-tcpm-ctcp] 1240 Sridharan, M., Tan, K., Bansal, D., and D. Thaler, 1241 "Compound TCP: A New TCP Congestion Control for High-Speed 1242 and Long Distance Networks", draft-sridharan-tcpm-ctcp-02 1243 (work in progress), November 2008. 1245 [IPv6_Parameters] 1246 IANA, "Internet Protocol Version 6 (IPv6) Parameters", 1247 2015, . 1250 [MEC] ETSI, "Multi-access Edge Computing", 2018, 1251 . 1254 [NGP] ETSI, "Next Generation Protocols (NGP); Recommendation for 1255 New Transport Technologies", 2018, 1256 . 1259 [QU2016] Qualcomm, "Leading the world to 5G", 2016, 1260 . 1263 [RFC2474] Nichols, K., Blake, S., Baker, F., and D. Black, 1264 "Definition of the Differentiated Services Field (DS 1265 Field) in the IPv4 and IPv6 Headers", RFC 2474, 1266 DOI 10.17487/RFC2474, December 1998, 1267 . 1269 [RFC2475] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z., 1270 and W. Weiss, "An Architecture for Differentiated 1271 Services", RFC 2475, DOI 10.17487/RFC2475, December 1998, 1272 . 1274 [RFC3175] Baker, F., Iturralde, C., Le Faucheur, F., and B. Davie, 1275 "Aggregation of RSVP for IPv4 and IPv6 Reservations", 1276 RFC 3175, DOI 10.17487/RFC3175, September 2001, 1277 . 1279 [Tactile] JDavid Szabo, et al. Proceedings of European Wireless 1280 2015; 21th European Wireless Conference, "Towards the 1281 Tactile Internet: Decreasing Communication Latency with 1282 Network Coding and Software Defined Networking", 2015, 1283 . 1285 [TCP-vegas] 1286 Peterson, L., "TCP Vegas: New Techniques for Congestion 1287 Detection and Avoidance - CiteSeer page on the 1994 1288 SIGCOMM paper", 1994. 1290 [TCP_Targets] 1291 Andreas Benthin, Stefan Mischke, University of Paderborn, 1292 "Bandwidth Allocation of TCP", 2004. 1294 [TIA] TIA 1039 Revision A, "QoS Signaling for IP QoS Support and 1295 Sender Authentication", 2015, . 1299 Appendix A. Acknowledgements 1301 The authors are very grateful to Fred Baker for his valuable 1302 contributions to this document. 1304 We appreciate the following people who made lots of contributions to 1305 this draft: Guoping Li, Boyan Tu, and Xuefei Tan, and thank Huawei 1306 Nanjing research team led by Feng Li to provide the Product on 1307 Concept (POC) development and test, the team members include Fengxin 1308 Sun, Xingwang Zhou, and Weiguang Wang. We also like to thank other 1309 people involved in the discussion of solution: Tao Ma from Future 1310 Network Strategy dept. 1312 Appendix B. Message Objects 1314 This section defines detailed objects used in different messages. 1316 B.1. Setup State Object 1317 0 1 2 3 1318 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1319 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1320 |0 0 0 0|ver| FI|R|SIS|P| Time | Hop_num |u| Total_latency | 1321 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1322 | State for each hop index | 1323 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1324 | Service ID list for hops | 1325 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-. . . + 1327 Type = 0, Setup state; 1329 Version: The version of the protocol for the QoS 1331 FI: Flow identification method, 1333 0: 5 tuples; 1: src,dst,port; 2: src,dst; 3: DSCP 1335 R: If the destination host report the received Setup state to 1337 the src address by Destination EH. 0: dont report; 1: report 1339 SIS: Service ID size; 0: 0bits, 1: 16bits, 2: 20bits, 3: 32bits 1341 P: 0: program HW for the QoS from src to dst; 1343 1: De-program HW for the QoS from src to dst 1345 Time: The life time of QoS forwarding state in second. 1347 Hop_num: The total hop number on the path set by host. It must be 1349 decremented at each hop after the processing. 1351 u: the unit of latency, 0: ms; 1: us 1353 Total_latency : Latency accumulated from each hop, each hop will 1355 add the latency in the device to this value. 1357 Figure 5: The Setup State Object 1359 Setup state for each hop index: each bit is the setup state on each 1360 hop on the path, 0: failed; 1: success. The 1st hop is at the most 1361 significant bit. 1363 Service ID list for hops: it is for all hops on the path, each 1364 service ID bit size is defined in SIS. The 1st service ID is at the 1365 top of the stack. Each hop add its service ID at the correct 1366 position indexed by the current hop number for the router. 1368 The Setup object is embedded into the hop-by-hop EH to setup the QoS 1369 in the device on the IP forwarding path. To keep the whole setup 1370 message size unchanged at each hop, the total hop number must be 1371 known at the source host. The total hop number can be detected by 1372 OAM. The service ID list is empty before the 1st hop receives the 1373 in-band signaling. Each hop then fill up the associated service ID 1374 into the correct place determined by the index of the hop. 1376 B.2. Bandwidth Object 1378 0 1 2 3 1379 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1380 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1381 |0 0 0 1| reserved | Minimum bandwidth | 1382 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1383 | Maximum bandwidth | 1384 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1386 Type = 1, 1388 Minimum bandwidth : The minimum bandwidth required, or CIR, unit 1389 Mbps 1391 Maximum bandwidth : The maximum bandwidth required, or PIR, unit 1392 Mbps 1394 Figure 6: The Bandwidth Object 1396 B.3. Burst Msg 1397 0 1 2 3 1398 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1399 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1400 |0 0 1 0| Burst size | 1401 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1403 Type = 2, 1405 Burst size : The burst size, unit M bytes 1407 Figure 7: The burst message 1409 B.4. Latency Object 1411 0 1 2 3 1412 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1413 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1414 |0 0 1 1|u| Latency | 1415 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1417 Type = 3, 1419 u: the unit of the latency 1421 0: ms; 1: us 1423 Latency: Expected maximum latency for each hop 1425 Figure 8: The Latency Object 1427 B.5. Authentication Object 1428 0 1 2 3 1429 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1430 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1431 |0 1 0 0| MAC_ALG | res | MAC data (variable length) | 1432 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+. . .+ 1434 Type = 4, 1436 MAC_ALG: Message Authentication Algorithm 1438 0: MD5; 1:SHA-0; 2: SHA-1; 3: SHA-256; 4: SHA-512 1440 MAC data: Message Authentication Data; 1442 Res: Reserved bits 1444 Size of signaling data (opt_len): Size of MAC data + 2 1446 MD5: 18; SHA-0: 22; SHA-1: 22; SHA-256: 34; SHA-512: 66 1448 Figure 9: The Authentication Object 1450 B.6. OAM Object 1452 0 1 2 3 1453 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1454 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1455 |0 1 0 1| OAM_t | OAM_len | OAM data (variable length) | 1456 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+. . .+ 1458 Type = 5, 1460 OAM_t : OAM type 1462 OAM_len : 8-bit unsigned integer. Length of the OAM data, in octets; 1464 OAM data: OAM data, details of OAM data are TBD. 1466 Figure 10: The OAM Object 1468 B.7. Forwarding State Object 1470 0 1 2 3 1471 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1472 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1473 |0 1 1 0|ver| FI|R|SIS|P| Time | Hop_num |u| Total_latency | 1474 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1475 | Forwarding state for each hop index | 1476 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1477 | Service ID list for hops | 1478 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-. . . + 1480 Type = 6, Forwarding state; 1482 All parameter definitions and process in the 1st row are same in 1484 the setup message. 1486 Forward state for each hop index : each bit is the fwd state on 1487 each 1489 hop on the path, 0: failed; 1: success; The 1st hop is at the 1491 most significant bit. 1493 Service ID list for hops: it is for all hops on the path, each index 1494 bit 1496 size is defined in SIS. The list is from the setup report message. 1498 Figure 11: The Forwarding State Object 1500 B.8. Setup State Report Object 1501 0 1 2 3 1502 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1503 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1504 |0 1 1 1|ver|H|u| Total_latency | Reserved | 1505 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1506 | State for each hop index | 1507 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1508 | Service ID list for hops | 1509 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-. . . + 1511 Type = 7, Setup state report; 1513 H: Hop number bit. When a host receives a setup message and form 1515 a setup report message, it must check if the Hop_num in setup 1517 message is zero. If it is zero, the H bit is set to one, and if 1519 it is not zero, the H bit is clear. This will notify the source 1521 of setup message that if the original Hop_num was correct. 1523 Following are directly copied from the setup message: 1525 u, Total_latency; 1527 State for each hop index 1529 Service ID list for hops. 1531 Figure 12: The Setup State Report Object 1533 B.9. Forward State Report Object 1534 0 1 2 3 1535 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1536 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1537 |1 0 0 0|ver|H|u| Total_latency | Reserved | 1538 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1539 | Forwarding state for each hop index | 1540 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1542 Type = 8, Forwarding state report; 1544 H: Hop number bit. When a host receives a Forward State message 1546 and form a Forward State Report message, it must check if the 1548 Hop_num in Forward State message is zero. If it is zero, the H bit 1550 is set to one, and if it is not zero, the H bit is clear. 1552 This will notify the source of Forward State message that if the 1554 original Hop_num was set correct. 1556 Following are directly copied from the Forward State message: 1558 u, Total_latency; 1560 Forwarding State for each hop index 1562 Figure 13: The Fwd State Report Object 1564 Authors' Addresses 1566 Lin Han 1567 Futurewei Technologies 1568 2330 Central Expressway 1569 Santa Clara, CA 95050 1570 USA 1572 Phone: +10 408 330 4613 1573 Email: lin.han@futurewei.com 1574 Yingzhen Qu 1575 Futurewei Technologies 1576 2330 Central Expressway 1577 Santa Clara, CA 95050 1578 USA 1580 Email: yingzhen.qu@futurewei.com 1582 Lijun Dong 1583 Futurewei Technologies 1584 2330 Central Expressway 1585 Santa Clara, CA 95050 1586 USA 1588 Email: lijun.dong@futurewei.com 1590 Richard Li 1591 Futurewei Technologies 1592 2330 Central Expressway 1593 Santa Clara, CA 95050 1594 USA 1596 Email: renwei.li@futurewei.com 1598 Thomas Nadeau 1599 Lucid Vision 1600 Hampton NH 03842 1601 USA 1603 Email: tnadeau@lucidvision.com 1605 Kevin Smith 1606 Vodafone 1607 UK 1609 Email: Kevin.Smith@vodafone.com 1611 Jeff Tantsura 1612 Apstra 1614 Email: jefftant.ietf@gmail.com