idnits 2.17.1 draft-bryant-detnet-mpls-dp-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (March 05, 2018) is 2237 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'IEEE8021Q' is mentioned on line 973, but not defined == Missing Reference: 'IEEE8021CB' is mentioned on line 990, but not defined == Outdated reference: A later version (-02) exists of draft-bryant-rtgwg-enhanced-vpn-01 == Outdated reference: A later version (-13) exists of draft-ietf-detnet-architecture-04 == Outdated reference: A later version (-04) exists of draft-ietf-detnet-dp-sol-01 == Outdated reference: A later version (-22) exists of draft-ietf-spring-segment-routing-mpls-12 Summary: 0 errors (**), 0 flaws (~~), 7 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Detnet Working Group S. Bryant 3 Internet-Draft M. Chen 4 Intended status: Standards Track Huawei Technologies 5 Expires: September 6, 2018 March 05, 2018 7 Operation of Deterministic Networks over MPLS 8 draft-bryant-detnet-mpls-dp-00 10 Abstract 12 This document specifies Deterministic Networking data plane operation 13 over an MPLS Packet Switched Networks. 15 This document is a derivative work from draft-ietf-detnet-dp-sol-01. 17 Whether this is published as a stand-alone text, or serves as a focal 18 point to refine the MPLS design and the key point are subsequently 19 re-merged with draft-ietf-detnet-dp-sol-01 is a matter for the DETNET 20 WG, as is the editorship of any WG text. 22 Status of This Memo 24 This Internet-Draft is submitted in full conformance with the 25 provisions of BCP 78 and BCP 79. 27 Internet-Drafts are working documents of the Internet Engineering 28 Task Force (IETF). Note that other groups may also distribute 29 working documents as Internet-Drafts. The list of current Internet- 30 Drafts is at http://datatracker.ietf.org/drafts/current/. 32 Internet-Drafts are draft documents valid for a maximum of six months 33 and may be updated, replaced, or obsoleted by other documents at any 34 time. It is inappropriate to use Internet-Drafts as reference 35 material or to cite them other than as "work in progress." 37 This Internet-Draft will expire on September 6, 2018. 39 Copyright Notice 41 Copyright (c) 2018 IETF Trust and the persons identified as the 42 document authors. All rights reserved. 44 This document is subject to BCP 78 and the IETF Trust's Legal 45 Provisions Relating to IETF Documents 46 (http://trustee.ietf.org/license-info) in effect on the date of 47 publication of this document. Please review these documents 48 carefully, as they describe your rights and restrictions with respect 49 to this document. Code Components extracted from this document must 50 include Simplified BSD License text as described in Section 4.e of 51 the Trust Legal Provisions and are provided without warranty as 52 described in the Simplified BSD License. 54 Table of Contents 56 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 57 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 58 2.1. Terms used in this document . . . . . . . . . . . . . . . 3 59 2.2. Abbreviations . . . . . . . . . . . . . . . . . . . . . . 4 60 3. Requirements language . . . . . . . . . . . . . . . . . . . . 5 61 4. DetNet Over an MPLS Underlay . . . . . . . . . . . . . . . . 5 62 5. DetNet over MPLS Encapsulation Components . . . . . . . . . . 8 63 5.1. Basic Data Plane Encapsulation . . . . . . . . . . . . . 8 64 5.2. DetNet Control Word . . . . . . . . . . . . . . . . . . . 9 65 5.3. Flow identification . . . . . . . . . . . . . . . . . . . 10 66 5.4. OAM Indication . . . . . . . . . . . . . . . . . . . . . 11 67 5.5. Flow Aggregation . . . . . . . . . . . . . . . . . . . . 11 68 5.6. Aggregation at the LSP . . . . . . . . . . . . . . . . . 12 69 5.7. Aggregating DetNet flows as a new DetNet flow . . . . . . 12 70 6. Simple Aggregation at the DetNet layer . . . . . . . . . . . 13 71 7. Indication of the DetNet Payload Type. . . . . . . . . . . . 13 72 8. Operation of the PREF Functions . . . . . . . . . . . . . . . 14 73 8.1. Operation of PR . . . . . . . . . . . . . . . . . . . . . 14 74 8.2. Operation of EF . . . . . . . . . . . . . . . . . . . . . 15 75 8.3. Packet reordering considerations . . . . . . . . . . . . 15 76 8.4. Indication of PREF processing . . . . . . . . . . . . . . 16 77 8.5. Placement of PR and EF in a node . . . . . . . . . . . . 16 78 9. Operation at DetNet Node types . . . . . . . . . . . . . . . 17 79 9.1. Operation at an Ingress Edge (PE) Node . . . . . . . . . 17 80 9.2. Operation at a Relay node (S-PE) . . . . . . . . . . . . 17 81 9.3. Operation at an Egress Edge (PE) Node . . . . . . . . . . 18 82 9.4. Operation at a Transit(P) Node . . . . . . . . . . . . . 18 83 10. Other DetNet data plane considerations . . . . . . . . . . . 19 84 10.1. Class of Service . . . . . . . . . . . . . . . . . . . . 19 85 10.2. Quality of Service . . . . . . . . . . . . . . . . . . . 19 86 10.3. Bidirectional traffic . . . . . . . . . . . . . . . . . 20 87 10.4. Layer 2 addressing and QoS Considerations . . . . . . . 21 88 11. Management and control considerations . . . . . . . . . . . . 22 89 11.1. S-Label assignment and distribution . . . . . . . . . . 22 90 11.2. Explicit routes . . . . . . . . . . . . . . . . . . . . 22 91 12. Security considerations . . . . . . . . . . . . . . . . . . . 22 92 13. IANA considerations . . . . . . . . . . . . . . . . . . . . . 23 93 13.1. Acknowledgments . . . . . . . . . . . . . . . . . . . . 23 94 14. References . . . . . . . . . . . . . . . . . . . . . . . . . 23 95 14.1. Normative References . . . . . . . . . . . . . . . . . . 23 96 14.2. Informative References . . . . . . . . . . . . . . . . . 23 98 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 26 100 1. Introduction 102 This document is a derivative work from [draft-ietf-detnet-dp-sol- 103 01]. 105 Editor's Note: We need to point to the exact version that this was 106 derived from, not a generic version of [I-D.ietf-detnet-dp-sol]. 108 Whether this is published as a stand-alone text, or serves as a focal 109 point to refine the MPLS design and the key point are subsequently 110 re-merged with draft-ietf-detnet-dp-sol-01 is a matter for the DETNET 111 WG. 113 Deterministic Networking (DetNet) is a service that can be offered by 114 a network to DetNet flows. DetNet provides these flows extremely low 115 packet loss rates and assured maximum end-to-end delivery latency. 116 General background and concepts of DetNet can be found in 117 [I-D.ietf-detnet-architecture]. 119 This document specifies the encapsulation and operation of 120 deterministic networking over an MPLS data-plane. The approach is 121 modeled on the operation of Pseudowires (PW) over an MPLS Packet 122 Switched Network (PSN) [RFC3985][RFC4385]. 124 The DetNet transport layer functionality that provides congestion 125 protection for DetNet flows is assumed to be in place in a DetNet 126 node. 128 This document does not define the associated control plane functions, 129 or Operations, Administration, and Maintenance (OAM). It also does 130 not specify traffic handling capabilities required to deliver 131 congestion protection and latency control for DetNet flows at the 132 DetNet transport layer. 134 2. Terminology 136 2.1. Terms used in this document 138 Editor's note: This section needs to be reviewed when the body of the 139 text is closer to completion. 141 This document uses the terminology established in the DetNet 142 architecture [I-D.ietf-detnet-architecture] and the DetNet Data Plane 143 Solution Alternatives [I-D.ietf-detnet-dp-alt]. 145 T-Label A label used to identify the LSP used to transport a DetNet 146 flow across an MPLS PSN, e.g., a hop-by-hop label used between label 147 switching routers (LSR). 149 S-Label A DetNet "service" label that is used between DetNet nodes 150 that implement also the DetNet service layer functions. An S-Label 151 is also used to identify a DetNet flow at DetNet service layer. 153 Local-ID A DetNet Edge and Relay node internal construct that 154 uniquely identifies a DetNet flow within a node and never appear on- 155 wire. It may be used to select proper forwarding and/or DetNet 156 specific service function. 158 PREF A Packet Replication and Elimination Function (PREF) does the 159 replication and elimination processing of DetNet flow packets in edge 160 or relay nodes. The replication function is essentially the existing 161 1+1 protection mechanism. The elimination function reuses and 162 extends the existing duplicate detection mechanism to operate over 163 multiple (separate) DetNet member flows of a DetNet compound flow. 165 DetNet Control Word A control word used for sequencing and 166 identifying duplicate packets at the DetNet service layer. 168 2.2. Abbreviations 170 Editor's note: This section needs to be reviewed when the body of the 171 text is closer to completion. 173 The following abbreviations used in this document: 175 AC Attachment Circuit. 177 CE Customer Edge equipment. 179 CoS Class of Service. 181 CW Control Word. 183 d-CW DetNet Control Word. 185 DetNet Deterministic Networking. 187 DF DetNet Flow. 189 L2VPN Layer 2 Virtual Private Network. 191 LSR Label Switching Router. 193 MPLS Multiprotocol Label Switching. 195 MPLS-TP Multiprotocol Label Switching - Transport Profile. 197 MS-PW Multi-Segment Pseudowire (MS-PW). 199 NSP Native Service Processing. 201 OAM Operations, Administration, and Maintenance. 203 PE Provider Edge. 205 PREF Packet Replication and Elimination Function. 207 PSN Packet Switched Network. 209 PW Pseudowire. 211 QoS Quality of Service. 213 TSN Time-Sensitive Network. 215 3. Requirements language 217 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 218 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 219 document are to be interpreted as described in [RFC2119]. 221 4. DetNet Over an MPLS Underlay 223 Figure 1 Shows the basic components of a DetNet enabled MPLS network 224 used to transport TSN traffic using an MPLS transport. 226 DetNet Edge Transit Relay Edge DetNet 227 End Sys Node Node Node Node End Sys 229 +-----+ End to End Service +-----+ 230 |Appln|<..............................................>|Appln| 231 +-----+ +---------+ DN Flow +---------+ +---------+ +-----+ 232 | TSN | | Service |<------->| Service |<>| Service | | TSN | 233 +-----+ +---+ +---+ +-----+ +---+ +---+ +---+ +---+ +-----+ 234 |DNXpt| |Xpt| |LSP| | LSP | |LSP| |LSP| |LSP| |Xpt| |DNXpt| 235 +--.--+ +-.-+ +-.-+ +-.-.-+ +-.-+ +-.-+ +-.-+ +-.-+ +--.--+ 236 : : : : :Link : :Link : : : 237 +-------+ /-------\+-----+ +------+ /-------\ 238 TSN |Sub N/W| |TSN N/W| 239 Link \-------/ \-------/ 241 LSP = MPLS Transport 242 DNXpt & Xpt = DetNet Transport 244 Figure 1: A Simple DetNet Enabled MPLS Network 246 TSN End Systems send and receive packets over the DetNet. The 247 supported packet types are IP and Ethernet. 249 Edge Nodes are responsible for the imposition and disposition of the 250 required DetNet encapsulation. These are functionally similar to 251 pseudowire (PW) Terminating Provide Edge (T-PE) equipment. 253 Relay nodes are strategically placed and used enhance the reliability 254 of delivery by enabling the replication of packets so that multiple 255 copies, possibly over multiple paths. They also reduce the impact of 256 replication by the eliminating surplus copies of DetNet packets. 257 Replication and elimination may also be performed at ingress and 258 egress edge nodes respectively. 260 Edge nodes, and relay nodes are aware of the needs of particular 261 DetNet flows and take care to process them in accordance with the 262 required performance needs. 264 Transit nodes are normal MPLS Label Switching Routers (LSRs). They 265 are unaware of the special requirements of DetNet flows, although 266 they may be configured to provide traffic engineering services to 267 them to enhance the prospect of them meeting the required service 268 level agreement (SLA). 270 The MPLS LSP may be provided by any MPLS method (LSP, RSVP-TE, MPLS- 271 TP, or MPLS Segment Routing (SR)). 273 Figure 2 illustrates how end to end MPLS-based DetNet service is 274 provided in a more detail. 275 In this case, the end systems are able to send and receive DetNet 276 flows. For example, an end system sends data encapsulated in MPLS. 277 The 'X' in the end systems, edge and relay nodes represents potential 278 DetNet flow packet replication and elimination points. Here the 279 relay nodes may change the underlying transport, for example 280 tunneling MPLS over IP [draft-xu-mpls-sr-over-ip], or simply 281 interconnect network segments. 283 DetNet DetNet 284 Service Transit Transit Service 285 DetNet | |<-Tnl->| |<-Tnl->| | DetNet 286 End | V 1 V V 2 V | End 287 System | +--------+ +--------+ +--------+ | System 288 +---+ | | R1 |=======| R2 |=======| R3 | | +---+ 289 | X...DFa...|._X_....|..DF1..|.__ ___.|..DF3..|...._X_.|.DFa..|.X | 290 |CE1|========| \ | | X | | / |======|CE2| 291 | | | | \_.|..DF2..|._/ \__.|..DF4..|._/ | | | | 292 +---+ | |=======| |=======| | +---+ 293 ^ +--------+ +--------+ +--------+ ^ 294 | Relay Node Relay Node Relay Node | 295 | | 296 |<--------------- End to End DetNet Service -------------->| 298 Figure 2: Flows in a DetNet Enabled MPLS Network 300 An example MPLS DetNet network fragment and packet flow is 301 illustrated in Figure 3. 303 1111 11111111 111111 222222 2222222 333 304 CE1----EN1--------R1-------R2-------R3--------EN2----CE2 305 \2 22222/ 3 / 306 \2222222 /----+ 3 / 307 +------R4------------------------+ 308 333333333333333333333333 310 Figure 3: Example Packet flow in DetNet Enabled MPLS Network 312 Customer Equipment CE1 send a packet into the DetNet enabled MPLS 313 network. Edge Node EN1 makes a copy of the packet and encapsulates 314 each copy as a DetNet packet (packet 1 and packet 2). It sends one 315 copy (1) to Relay Node R1 and the other copy (2) to Relay Node R4. 316 R1 sends packet copy 1 to R2. R4 sends one copy to R2, and a further 317 copy (3) to EN2. R2 receives copy (2) before copy 1, and so 318 eliminated copy (1) sending only (2) to EN2. EN2 receives copy (3) 319 first sending it to CE2 and eliminating copy (2). Note the number 320 illustrates the replication instance and should not be confused with 321 the sequence number which remains unchanged in all packet copies. 323 The above is of course illustrative of many network scenarios that 324 can be configured. Between a pair of relay nodes there may be one or 325 more transport nodes that simply forward the DetNet traffic, but 326 these are omitted for clarity. 328 5. DetNet over MPLS Encapsulation Components 330 To carry DetNet over MPLS the following is required: 332 1. A method of identifying the MPLS payload type. 334 2. A method of identifying the DetNet flow group to the processing 335 element. 337 3. A method of distinguishing DetNet OAM packets from DetNet data 338 packets. 340 4. A method of carrying the DetNet sequence number. 342 5. A suitable LSP to deliver the packet to the egress PE. 344 6. A method of carrying queuing and forwarding indication. 346 In this design an MPLS service label (the S-Label), similar to a 347 pseudowire (PW) label [RFC3985], is used to identify both the DetNet 348 flow identity and the payload MPLS payload type satisfying (1) and 349 (2) in the list above. OAM traffic discrimination happens through 350 the use of the Associated Channel method described in [RFC4385]. The 351 sequence number is carried in the DetNet Control word which carries 352 the Data/OAM discriminator. The LSP used to transport the DetNet 353 packet may be of any type (MPLS-LDP, MPLS-TE, MPLS-TP[RFC5921], or 354 MPLS-SR[I-D.ietf-spring-segment-routing-mpls]). The LSP (T-Label) 355 label and/or the S-Label may be used to indicate the queue processing 356 as well as the forwarding parameters. 358 Note that when the network consists only of DetNet enabled nodes with 359 no aggregation, Penultimate Hop Popping (PHP) means that the only 360 label in the label stack is the S-label. 362 5.1. Basic Data Plane Encapsulation 364 Figure 4 shows a DetNet data plane MPLS encapsulation. This is 365 modeled on the encapsulation of pseudowires over MPLS [RFC3985]. 367 The encapsulation consists of: 369 o DetNet control word (d-CW) containing sequencing information for 370 packet replication and duplicate elimination purposes, and the OAM 371 indicator. There MUST a separate sequence number space for each 372 DetNet flow. 374 o DetNet Label (S-label) that identifies a DetNet flow to the peer 375 node that is to process it. The S-Label is allocated from the 376 platform label space. 378 o Zero or more MPLS transport LSP label(s) (T-label) used to direct 379 the packet along the label switched path (LSP) to the next peer 380 node along the path. When Penultimate Hop Popping is in use there 381 will be no label T-label in the protocol stack on the final hop. 383 o The necessary data-link encapsulation is then applied prior to 384 transmission over the physical media. 386 RFC Editor - if you ever get this text please remove this para (text 387 to make compiler work). 389 +---------------------------------+ 390 | | 391 | DetNet Flow | 392 | Payload Packet | 393 | | 394 +---------------------------------+ <--\ 395 | DetNet Control Word | | 396 +=================================+ +--> DetNet data plane 397 | S-Label | | MPLS encapsulation 398 +---------------------------------+ <--/ 399 | T-Label(s) | 400 +---------------------------------+ 401 | Data-Link | 402 +---------------------------------+ 403 | Physical | 404 +---------------------------------+ 406 Figure 4: MPLS Encapsulation of DetNet 408 Flow aggregation may be necessary to achieve the required scaling. 409 The extensions to basic encapsulation needed to support flow 410 aggregation are described in Section 5.5. 412 5.2. DetNet Control Word 414 A DetNet Control Word (d-CW) conforms to the Generic PW MPLS Control 415 Word (PWMCW) defined in [RFC4385] and is shown in Figure 5. In a 416 DetNet data packet the upper nibble of the d-CW MUST be set to zero 417 (0). The effective sequence number bit length is between 0 and 28 418 bits, and configured either by a control plane or manually for each 419 DetNet flow. The sequence number is aligned to the right (least 420 significant bits) and unused bits MUST be set to zero (0). Each 421 DetNet flow MUST have its own sequence number counter. The sequence 422 number is incremented by one for each new packet. 424 Editor's note: We need to consider whether it is better to allow a 425 multiplicity of sequence number lengths with a length configured for 426 each flow, or a uniform sequence number of 28. On reflection it 427 seems better to go for the simplicity of a standard length. If for 428 any reason a different length becomes desirable, then it is 429 relatively simple to define another type of DetNet d-CW with a 430 different standard sequence number length and there is no ambiguity 431 of operation since the sequence number length is a parameter of the 432 S-Label. 434 The d-CW MUST always be present in a packet. In cases where the 435 sequence number is not used (e.g., for DetNet-t-flows) the control 436 plane or the manual configuration has to define zero (0) bit length 437 sequence number and the value of the sequence number MUST be set to 438 zero (0). 440 Editors Note: Do we set length zero or say it is not used? Also need 441 to add text to stop relay nodes and egress edge nodes from processing 442 the s/n otherwise only one packet ever gets through! 444 0 1 2 3 445 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 446 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 447 |0 0 0 0| Sequence Number | 448 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 450 Figure 5: DetNet Control Word 452 5.3. Flow identification 454 DetNet flow identification at a DetNet service layer is realized by 455 an S-label. It maps a DetNet flow to a specific d-CW in a DetNet 456 node. 458 For a data flow the S-label used for flow identification MUST be 459 bottom label of the label stack for a DetNet-s- or DetNet-st-flow and 460 MUST precede the d-CW. 462 Editor's note: We have specified the above for _data_ to leave the 463 door open for the GAL based OAM method as an alternative to the ACH 464 mechanism currently specified. When using GAL the GAL label would be 465 after the S-Label. Do we leave this option in or shut the door on 466 it? 468 An S-label for a single DetNet flow MUST be unique at the peer at the 469 node that is to process it. The S-label is stored in the platform 470 label table to allow for DetNet packet processing independent of the 471 interface on which the packet is received on. 473 5.4. OAM Indication 475 OAM follows the procedures set out in [RFC5085] with the restriction 476 that only Virtual Circuit Connectivity Verification (VCCV) type 1 is 477 supported. 479 Editor's Note - is this restriction acceptable? Do we need to 480 support GAL based OAM indication? 482 As shown in Figure 3 of [RFC5085] when the first nibble of the d-CW 483 is 0x0 the payload following the d-CW is normal user data. However, 484 when the first nibble of the d-CW is 0X1, the payload that follows 485 the d-DW is an OAM payload with the OAM type indicated by the value 486 in the d-CW Channel Type field. 488 The reader is referred to [RFC5085] for a more detailed description 489 of the Associated Channel mechanism, and to the DetNet work on OAM 490 for more information DetNet OAM. 492 5.5. Flow Aggregation 494 The ability to aggregate individual flows, and their associated 495 resource control, into a larger aggregate is an important technique 496 for improving scaling of control in the data, management and control 497 planes. The DetNet data plane allows for the aggregation of DetNet 498 flows, to improved scaling. There are three methods of introducing 499 flow aggregation: 501 1. Aggregate at the LSP (Transport) 503 2. Aggregating DetNet flows as a new DetNet flow 505 3. Simple Aggregation at the DetNet layer 507 A further method of using SR to perform aggregation is for further 508 study. 510 The resource control and management aspects of aggregation (including 511 the queuing/shaping/ policing implications) will be covered in other 512 documents. 514 5.6. Aggregation at the LSP 516 DetNet flows transported via MPLS can leverage MPLS-TE's existing 517 support for hierarchical LSPs (H-LSPs), see [RFC4206]. H-LSPs are 518 typically used to aggregate control and resources, they may also be 519 used to provide OAM or protection for the aggregated LSPs. Arbitrary 520 levels of aggregation naturally falls out of the definition for 521 hierarchy and the MPLS label stack [RFC3032]. DetNet nodes which 522 support aggregation (LSP hierarchy) map one or more LSPs (labels) 523 into and from an H-LSP. Both carried LSPs and H-LSPs may or may not 524 use the TC field, i.e., L-LSPs or E-LSPs. Such nodes will need to 525 ensure that traffic from aggregated LSPs are placed (shaped/policed/ 526 enqueued) onto the H-LSPs in a fashion that ensures the required 527 DetNet service is preserved. 529 Additional details of the traffic control capabilities needed at a 530 DetNet-aware node may be covered in the new service descriptions 531 mentioned above or in separate future documents. Management and 532 control plane mechanisms will also need to ensure that the service 533 required on the aggregate flow (H-LSP or DSCP) are provided, which 534 may include the discarding or remarking mentioned in the previous 535 sections. 537 5.7. Aggregating DetNet flows as a new DetNet flow 539 An aggregate can be built by layering DetNet flows as shown below: 541 +---------------------------------+ 542 | | 543 | DetNet Flow | 544 | Payload Packet | 545 | | 546 +---------------------------------+ <--\ 547 | DetNet Control Word | | 548 +=================================+ | 549 | S-Label | | 550 +---------------------------------+ +----DetNet data plane 551 | DetNet Control Word | | MPLS encapsulation 552 +=================================+ | 553 | A-Label | | 554 +---------------------------------+ <--/ 555 | T-Label(s) | 556 +---------------------------------+ 557 | Data-Link | 558 +---------------------------------+ 559 | Physical | 560 +---------------------------------+ 561 Both the Aggregation (A) label and the S-label have their MPLS S bit 562 set indicating bottom of stack, and the d-CW allows the PREF 563 functions to work. 565 It is a property of the A-label that what follows is d-CW followed by 566 an S-label. A relay node processing the A-label would not know the 567 underlying payload type. This would only be known to a node that was 568 a peer of the node imposing the S-label. However there is no real 569 need for it to know the payload type during aggregation processing. 571 6. Simple Aggregation at the DetNet layer 573 Another approach would be not to include a d-CW for the aggregated 574 flow. This would be functionally similar to aggregation at the 575 transport layer using H-LSPs, but would confine knowledge of the 576 aggregation to the DetNet layer. Such an approach shares the 577 disadvantage that PREF operations would not be possible. OAM 578 operation in this mode is for further study. 580 +---------------------------------+ 581 | | 582 | DetNet Flow | 583 | Payload Packet | 584 | | 585 +---------------------------------+ <--\ 586 | DetNet Control Word | | 587 +=================================+ | 588 | S-Label | +----DetNet data plane 589 +---------------------------------+ | MPLS encapsulation 590 | A-Label | | 591 +---------------------------------+ <--/ 592 | T-Label(s) | 593 +---------------------------------+ 594 | Data-Link | 595 +---------------------------------+ 596 | Physical | 597 +---------------------------------+ 599 7. Indication of the DetNet Payload Type. 601 The only node types that needs to know the payload type is the 602 ingress node which has to know how to process the packet it receives 603 on the ingress AC, and egress edge node which has to know how to 604 prepare the packet for transmission on the egress AC. 606 On ingress a DetNet edge node has to classify the packets into those 607 that are for transmission as Detnet packets and those that are for 608 transmission as "normal" packets at one of more lower priorities. 610 The packet type is indicated to the egress edge node through the 611 value of the S-label. Thus when the egress edge node looks up the 612 S-label one of the parameters returned is the packet type which in 613 turn tells the egress edge node how to prepare the packet for 614 transmission over the egress AC. 616 Editor's note: Do we only find one type on the ingress and egress or 617 do we need to run missed mode, i.e. will we have to send some packets 618 across the network as Ethernet packets and some as IP packets? 620 8. Operation of the PREF Functions 622 The Packet Replication and Elimination Functions (PREF) are designed 623 to enhance the reliability of delivery by network layer packet 624 replication (PR), whilst at the same time minimizing network 625 congestion and the duplicate delivery of packets over an egress AC by 626 eliminating duplicate packets (EF). 628 PR and EF are independent functions that operate on a DetNet flow at 629 strategic places in the network. The placement of a function is a 630 matter for the network operator. 632 8.1. Operation of PR 634 A PR function creates two or more copies of a packet, and forwards a 635 copy to each of the designated peers of the replicating node. A PR 636 function may be placed in an ingress edge node, a relay node or an 637 egress edge node. 639 We consider first a DetNet relay node. The packet is received from 640 the upstream DetNet peer, and if present the T-label is popped. The 641 S-label is looked up in the platform label table and if the 642 forwarding instructions indicate that replication is required, the 643 following happens for each next hop in the DetNet layer: 645 o A copy of the payload is made. 647 o An identical copy of the d-CW from the original packet is pushed. 649 o The S-Label required by the next hop in the DetNet layer is 650 pushed. 652 o The DetNet packet copy is forwarded on the LSP to the next hop in 653 the DetNet layer using normal MPLS forwarding. 655 An ingress node operates in the same way as a relay node, except that 656 it is responsible for initial encapsulation of the packet. The 657 packet is received from the AC, classified and prepared for 658 forwarding over the DetNet network as described in Section 9.1, 659 except that for each next-hop in the DetNet Layer (i.e. each node 660 that is to receive a copy of the DetNet packet) : 662 o A copy of the payload is made. 664 o The d-CW is constructed using the next sequence number in the 665 sequence associated with this service. 667 o An identical copy of the d-CW is pushed. 669 o The S-Label required by the next hop in the DetNet layer to 670 recognize this service is pushed. 672 o The DetNet packet copy is forwarded on the LSP to the next hop in 673 the DetNet layer using normal MPLS forwarding. 675 An egress node operates in the same way as 676 as described in Section 9.1, except that where the S-Label indicates 677 that the the packet is to be forwarded to a multiply attached system 678 in the payload layer a similar copy (modified as needed to conform to 679 any MAC requirements) is forwarded out of each egress AC. 681 Editor's Note: For normal PW, there will be one to one AC to PW 682 binding relationship, for DetNet, no matter the ingress or egress 683 side, there may be multiple AC corresponding to a single DetNet PW 684 (S-label), should we state explicitly? 686 8.2. Operation of EF 688 The EF function eliminates duplicate copies of a packet. The node 689 identifies the service from the packet S-Label. If the S-Label 690 indicates that the packet elimination function is required to operate 691 on this service at this node, it uses the sequence number in the d-CW 692 to determine whether or not the packet is a duplicate that must be 693 eliminated, and precedes accordingly. 695 The EF may be placed in a Relay node, or an Edge Egress node. 697 8.3. Packet reordering considerations 699 The DetNet service layer introduces packet reordering functionality 700 for use in DetNet edge and relay node and end system packet 701 processing. The reordering functionality MAY be enabled in a DetNet 702 node. The reordering functionality relies on a presence of sequence 703 numbers the d-CW. 705 8.4. Indication of PREF processing 707 The indication that PR or EF processing is needed at a DetNet Relay 708 node or a DetNet egress edge node is carried as an implicit 709 characteristic of the S-Label. Thus, when a service is established 710 the ingress edge node is configured to use an active rather than a 711 static zero sequence number, and DetNet relays and egress edge nodes 712 are configured to run PR and/or EF functionality on on services 713 identified by specific S-labels. 715 8.5. Placement of PR and EF in a node 717 This section is not normative. 719 The placement of the PR and EF functions within the node is a matter 720 for the node designers, and this specification makes no determination 721 on this matter. However the placement may well have implications for 722 the management and control of a node, and thus the following is worth 723 noting. 725 In a bladed system a common processing model is to analyze the packet 726 for forwarding close to the ingress interface and to either to fully 727 prepare it for forwarding as part of ingress processing, or to 728 package it with internal metadata for final preparation close to the 729 egress interface. In such systems the natural place to perform 730 replication is as part of the ingress processing since egress 731 processors often do not have the capability (for example processing 732 power) to further process the packet, often do not normally have the 733 required data paths to facilitate replication for transmission to 734 other line cards. 736 Furthermore, in bladed systems it is to be expected that a packet 737 from a peer in the MPLS layer may arrive on any of the blades. 738 Whilst in principle some constraints could be applied on which 739 interfaces the packets will arrive on, experience shows that such 740 constraints are operationally impractical. To perform elimination 741 state must be shared amongst the forwarders performing elimination. 742 Sharing the state required for elimination between forwarders on 743 different blades without sacrificing performance is technically 744 difficult. Thus, the natural place for elimination in a distributed 745 design is close to the egress interface. 747 On mono-processor systems these constraints do not apply in quite the 748 same way, although even in this case it is necessary for the 749 equipment designer to consider the implications of particular 750 forwarder design, for example the allocation of Network Processing 751 Unit (NPU) cores to particular interfaces. 753 A bladed system may place the PR and EF functions on a processing 754 function other than the ingress or egress forwarding processor, 755 whereupon the mono-processor considerations apply. 757 As noted above the placement of the PR and EF functions is a matter 758 for the system designer. However it is important that nothing in the 759 control, configuration, or OAM system results in undue difficulties 760 for any of the forwarding models. 762 9. Operation at DetNet Node types 764 This section considers the operations at the various DetNet node 765 types in more detail. 767 9.1. Operation at an Ingress Edge (PE) Node 769 An ingress Edge Node first classifies received traffic into DetNet 770 flows and non-Detnet traffic. The DetNet flows are sent over the 771 DetNet network using the procedures defined in the document. The 772 classification process and the handling of non-DetNet traffic is out 773 of scope for this document. 775 The processing of non-DetNet flows is currently outside the scope of 776 this document 778 The packet is encapsulated as described in Section 5.1. First, if 779 the flow is to be carried as IP the MAC header and checksum are 780 removed. Otherwise the checksum is removed. The d-CW is constructed 781 using the next sequence number associated with the service, and is 782 pushed. The S-label corresponding to this DetNet flow is then 783 pushed. The packet is then forwarded to the required egress edge- 784 node by pushing the required T-Labels and the required data-link 785 headers. 787 Editor's Note: we need text on how we indicate IPv4 vs IPv6. In MPLS 788 they are carried on different FECs. Presumable we need to have a 789 different S-Label for each IP type. This implies a different s/n for 790 each IP type, but since it seems unlikely that we need to maintain 791 them in a common sequence that looks to be OK. 793 9.2. Operation at a Relay node (S-PE) 795 At a relay node packet forwarding follows the same process used in a 796 PW S-PE [RFC5659], save for the additional processing required for 797 any required PREF processing. 799 The packet is received from the incoming LSP and any T-labels which 800 are still present are popped. The S-label is looked up in the 801 platform label table to determine the forwarding parameters. If PREF 802 processing is required the process described in Section 8 is 803 followed, and if required the locally held sequence number 804 information associated with the S-label is updated to avoid future 805 duplicate forwarding. 807 For each copy of the packet to be forwarded the S-label is swapped 808 for the S-label required by either the next relay node, or by the 809 egress edge node. For each copy of the packet, the T-label(s) needed 810 to reach the next relay node or the egress edge node is pushed and 811 the packet forwarded towards its next hop in the MPLS layer. 813 9.3. Operation at an Egress Edge (PE) Node 815 At an egress edge node, the S-Label is looked up in the platform 816 label table and is used to determine the egress packet processing 817 parameters. The sequence number in d-CW and the recorded sequence 818 number history is compared to perform any required ER processing 819 Section 8.2. If the packet is not eliminated, the d-CW is stripped. 821 If the packet is to be treated as an IP packet, this is processed in 822 the normal way for an IP packet egressing an MPLS tunnel (for example 823 the data-link header is constructed) and the packet forwarded towards 824 its destination. 826 Editor's note: Do we need to look the IP address up in a forwarding 827 table, if so, which one, or is the next hop a forwarding parameter. 828 Is the MAC address a forwarding parameter, or do we need to run ARP 829 as normal? 831 If the packet is to be treated as an Ethernet packet it is forwarded 832 unmodified, save for the addition of the CRC as described in 833 [RFC4448]. 835 9.4. Operation at a Transit(P) Node 837 Operation at a transit (P) node is normal MPLS forwarding. The outer 838 label is either swapped or popped as required, and the packet is 839 forwarded along the LSP. If an entropy label is present in the label 840 stack, this may be used by the Equal Cost Multi-Path (ECMP) selection 841 process. No other label is inspected as part of forwarding. 843 The Traffic Class field and/or incoming LSP transport label may be 844 used to indicate how the packet is to be processed and queued 845 including which forwarding resources are to be applied to the packet 846 ((RFC3270}}. Any of the methods of constructing a physical LSP (for 847 RSVP-TE signaling or MPLS-TP style controller based configuration) or 848 a virtual LSP (Segment Routing 850 [I-D.ietf-spring-segment-routing-mpls]) may be used to indicate not 851 only the next hop, but the priority of processing and any physical 852 resources dedicated to the service [I-D.bryant-rtgwg-enhanced-vpn]. 854 10. Other DetNet data plane considerations 856 10.1. Class of Service 858 Class and quality of service, i.e., CoS and QoS, are terms that are 859 often used interchangeably and confused. In the context of DetNet, 860 CoS is used to refer to mechanisms that provide traffic forwarding 861 treatment based on aggregate group basis and QoS is used to refer to 862 mechanisms that provide traffic forwarding treatment based on a 863 specific DetNet flow basis. Examples of existing network level CoS 864 mechanisms include DiffServ which is enabled by IP header 865 differentiated services code point (DSCP) field [RFC2474] and MPLS 866 label traffic class field [RFC5462], and at Layer-2, by IEEE 802.1p 867 priority code point (PCP). 869 CoS for DetNet flows carried in PWs and MPLS is provided using the 870 existing MPLS Differentiated Services (DiffServ) architecture 871 [RFC3270]. Both E-LSP and L-LSP MPLS DiffServ modes MAY be used to 872 support DetNet flows. The Traffic Class field (formerly the EXP 873 field) of an MPLS label follows the definition of [RFC5462] and 874 [RFC3270]. The Uniform, Pipe, and Short Pipe DiffServ tunneling and 875 TTL processing models are described in [RFC3270] and [RFC3443] and 876 MAY be used for MPLS LSPs supporting DetNet flows. MPLS ECN MAY also 877 be used as defined in ECN [RFC5129] and updated by [RFC5462]. 879 One additional consideration for DetNet nodes which support CoS 880 services is that they MUST ensure that the CoS service classes do not 881 impact the congestion protection and latency control mechanisms used 882 to provide DetNet QoS. This requirement is similar to requirement 883 for MPLS LSRs to that CoS LSPs do not impact the resources allocated 884 to TE LSPs via [RFC3473]. 886 10.2. Quality of Service 888 Quality of Service (QoS) mechanisms for flow specific traffic 889 treatment typically includes a guarantee/agreement for the service, 890 and allocation of resources to support the service. Example QoS 891 mechanisms include discrete resource allocation, admission control, 892 flow identification and isolation, and sometimes path control, 893 traffic protection, shaping, policing and remarking. Example 894 protocols that support QoS control include Resource ReSerVation 895 Protocol (RSVP) [RFC2205] and RSVP-TE [RFC3209] and [RFC3473]. The 896 existing MPLS mechanisms defined to support CoS [RFC3270] can also be 897 used to reserve resources for specific traffic classes. 899 In addition to explicit routes Section 11.2, and packet replication 900 and elimination, described in Section 8 above, DetNet provides zero 901 congestion loss and bounded latency and jitter. As described in 902 [I-D.ietf-detnet-architecture], there are different mechanisms that 903 maybe used separately or in combination to deliver a zero congestion 904 loss service. These mechanisms are provided by the either the MPLS 905 or IP layers, and may be combined with the mechanisms defined by the 906 underlying network layer such as 802.1TSN. 908 A baseline set of QoS capabilities for DetNet flows carried over MPLS 909 can be provided with Traffic Engineering (MPLS-TE) [RFC3209] and 910 [RFC3473]. TE LSPs can also support explicit routes (path pinning). 911 Current service definitions for packet TE LSPs can be found in 912 "Specification of the Controlled Load Quality of Service", [RFC2211], 913 "Specification of Guaranteed Quality of Service", [RFC2212], and 914 "Ethernet Traffic Parameters", [RFC6003]. Additional service 915 definitions are expected in future documents to support the full 916 range of DetNet services. In all cases, the existing label-based 917 marking mechanisms defined for TE-LSPs and even E-LSPs are use to 918 support the identification of flows requiring DetNet QoS. 920 Packets that are marked with a DetNet Class of Service value, but 921 that have not been the subject of a completed reservation, can 922 disrupt the QoS offered to properly reserved DetNet flows by using 923 resources allocated to the reserved flows. Therefore, the network 924 nodes of a DetNet network MUST: 926 o Defend the DetNet QoS by discarding or remarking (to a non-DetNet 927 CoS) packets received that are not the subject of a completed 928 reservation. 930 o Not use a DetNet reserved resource, e.g. a queue or shaper 931 reserved for DetNet flows, for any packet that does not carry a 932 DetNet Class of Service marker. 934 10.3. Bidirectional traffic 936 Some DetNet applications generate bidirectional traffic. Using MPLS 937 definitions [RFC5654] there are associated bidirectional flows, and 938 co-routed bidirectional flows. MPLS defines a point-to-point 939 associated bidirectional LSP as consisting of two unidirectional 940 point-to-point LSPs, one from A to B and the other from B to A, which 941 are regarded as providing a single logical bidirectional transport 942 path. This would be analogous of standard IP routing, or PWs running 943 over two reciprocal unidirectional LSPs. MPLS defines a point-to- 944 point co-routed bidirectional LSP as an associated bidirectional LSP 945 which satisfies the additional constraint that its two unidirectional 946 component LSPs follow the same path (in terms of both nodes and 947 links) in both directions. An important property of co-routed 948 bidirectional LSPs is that their unidirectional component LSPs share 949 fate. In both types of bidirectional LSPs, resource allocations may 950 differ in each direction. The concepts of associated bidirectional 951 flows and co-routed bidirectional flows can be applied to DetNet 952 flows as well. PWs [RFC3985] are always created as bidirectional 953 constructs. 955 While the MPLS data planes must support bidirectional DetNet flows, 956 there are no special bidirectional features with respect to the data 957 plane other than need for the two directions take the same paths. 958 Fate sharing and associated vs co-routed bidirectional flows can be 959 managed at the control level. Note, that there is no stated 960 requirement for bidirectional DetNet flows to be supported using the 961 same MPLS Labels in each direction, and indeed to do so would 962 introduce significant implementation issues. Control mechanisms will 963 be need to support such bidirectional flows DetNet over MPLS, but 964 such mechanisms are out of scope of this document. An example 965 control plane solution for MPLS can be found in [RFC7551]. 967 10.4. Layer 2 addressing and QoS Considerations 969 Editor's note: Add references needed by this section 971 The Time-Sensitive Networking (TSN) Task Group of the IEEE 802.1 972 Working Group have defined (and are defining) a number of amendments 973 to IEEE 802.1Q [IEEE8021Q] that provide zero congestion loss and 974 bounded latency in bridged networks. IEEE 802.1CB [IEEE8021CB] 975 defines packet replication and elimination functions that should 976 prove both compatible with and useful to, DetNet networks. 978 As is the case for DetNet, a Layer 2 network node such as a bridge 979 may need to identify the specific DetNet flow to which a packet 980 belongs in order to provide the TSN/DetNet QoS for that packet. It 981 also will likely need a CoS marking, such as the priority field of an 982 IEEE Std 802.1Q VLAN tag, to give the packet proper service. 984 Although the flow identification methods described in IEEE 802.1CB 985 [IEEE8021CB] are flexible, and in fact, include IP 5-tuple 986 identification methods, the baseline TSN standards assume that every 987 Ethernet frame belonging to a TSN stream (i.e. DetNet flow) carries 988 a multicast destination MAC address that is unique to that flow 989 within the bridged network over which it is carried. Furthermore, 990 IEEE 802.1CB [IEEE8021CB] describes three methods by which a packet 991 sequence number can be encoded in an Ethernet frame. 993 Ensuring that the proper Ethernet VLAN tag priority and destination 994 MAC address are used on a DetNet/TSN packet may require further 995 clarification of the customary L2/L3 transformations carried out by 996 routers and edge label switches. Edge nodes may also have to move 997 sequence number fields among Layer 2, PW, and IPv6 encapsulations. 999 11. Management and control considerations 1001 While management plane and control planes are traditionally 1002 considered separately, from the Data Plane perspective there is no 1003 practical difference based on the origin of flow provisioning 1004 information. This document therefore does not distinguish between 1005 information provided by a control plane protocol, e.g., RSVP-TE 1006 [RFC3209] and [RFC3473], or by a network management mechanisms, e.g., 1007 RestConf [RFC8040] and YANG [RFC7950]. 1009 Editor's note: Not sure if RSVP-TE can be used, indeed unsure what 1010 routing protocols can be use other than to create point to point MPLS 1011 transport paths. Normally we construct a a single path through the 1012 network with RSVP-TE, but here we need to construct an explicit mesh 1013 at the DetNet layer. The classical routing protocols are not really 1014 capable of constructing graphs of the sort needed here either. 1016 11.1. S-Label assignment and distribution 1018 A mechanism based on the existing PW label distribution protocol 1019 [RFC8077] can be used to exchange labels between DetNet nodes. The 1020 protocol may however need extending depending on the preferred format 1021 of the DetNet flow identifiers. 1023 A mechanism to create the flow graph through the relay nodes will 1024 need to be specified. Initially this is likely to be based on a 1025 network controller of some sort rather than a distributed routing 1026 protocol. 1028 The details of the control plane protocol solution required for the 1029 label distribution and the management of the label number space are 1030 out of scope of this document. 1032 11.2. Explicit routes 1034 Editor's note describe the applicability of explicit routes as a 1035 method of constructing paths 1037 12. Security considerations 1039 The security considerations of DetNet in general are discussed in 1040 [I-D.ietf-detnet-architecture] and [I-D.sdt-detnet-security]. Other 1041 security considerations will be added in a future version of this 1042 draft. 1044 13. IANA considerations 1046 This document makes no IANA requests. 1048 13.1. Acknowledgments 1050 We acknowledge the authors of draft-ietf-detnet-dp-sol-01. 1052 14. References 1054 14.1. Normative References 1056 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1057 Requirement Levels", BCP 14, RFC 2119, 1058 DOI 10.17487/RFC2119, March 1997, . 1061 [RFC4448] Martini, L., Ed., Rosen, E., El-Aawar, N., and G. Heron, 1062 "Encapsulation Methods for Transport of Ethernet over MPLS 1063 Networks", RFC 4448, DOI 10.17487/RFC4448, April 2006, 1064 . 1066 [RFC5085] Nadeau, T., Ed. and C. Pignataro, Ed., "Pseudowire Virtual 1067 Circuit Connectivity Verification (VCCV): A Control 1068 Channel for Pseudowires", RFC 5085, DOI 10.17487/RFC5085, 1069 December 2007, . 1071 14.2. Informative References 1073 [I-D.bryant-rtgwg-enhanced-vpn] 1074 Bryant, S. and J. Dong, "Enhanced Virtual Private Networks 1075 (VPN+)", draft-bryant-rtgwg-enhanced-vpn-01 (work in 1076 progress), October 2017. 1078 [I-D.ietf-detnet-architecture] 1079 Finn, N., Thubert, P., Varga, B., and J. Farkas, 1080 "Deterministic Networking Architecture", draft-ietf- 1081 detnet-architecture-04 (work in progress), October 2017. 1083 [I-D.ietf-detnet-dp-alt] 1084 Korhonen, J., Farkas, J., Mirsky, G., Thubert, P., 1085 Zhuangyan, Z., and L. Berger, "DetNet Data Plane Protocol 1086 and Solution Alternatives", draft-ietf-detnet-dp-alt-00 1087 (work in progress), October 2016. 1089 [I-D.ietf-detnet-dp-sol] 1090 Korhonen, J., Andersson, L., Jiang, Y., Finn, N., Varga, 1091 B., Farkas, J., Bernardos, C., Mizrahi, T., and L. Berger, 1092 "DetNet Data Plane Encapsulation", draft-ietf-detnet-dp- 1093 sol-01 (work in progress), January 2018. 1095 [I-D.ietf-spring-segment-routing-mpls] 1096 Bashandy, A., Filsfils, C., Previdi, S., Decraene, B., 1097 Litkowski, S., and R. Shakir, "Segment Routing with MPLS 1098 data plane", draft-ietf-spring-segment-routing-mpls-12 1099 (work in progress), February 2018. 1101 [I-D.sdt-detnet-security] 1102 Mizrahi, T., Grossman, E., Hacker, A., Das, S., Dowdell, 1103 J., Austad, H., Stanton, K., and N. Finn, "Deterministic 1104 Networking (DetNet) Security Considerations", draft-sdt- 1105 detnet-security-01 (work in progress), July 2017. 1107 [RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S. 1108 Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1 1109 Functional Specification", RFC 2205, DOI 10.17487/RFC2205, 1110 September 1997, . 1112 [RFC2211] Wroclawski, J., "Specification of the Controlled-Load 1113 Network Element Service", RFC 2211, DOI 10.17487/RFC2211, 1114 September 1997, . 1116 [RFC2212] Shenker, S., Partridge, C., and R. Guerin, "Specification 1117 of Guaranteed Quality of Service", RFC 2212, 1118 DOI 10.17487/RFC2212, September 1997, . 1121 [RFC2474] Nichols, K., Blake, S., Baker, F., and D. Black, 1122 "Definition of the Differentiated Services Field (DS 1123 Field) in the IPv4 and IPv6 Headers", RFC 2474, 1124 DOI 10.17487/RFC2474, December 1998, . 1127 [RFC3032] Rosen, E., Tappan, D., Fedorkow, G., Rekhter, Y., 1128 Farinacci, D., Li, T., and A. Conta, "MPLS Label Stack 1129 Encoding", RFC 3032, DOI 10.17487/RFC3032, January 2001, 1130 . 1132 [RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V., 1133 and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP 1134 Tunnels", RFC 3209, DOI 10.17487/RFC3209, December 2001, 1135 . 1137 [RFC3270] Le Faucheur, F., Wu, L., Davie, B., Davari, S., Vaananen, 1138 P., Krishnan, R., Cheval, P., and J. Heinanen, "Multi- 1139 Protocol Label Switching (MPLS) Support of Differentiated 1140 Services", RFC 3270, DOI 10.17487/RFC3270, May 2002, 1141 . 1143 [RFC3443] Agarwal, P. and B. Akyol, "Time To Live (TTL) Processing 1144 in Multi-Protocol Label Switching (MPLS) Networks", 1145 RFC 3443, DOI 10.17487/RFC3443, January 2003, 1146 . 1148 [RFC3473] Berger, L., Ed., "Generalized Multi-Protocol Label 1149 Switching (GMPLS) Signaling Resource ReserVation Protocol- 1150 Traffic Engineering (RSVP-TE) Extensions", RFC 3473, 1151 DOI 10.17487/RFC3473, January 2003, . 1154 [RFC3985] Bryant, S., Ed. and P. Pate, Ed., "Pseudo Wire Emulation 1155 Edge-to-Edge (PWE3) Architecture", RFC 3985, 1156 DOI 10.17487/RFC3985, March 2005, . 1159 [RFC4206] Kompella, K. and Y. Rekhter, "Label Switched Paths (LSP) 1160 Hierarchy with Generalized Multi-Protocol Label Switching 1161 (GMPLS) Traffic Engineering (TE)", RFC 4206, 1162 DOI 10.17487/RFC4206, October 2005, . 1165 [RFC4385] Bryant, S., Swallow, G., Martini, L., and D. McPherson, 1166 "Pseudowire Emulation Edge-to-Edge (PWE3) Control Word for 1167 Use over an MPLS PSN", RFC 4385, DOI 10.17487/RFC4385, 1168 February 2006, . 1170 [RFC5129] Davie, B., Briscoe, B., and J. Tay, "Explicit Congestion 1171 Marking in MPLS", RFC 5129, DOI 10.17487/RFC5129, January 1172 2008, . 1174 [RFC5462] Andersson, L. and R. Asati, "Multiprotocol Label Switching 1175 (MPLS) Label Stack Entry: "EXP" Field Renamed to "Traffic 1176 Class" Field", RFC 5462, DOI 10.17487/RFC5462, February 1177 2009, . 1179 [RFC5654] Niven-Jenkins, B., Ed., Brungard, D., Ed., Betts, M., Ed., 1180 Sprecher, N., and S. Ueno, "Requirements of an MPLS 1181 Transport Profile", RFC 5654, DOI 10.17487/RFC5654, 1182 September 2009, . 1184 [RFC5659] Bocci, M. and S. Bryant, "An Architecture for Multi- 1185 Segment Pseudowire Emulation Edge-to-Edge", RFC 5659, 1186 DOI 10.17487/RFC5659, October 2009, . 1189 [RFC5921] Bocci, M., Ed., Bryant, S., Ed., Frost, D., Ed., Levrau, 1190 L., and L. Berger, "A Framework for MPLS in Transport 1191 Networks", RFC 5921, DOI 10.17487/RFC5921, July 2010, 1192 . 1194 [RFC6003] Papadimitriou, D., "Ethernet Traffic Parameters", 1195 RFC 6003, DOI 10.17487/RFC6003, October 2010, 1196 . 1198 [RFC7551] Zhang, F., Ed., Jing, R., and R. Gandhi, Ed., "RSVP-TE 1199 Extensions for Associated Bidirectional Label Switched 1200 Paths (LSPs)", RFC 7551, DOI 10.17487/RFC7551, May 2015, 1201 . 1203 [RFC7950] Bjorklund, M., Ed., "The YANG 1.1 Data Modeling Language", 1204 RFC 7950, DOI 10.17487/RFC7950, August 2016, 1205 . 1207 [RFC8040] Bierman, A., Bjorklund, M., and K. Watsen, "RESTCONF 1208 Protocol", RFC 8040, DOI 10.17487/RFC8040, January 2017, 1209 . 1211 [RFC8077] Martini, L., Ed. and G. Heron, Ed., "Pseudowire Setup and 1212 Maintenance Using the Label Distribution Protocol (LDP)", 1213 STD 84, RFC 8077, DOI 10.17487/RFC8077, February 2017, 1214 . 1216 Authors' Addresses 1218 Stewart Bryant 1219 Huawei Technologies 1221 Email: stewart.bryant@gmail.com 1223 Mach Chen 1224 Huawei Technologies 1226 Email: mach.chen@huawei.com