idnits 2.17.1 draft-ietf-detnet-dp-sol-mpls-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (June 30, 2018) is 2125 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'Network' is mentioned on line 332, but not defined == Missing Reference: 'TBD' is mentioned on line 1570, but not defined == Outdated reference: A later version (-22) exists of draft-ietf-spring-segment-routing-mpls-14 == Outdated reference: A later version (-13) exists of draft-ietf-detnet-architecture-05 Summary: 0 errors (**), 0 flaws (~~), 5 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 DetNet J. Korhonen, Ed. 3 Internet-Draft 4 Intended status: Standards Track B. Varga, Ed. 5 Expires: January 1, 2019 Ericsson 6 June 30, 2018 8 DetNet MPLS Data Plane Encapsulation 9 draft-ietf-detnet-dp-sol-mpls-00 11 Abstract 13 This document specifies Deterministic Networking data plane 14 encapsulation solutions. The described data plane solutions is 15 applied over an MPLS Packet Switched Networks. 17 Status of This Memo 19 This Internet-Draft is submitted in full conformance with the 20 provisions of BCP 78 and BCP 79. 22 Internet-Drafts are working documents of the Internet Engineering 23 Task Force (IETF). Note that other groups may also distribute 24 working documents as Internet-Drafts. The list of current Internet- 25 Drafts is at https://datatracker.ietf.org/drafts/current/. 27 Internet-Drafts are draft documents valid for a maximum of six months 28 and may be updated, replaced, or obsoleted by other documents at any 29 time. It is inappropriate to use Internet-Drafts as reference 30 material or to cite them other than as "work in progress." 32 This Internet-Draft will expire on January 1, 2019. 34 Copyright Notice 36 Copyright (c) 2018 IETF Trust and the persons identified as the 37 document authors. All rights reserved. 39 This document is subject to BCP 78 and the IETF Trust's Legal 40 Provisions Relating to IETF Documents 41 (https://trustee.ietf.org/license-info) in effect on the date of 42 publication of this document. Please review these documents 43 carefully, as they describe your rights and restrictions with respect 44 to this document. Code Components extracted from this document must 45 include Simplified BSD License text as described in Section 4.e of 46 the Trust Legal Provisions and are provided without warranty as 47 described in the Simplified BSD License. 49 Table of Contents 51 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 52 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 53 2.1. Terms used in this document . . . . . . . . . . . . . . . 4 54 2.2. Abbreviations . . . . . . . . . . . . . . . . . . . . . . 4 55 3. Requirements language . . . . . . . . . . . . . . . . . . . . 6 56 4. MPLS DetNet data plane overview . . . . . . . . . . . . . . . 6 57 4.1. DetNet data plane encapsulation requirements . . . . . . 12 58 5. DetNet encapsulation . . . . . . . . . . . . . . . . . . . . 13 59 5.1. End-system specific considerations . . . . . . . . . . . 13 60 5.2. DetNet domain specific considerations . . . . . . . . . . 15 61 5.2.1. DetNet Layer Two Service . . . . . . . . . . . . . . 15 62 5.2.2. DetNet Routing Service (IP over MPLS) . . . . . . . . 16 63 5.3. DetNet Inter-Working Function (DN-IWF) . . . . . . . . . 17 64 5.3.1. Networks with multiple technology segments . . . . . 17 65 5.3.2. DN-IWF related considerations . . . . . . . . . . . . 18 66 6. MPLS-based DetNet data plane solution . . . . . . . . . . . . 19 67 6.1. DetNet over MPLS Encapsulation Components . . . . . . . . 19 68 6.2. MPLS data plane encapsulation . . . . . . . . . . . . . . 21 69 6.3. DetNet control word . . . . . . . . . . . . . . . . . . . 22 70 6.4. Flow Identification . . . . . . . . . . . . . . . . . . . 23 71 6.5. Indication of the DetNet Payload Type . . . . . . . . . . 23 72 6.6. OAM Indication . . . . . . . . . . . . . . . . . . . . . 24 73 6.7. Flow Aggregation . . . . . . . . . . . . . . . . . . . . 24 74 6.7.1. Aggregation at the LSP . . . . . . . . . . . . . . . 25 75 6.7.2. Aggregating DetNet flows as a new DetNet flow . . . . 25 76 6.7.3. Simple Aggregation at the DetNet layer . . . . . . . 26 77 6.8. Service Layer Considerations . . . . . . . . . . . . . . 27 78 6.8.1. Edge node processing . . . . . . . . . . . . . . . . 27 79 6.8.2. Relay node processing . . . . . . . . . . . . . . . . 28 80 6.9. Other DetNet data plane considerations . . . . . . . . . 29 81 6.9.1. Class of Service . . . . . . . . . . . . . . . . . . 29 82 6.9.2. Quality of Service . . . . . . . . . . . . . . . . . 30 83 6.9.3. Cross-DetNet flow resource aggregation . . . . . . . 31 84 6.9.4. Layer 2 addressing and QoS Considerations . . . . . . 32 85 6.9.5. Time Synchronization . . . . . . . . . . . . . . . . 32 86 7. Management and control considerations . . . . . . . . . . . . 33 87 7.1. MPLS-based data plane . . . . . . . . . . . . . . . . . . 33 88 7.1.1. S-Label assignment and distribution . . . . . . . . . 33 89 7.1.2. Explicit routes . . . . . . . . . . . . . . . . . . . 33 90 7.2. Packet replication and elimination . . . . . . . . . . . 34 91 7.3. Congestion protection and latency control . . . . . . . . 35 92 7.4. Bidirectional traffic . . . . . . . . . . . . . . . . . . 35 93 7.5. Flow aggregation control . . . . . . . . . . . . . . . . 35 94 8. DetNet IP Operation over DetNet MPLS Service . . . . . . . . 35 95 9. IEEE 802.1 TSN Interconnection over DetNet MPLS Service . . . 36 96 10. DetNet MPLS Transport Layer Operation over IEEE 802.1 TSN 97 Sub-Networks . . . . . . . . . . . . . . . . . . . . . . . . 36 98 11. DetNet MPLS Transport Layer Operation over IP DetNet PSNs . . 36 99 12. Security considerations . . . . . . . . . . . . . . . . . . . 38 100 13. IANA considerations . . . . . . . . . . . . . . . . . . . . . 38 101 14. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 39 102 15. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 41 103 16. References . . . . . . . . . . . . . . . . . . . . . . . . . 41 104 16.1. Normative references . . . . . . . . . . . . . . . . . . 41 105 16.2. Informative references . . . . . . . . . . . . . . . . . 44 106 Appendix A. Example of DetNet data plane operation . . . . . . . 45 107 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 46 109 1. Introduction 111 Deterministic Networking (DetNet) is a service that can be offered by 112 a network to DetNet flows. DetNet provides these flows with a low 113 packet loss rates and assured maximum end-to-end delivery latency. 114 General background and concepts of DetNet can be found in 115 [I-D.ietf-detnet-architecture]. 117 This document specifies the DetNet data plane and the on-wire 118 encapsulation of DetNet flows over an MPLS-based Packet Switched 119 Network (PSN). The specified encapsulation provides the building 120 blocks to enable the DetNet service layer functions and allow flow 121 identification as described in the DetNet Architecture. 123 The DetNet transport layer functionality that provides congestion 124 protection for DetNet flows is assumed to be in place in a DetNet 125 node. 127 Furthermore, this document also describes how DetNet flows are 128 identified, and how a DetNet Relay/Edge/Transit nodes works. It also 129 describes the function and operation of the Packet Replication (PRF) 130 Packet Elimination (PEF) and Packet Ordering (POF) functions in the 131 MPLS data plane. 133 This document does not define the associated control plane functions, 134 or Operations, Administration, and Maintenance (OAM). It also does 135 not specify traffic handling capabilities required to deliver 136 congestion protection and latency control for DetNet flows at the 137 DetNet transport layer. 139 2. Terminology 140 2.1. Terms used in this document 142 This document uses the terminology established in the DetNet 143 architecture [I-D.ietf-detnet-architecture] and the DetNet Data Plane 144 Solution Alternatives [I-D.ietf-detnet-dp-alt]. 146 T-Label A label used to identify the LSP used to transport a 147 DetNet flow across an MPLS PSN, e.g., a hop-by-hop 148 label used between label switching routers (LSR). 150 S-Label A DetNet "service" label that is used between DetNet 151 nodes that implement also the DetNet service layer 152 functions. An S-Label is also used to identify a 153 DetNet flow at DetNet service layer. 155 PEF A Packet Elimination Function (PEF) eliminates 156 duplicate copies of packets received by an edge or a 157 relay node to prevent excess packets flooding the 158 network, or to prevent duplicate packets being sent out 159 of the DetNet domain. 161 PRF A Packet Replication Function (PRF) replicates DetNet 162 flow packets in an edge or a relay node and forwards 163 them to one or more next hops in the DetNet domain. 164 The number of packet copies sent to each next hop is a 165 DetNet Flow specific parameter at the node doing the 166 replication. 168 POF A Packet Order Function (POF) re-orders packets within 169 a DetNet flow that are received out of order. This 170 function may be implemented at an edge or a relay node. 172 PREOF Collective name for Packet Replication, Elimination, 173 and Ordering Functions. 175 d-CW A DetNet Control Word (d-CW) is used for sequencing and 176 identifying duplicate packets of a DetNet flow at the 177 DetNet service layer. 179 2.2. Abbreviations 181 The following abbreviations used in this document: 183 AC Attachment Circuit. 185 CE Customer Edge equipment. 187 CoS Class of Service. 189 CW Control Word. 191 d-CW DetNet Control Word. 193 DetNet Deterministic Networking. 195 DF DetNet Flow. 197 DN-IWF DetNet Inter-Working Function. 199 L2 Layer 2. 201 L2VPN Layer 2 Virtual Private Network. 203 L3 Layer 3. 205 LSR Label Switching Router. 207 MPLS Multiprotocol Label Switching. 209 MPLS-TE Multiprotocol Label Switching - Traffic Engineering. 211 MPLS-TP Multiprotocol Label Switching - Transport Profile. 213 MS-PW Multi-Segment PseudoWire (MS-PW). 215 NSP Native Service Processing. 217 OAM Operations, Administration, and Maintenance. 219 PE Provider Edge. 221 PEF Packet Elimination Function. 223 PRF Packet Replication Function. 225 PREOF Packet Replication, Elimination and Ordering Functions. 227 POF Packet Ordering Function. 229 PSN Packet Switched Network. 231 PW PseudoWire. 233 QoS Quality of Service. 235 S-PE Switching Provider Edge. 237 T-PE Terminating Provider Edge. 239 TSN Time-Sensitive Network. 241 3. Requirements language 243 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 244 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 245 "OPTIONAL" in this document are to be interpreted as described in BCP 246 14 [RFC2119] [RFC8174] when, and only when, they appear in all 247 capitals, as shown here. 249 4. MPLS DetNet data plane overview 251 This document describes how DetNet flows are carried over MPLS 252 networks. The DetNet Architecture, [I-D.ietf-detnet-architecture], 253 decomposes the DetNet data plane into two layers: a service layer and 254 a transport layer. The basic approach defined in this document 255 supports the DetNet service layer based on existing pseudowire (PW) 256 encapsulations and mechanisms, and supports the DetNet transport 257 layer based on existing MPLS Traffic Engineering encapsulations and 258 mechanisms. Background on PWs can be found in [RFC3985] and 259 [RFC3031]. 261 DetNet MPLS 262 . 263 . 264 +-----------+ 265 | Service | d-CW, S-Label 266 +-----------+ 267 | Transport | T-Label(s) 268 +-----------+ 269 . 270 . 272 Figure 1: DetNet adaptation to MPLS data plane 274 The MPLS DetNet data plane approach defined in this document is shown 275 in Figure 1. The service layer is supported by a DetNet control word 276 (d-CW) which conforms to the Generic PW MPLS Control Word (PWMCW) 277 defined in [RFC4385]. A d-CW identifying service label (S-Label) is 278 also used. The transport layer is supported by one or labels 279 (T-Labels). 281 TSN Edge Transit Edge TSN 282 End System Node Node Node End System 283 (T-PE) (LSR) (T-PE) 285 +---------+ +.........+ +.........+ +---------+ 286 | Appl. |<--:Svc Proxy:--End to End Svc.--:Svc Proxy:-->| Appl. | 287 +---------+ +---------+ +---------+ +---------+ 288 | TSN | |TSN| |Svc|<-- DetNet flow -->: Service :-->| TSN | 289 +---------+ +---+ +---+ +---------+ +---------+ +---------+ 290 |Transport| |Trp| |Trp| |Transport| |Trp| |Trp| |Transport| 291 +-------.-+ +--.+ +-.-+ +--.----.-+ +-.-+ +-.-+ +---.-----+ 292 : Link : / ,-----. \ : Link : / ,-----. \ 293 +........+ +-[ Sub ]-+ +........+ +-[ Sub ]-+ 294 [Network] [Network] 295 `-----' `-----' 297 |<- TSN ->| |<----- DetNet MPLS ---->| |<-- TSN --->| 299 Figure 2: A TSN over DetNet MPLS Enabled Network 301 Figure 2 shows several node types defined in 302 [I-D.ietf-detnet-architecture]. DetNet Edge Nodes sit at the 303 boundary of a DetNet domain. They are responsible for mapping non- 304 DetNet aware traffic to DetNet services. They also support the 305 imposition and disposition of the required DetNet encapsulation. 306 These are functionally similar to pseudowire (PW) Terminating 307 Provider Edge (T-PE) nodes which use MPLS-TE LSPs. 309 Transit nodes are normal MPLS Label Switching Routers (LSRs). They 310 are generally unaware of the special requirements of DetNet flows, 311 although they need to provide traffic engineering services and proper 312 QoS to the LSPs associated with DetNet flows to enhance the prospect 313 of the LSPs meeting the DetNet service requirements. Some 314 implementations of transit nodes may be DetNet aware, but such nodes 315 just support the DetNet transport layer. 317 The MPLS LSP may be provided by any MPLS method (provisioned, RSVP- 318 TE, MPLS- TP, or MPLS Segment Routing (SR)). 320 IP DetNet Relay Transit Relay IP DetNet 321 End System Node Node Node End System 322 (T-PE) (LSR) (T-PE) 323 +---------+ +---------+ 324 | Appl. |<--------------- End to End Service ---------->| Appl. | 325 +---------+ .....-----+ +-----..... +---------+ 326 | Service |<---: Service |-- DetNet flow ---| Service :-->| Service | 327 +---------+ +---------+ +---------+ +---------+ +---------+ 328 |Transport| |Trp| |Trp| |Transport| |Trp| |Trp| |Transport| 329 +-------.-+ +-.-+ +-.-+ +---.---.-+ +-.-+ +-.-+ +---.-----+ 330 : Link : / ,-----. \ : Link : / ,-----. \ 331 +........+ +-[ Sub ]-+ +........+ +-[ Sub ]-+ 332 [Network] [Network] 333 `-----' `-----' 335 |<-DN IP->| |<---- DetNet MPLS ---->| |<-DN IP->| 337 Figure 3: DetNet (DN) IP Over MPLS Network 339 Figure 3 and Figure 4, show different cases where relay nodes may be 340 used. Relay nodes are similar to edge nodes in that both are aware 341 of the needs of particular DetNet flows and take care to process them 342 in accordance with the required performance needs. They differ in 343 that relay nodes sit within a DetNet domain while edge nodes always 344 sit at DetNet domain boundaries. Both node types can enhance the 345 reliability of delivery by enabling the replication of packets so 346 that multiple copies, possibly over multiple paths are forwarded 347 through the DetNet domain. They also reduce the impact of 348 replication by eliminating surplus copies of DetNet packets. Relay 349 nodes may sit the boundary of an MPLS domain when the non-MPLS domain 350 is DetNet aware. Relay nodes are functionally similar to PW S-PEs 351 or, when at the edge of an MPLS network, T-PEs [RFC6073]. 353 Figure 4 illustrates how DetNet can provide services for IEEE 354 802.1TSN end systems, CE1 and CE2, over a DetNet enabled network. 355 The edge nodes, E1 and E2, insert and remove required DetNet data 356 plane encapsulation. The 'X' in the edge nodes and relay node, R1, 357 represent a potential DetNet flow packet replication and elimination 358 point. This conceptually parallels L2VPN services, and could 359 leverage existing related solutions as discussed below. 361 TSN |<------- End to End DetNet Service ------>| TSN 362 Service | Transit Transit | Service 363 TSN (AC) | |<-Tnl->| |<-Tnl->| | (AC) TSN 364 End | V V 1 V V 2 V V | End 365 System | +--------+ +--------+ +--------+ | System 366 +---+ | | E1 |=======| R1 |=======| E2 | | +---+ 367 | |--|----|._X_....|..DF1..|.._ _...|..DF3..|...._X_.|---|---| | 368 |CE1| | | \ | | X | | / | | |CE2| 369 | | | \_.|..DF2..|._/ \_..|..DF4..|._/ | | | 370 +---+ | |=======| |=======| | +---+ 371 ^ +--------+ +--------+ +--------+ ^ 372 | Edge Node Relay Node Edge Node | 373 | (T-PE) (S-PE) (T-PE) | 374 | | 375 |<--TSN--> <---- TSN Over DetNet MPLS ----> <--TSN-->| 376 | | 377 |<--- Emulated Time Sensitive Networking (TSN) Service --->| 379 DFx = DetNet Flow x 381 Figure 4: IEEE 802.1TSN over DetNet 383 Figure 5 illustrates how an end to end MPLS-based DetNet service is 384 provided in a more detail. In this case, the end systems, CE1 and 385 CE2, are able to send and receive DetNet flows, and R1 and R2 are 386 relay nodes as they sit in the middle of a DetNet network. For 387 example, an end system sends data encapsulated in MPLS. The 'X' in 388 the end systems, and relay nodes represents potential DetNet flow 389 packet replication and elimination points. Here the relay nodes may 390 change the underlying transport, for example tunneling MPLS over IP 391 Section 11, or simply interconnect network segments. 393 DetNet DetNet 394 MPLS Service Transit Transit Service MPLS 395 DetNet | |<-Tnl->| |<-Tnl->| | DetNet 396 End | V 1 V V 2 V | End 397 System | +--------+ +--------+ +--------+ | System 398 +---+ | | R1 |=======| R2 |=======| R3 | | +---+ 399 | X...DFa...|._X_....|..DF1..|.__ ___.|..DF3..|...._X_.|.DFa..|.X | 400 |CE1|========| \ | | X | | / |======|CE2| 401 | | | | \_.|..DF2..|._/ \__.|..DF4..|._/ | | | | 402 +---+ | |=======| |=======| | +---+ 403 ^ +--------+ +--------+ +--------+ ^ 404 | Relay Node Relay Node Relay Node | 405 | (S-PE) (S-PE) (S-PE) | 406 | | 407 |<---------------------- DetNet MPLS --------------------->| 408 | | 409 |<--------------- End to End DetNet Service -------------->| 411 Figure 5: MPLS-Based Native DetNet 413 Figure 6 illustrates how an end to end MPLS-based DetNet service is 414 provided where the end systems are not able to send and receive 415 DetNet flows. In this example, the nodes labeled CE1 and CE2 could 416 be non-DetNet aware IP routers or hosts. Note that E1 and E2 are 417 edge nodes as they sit boundaries of the DetNet enabled domain. 419 IP IP 420 Non Service Transit Transit Service Non 421 DetNet |<-Tnl->| |<-Tnl->| DetNet 422 End | V 1 V V 2 V | End 423 System | +--------+ +--------+ +--------+ | System 424 +---+ | | E1 |=======| R2 |=======| E3 | | +---+ 425 | |--------|._X_....|..DF1..|.__ ___.|..DF3..|...._X_.|------| | 426 |CE1| | | \ | | X | | / | | |CE2| 427 | | | | \_.|..DF2..|._/ \__.|..DF4..|._/ | | | | 428 +---+ | |=======| |=======| | +---+ 429 +--------+ +--------+ +--------+ 430 ^ Edge Node Relay Node Edge Node^ 431 | (T-PE) (S-PE) (T-PE) | 432 | | 433 <--IP-->| <----- IP Over DetNet MPLS ----> |<--IP--> 434 | | 435 |<------ End to End DetNet Service ------->| 437 Figure 6: MPLS-Based DetNet (non-MPLS End System) 439 Figure 7 illustrates how end to end DetNet service is provided where 440 the end systems are able to send and receive IP DetNet flows, e.g., 441 per [I-D.ietf-detnet-dp-sol-ip], and the MPLS nodes optionally 442 provide service protection. In this case R1 and R3 are T-PEs and R2 443 is an S-PE and the DetNet service is end-to-end. 445 DetNet DetNet 446 IP Service Transit Transit Service IP 447 DetNet |<-Tnl->| |<-Tnl->| DetNet 448 End | V 1 V V 2 V | End 449 System | +--------+ +--------+ +--------+ | System 450 +---+ | | R1 |=======| R2 |=======| R3 | | +---+ 451 | |--------|._X_....|..DF1..|.__ ___.|..DF3..|...._X_.|------| | 452 |CE1| | | \ | | X | | / | | |CE2| 453 | | | | \_.|..DF2..|._/ \__.|..DF4..|._/ | | | | 454 +---+ | |=======| |=======| | +---+ 455 ^ +--------+ +--------+ +--------+ ^ 456 | Relay Node Relay Node Relay Node | 457 | (T-PE) (S-PE) (T-PE) | 458 | | 459 |<-DN IP-> <------ DetNet MPLS ------> <-DN IP->| 460 | | 461 |<--------------- End to End DetNet Service -------------->| 463 Figure 7: DetNet IP over DetNet (DN) MPLS 465 An example MPLS DetNet network fragment and packet flow is 466 illustrated in Figure 8. 468 1 1.1 1.1 1.2.1 1.2.1 1.2.2 469 CE1----EN1--------R1-------R2-------R3--------EN2-----CE2 470 \ 1.2.1 / / 471 \1.2 /-----+ / 472 +------R4------------------------+ 473 1.2.2 475 Figure 8: Example Packet flow in DetNet Enabled MPLS Network 477 In Figure 8 the numbers are used to identify the instance of a 478 packet. Packet 1 is the original packet, and packets 1.1, and 1.2 479 are two first generation copies of packet 1. Packet 1.2.1 is a 480 second generation copy of packet 1.2 etc. Note that these numbers 481 never appear in the packet, and are not to be confused with sequence 482 numbers, labels or any other identifier that appears in the packet. 483 They simply indicate the generation number of the original packet so 484 that its passage through the network fragment can be identified to 485 the reader. 487 Customer Equipment CE1 sends a packet into the DetNet enabled MPLS 488 network. This is packet (1). Edge Node EN1 encapsulates the packet 489 as a DetNet Packet and sends it to Relay node R1 (packet 1.1). EN1 490 makes a copy of the packet (1.2), encapsulates it and sends this copy 491 to Relay node R4. 493 Note that along the MPLS path from EN1 to R1 there may be zero or 494 more LSRs which, for clarity, are not shown. The same is true for 495 any other path between two DetNet entities shown in Figure 8. 497 Relay node R4 has been configured to send one copy of the packet to 498 Relay Node R2 (packet 1.2.1) and one copy to Edge Node EN2 (packet 499 1.2.2). 501 R2 receives packet copy 1.2.1 before packet copy 1.1 arrives, and, 502 having been configured to perform packet elimination on this DetNet 503 flow, forwards packet 1.2.1 to Relay Node R3. Packet copy 1.1 is of 504 no further use and so is discarded by R2. 506 Edge Node EN2 receives packet copy 1.2.2 from R4 before it receives 507 packet copy 1.2.1 from R2 via relay Node R3. EN2 therefore strips 508 any DetNet encapsulation from packet copy 1.2.2 and forwards the 509 packet to CE2. When EN2 receives the later packet copy 1.2.1 this is 510 discarded. 512 The above is of course illustrative of many network scenarios that 513 can be configured. Between a pair of relay nodes there may be one or 514 more transport nodes that simply forward the DetNet traffic, but 515 these are omitted for clarity. 517 4.1. DetNet data plane encapsulation requirements 519 Two major groups of scenarios can be distinguished which require flow 520 identification during transport: 522 1. DetNet function related scenarios: 524 * Congestion protection and latency control: usage of allocated 525 resources (queuing, policing, shaping). 527 * Explicit routes: select/apply the flow specific path. 529 * Service protection: recognize DetNet compound and member flows 530 for replication an elimination. 532 2. OAM function related scenarios: 534 * troubleshooting (e.g., identify misbehaving flows, etc.) 535 * recognize flow(s) for analytics (e.g., increase counters, 536 etc.) 538 * correlate events with flows (e.g., volume above threshold, 539 etc.) 541 * etc. 543 The DetNet data plane allows for the aggregation of DetNet flows, 544 e.g., via MPLS hierarchical LSPs, to improved scaling. When DetNet 545 flows are aggregated, transit nodes may have limited ability to 546 provide service on per-flow DetNet identifiers. Therefore, 547 identifying each individual DetNet flow on a transit node may not be 548 achieved in some network scenarios, but DetNet service can still be 549 assured in these scenarios through resource allocation and control. 551 A node operating on a DetNet flow in the Detnet layer, i.e. a node 552 processing a DetNet packet which has the S-label as top of stack uses 553 the local context associated with that S-label to determine what 554 local operation(s) are applied to that packet. The S-label has to be 555 unique on each edge and relay node, which is achieved by using a 556 label taken from the platform label space [RFC3031]. 558 5. DetNet encapsulation 560 5.1. End-system specific considerations 562 Data-flows requiring DetNet service are generated and terminated on 563 end-systems. Encapsulation depends on application and its 564 preferences. In a DetNet (or even a TSN) domain the DN (TSN) 565 functions use at most two flow parameters, namely Flow-ID and 566 Sequence Number. However, an application may exchange further flow 567 related parameters (e.g., time-stamp), which are not considered by DN 568 functions. 570 Two types of end-systems are distinguished: 572 o L2 (Ethernet) end-system: application directly over L2. 574 o L3 (IP) end-system: application over L3. 576 In case of Ethernet end-systems the application data is encapsulated 577 directly in L2. From the DN domain perspective no upper layer 578 protocols are visible. The Data-flow uses only Ethernet tag(s) and 579 further flow specific parameters (if needed) are hidden inside the 580 protocol data unit (PDU). 582 The IP end-system scenario is different. Data-flows are encapsulated 583 directly in L3 (i.e., IP) and the application may use further upper 584 layer protocols (e.g., Real-time Transport Protocol (RTP)). Many 585 valid combinations exist, and it may be application specific how the 586 IP header fields are used. Also, usage of further upper layer 587 protocols depends on application requirements (e.g., time-stamp). 588 See [I-D.ietf-detnet-dp-sol-ip] more details. 590 [Editor's note: IP solution document does not really detail 591 anything beyond 6-tuple.] 593 As a general rule, DetNet domains MUST be capable of forwarding any 594 Data-flows and the DetNet domain MUST NOT mandate the end-system 595 encapsulation format. 597 Furthermore, no application-level-proxy function is envisioned inside 598 the DetNet domain, so end-systems peer with end-systems using the 599 same application encapsulation format (see figure below): 601 o L2 end-systems peer with L2 end-systems and 603 o L3 end-systems peer with L3 end-systems. 605 +-----+ 606 | X | +-----+ 607 +-----+ | X | 608 | Eth | ________ +-----+ 609 +-----+ _____ / \ | Eth | 610 \ / \__/ \___ +-----+ 611 \ / \ / 612 0======== tunnel-1 ========0_ 613 | \ 614 \ | 615 0========= tunnel-2 =========0 616 / \ __/ \ 617 +-----+ \__ DetNet domain / \ 618 | X | \ __ / +-----+ 619 +-----+ \_______/ \_____/ | X | 620 | IP | +-----+ 621 +-----+ | IP | 622 +-----+ 624 Figure 9: End-systems and the DetNet domain 626 5.2. DetNet domain specific considerations 628 From a connection type perspective, three scenarios are 629 distinguished: 631 1. Directly attached: end-system is directly connected to an edge 632 node. 634 2. Indirectly attached: end-system is behind a (L2-TSN / L3-DetNet) 635 sub-network. 637 3. DN integrated: end-system is part of the DetNet domain. 639 L3 end-systems may use any of these connection types, however L2 end- 640 systems may use only the first two (directly or indirectly attached). 641 DetNet domain MUST allow communication between any end-systems of the 642 same type (L2-L2, L3-L3), independent of their connection type and 643 DetNet capability. However directly attached and indirectly attached 644 end-systems have no knowledge about the DetNet domain and its 645 encapsulation format at all. See Figure 10 for L3 end-system 646 scenarios. 648 ____+----+ 649 +----+ _____ / | ES3| 650 | ES1|____ / \__/ +----+___ 651 +----+ \ / \ 652 + | 653 ____ \ _/ 654 +----+ __/ \ +__ DetNet domain / 655 | ES2|____/ L2/L3 |___/ \ __ __/ 656 +----+ \_______/ \_______/ \___/ 658 Figure 10: Connection types of L3 end-systems 660 5.2.1. DetNet Layer Two Service 662 The simplest DetNet service is to provide tunneling for layer two, 663 where the connected hosts are in the same broadcast (BC) domain. 664 Forwarding over the DetNet domain is based on L2 (MAC) addresses 665 (i.e. dst-MAC), or on received interface [RFC3985]. In both cases 666 the L2 headers MUST either be kept, or provision must be made for 667 their reconstruction at egress from the DetNet domain. 669 +------+ 670 | X | 671 +------+ +------+ 672 | X | | IP | 673 +------+ +------+ 674 End-system | L2 | | L2 | 675 +-----+======+-+======+--+------+ 676 DetNet tunnel | Shim | 677 +------+ 678 | MPLS | 679 +------+ 680 | L2 | 681 +------+ 683 Examples: 685 +------+ +------+ 686 | X | | X | 687 +------+ +------+ +------+ 688 | X | | IP | | IP | 689 +------+ +------+ +------+ 690 | L2 | | L2 | | L2 | 691 +-----+======+--+======+--+======+-----+ 692 | d-CW | | d-CW | | d-CW | 693 +------+ +------+ +------+ 694 | MPLS | | MPLS | | MPLS | 695 +------+ +------+ +------+ 696 | L2 | | L2 | | UDP | 697 +------+ +------+ +------+ 698 | IP | 699 +------+ 700 | L2 | 701 +------+ 703 Figure 11: Encapsulation format for DetNet Layer Two Service 705 As shown in Figure 11 both L2 and L3 end-systems can be served by 706 such a DetNet L2 encapsulation service. This encapsulation service 707 may be carried over MPLS natively Section 6.2, of over MPLS over IP 708 Section 11. 710 5.2.2. DetNet Routing Service (IP over MPLS) 712 IP traffic and IP DetNet flows, see [I-D.ietf-detnet-dp-sol-ip], can 713 be carried over a DetNet MPLS domain. In such cases, the IP headers 714 are modified per standard router behavior, e.g., TTL handling. 716 Figure 12 shows the encapsulation of an IP flow over MPLS as well as 717 when MPLS is carried over an IP PSN, see Section 11. 719 +------+ 720 | X | 721 +------+ 722 End-system | IP | 723 +-----+======+--+------+ 724 DetNet tunnel | Shim | 725 +------+ 726 | MPLS | 727 +------+ 728 | L2 | 729 +------+ 731 Examples: 733 +------+ +------+ 734 | X | | X | 735 +------+ +------+ 736 | IP | | IP | 737 +-----+======+--+======+-----+ 738 | d-CW | | d-CW | 739 +------+ +------+ 740 | MPLS | | MPLS | 741 +------+ +------+ 742 | L2 | | UDP | 743 +------+ +------+ 744 | IP | 745 +------+ 746 | L2 | 747 +------+ 749 Figure 12: Encapsulation format for DetNet Routing in MPLS PSN for L3 750 end-systems 752 5.3. DetNet Inter-Working Function (DN-IWF) 754 5.3.1. Networks with multiple technology segments 756 There are networking scenarios, where the DetNet domain contains 757 multiple technology segments (IP, MPLS, ..) and all those segments 758 are under the same administrative control (see Figure 13). 759 Furthermore, DetNet nodes may be interconnected via TSN segments. 761 An important aspect of DetNet network design is the placement of 762 DetNet functions across the domain. Designs based on segment-by- 763 segment optimization can provide only sub-optimal solutions. In 764 order to achieve global optimized Inter-Working Functions (DN-IWF) 765 can be placed at segment edge nodes, which stitch together DetNet 766 flows across connected segments. 768 DN-IWF may ensure that flow attributes are correlated across segment 769 edges. For example, there are two DetNet functions which require 770 Sequence Numbers: (1) PEF: removes duplications from flows and (2) 771 POF: ensures in-order-delivery of packet in a flow. Stitching flows 772 together and correlating attributes means for example that 773 replication of packets can happen in one segment and elimination of 774 duplicates in a different one. 776 ______ 777 ____ / \__ 778 ____ / \__/ \___ ______ 779 +----+ __/ +======+ +==+ \ +----+ 780 |src |__/ Seg1 ) | | \ Seg3 \____| dst| 781 +----+ \_______+ \ Segment-2 | \+_____/ +----+ 782 \======+_ _+===/ 783 \ __ __/ 784 \_______/ \___/ 786 +------------+ 787 +---------------E----+ | | 788 +----+ | | +----R---+ | +----+ 789 |src |-------R +---+ | E----------+ dst| 790 +----+ | | E--------+ +----+ 791 +-----------R | 792 +-----------------+ 794 Figure 13: Optimal replication and elimination placement across 795 technology segments example 797 5.3.2. DN-IWF related considerations 799 The goal of DN-IWF is to (1) match and (2) translate segment specific 800 flow attributes. The DN-IWF ensures that segment specific attributes 801 comprise per domain unique attributes for the whole DetNet domain. 802 This characteristic can ensure that DetNet functions can be based on 803 per domain attributes and not per segment attributes. 805 The two DetNet specific attributes have the following 806 characteristics: 808 o Flow-ID: it is same in all packets of a flow 810 o Sequence Number: it is different packet-by-packet 812 For the Flow-ID the DN-IWF can implement a static mapping. The 813 situation is more complicated for Sequence Number as it is different 814 packet-by-packet, so it may need more sophisticated translation 815 unless its format is exactly the same in the two technology segments. 816 In this later case the DN-IWF can simple copy the Sequence Number 817 field between the tunneling encapsulation of the two technology 818 segments. 820 In case of three technology segments (IP, MPLS and TSN) three DN-IWF 821 functions can be specified. In the rest of this section the focus is 822 on the (1) IP - MPLS network scenario. Note: the use-cases are out- 823 of-scope for (2) TSN - IP, (3) TSN - MPLS. 825 Simplest implementation of DN-IWF is provided if the flow attributes 826 have the same format. Such a common denominator of the tunnel 827 encapsulation format is the pseudowire encapsulation over both IP and 828 MPLS. 830 +--------+ 831 | | 832 | X X | 833 | ____ | 834 | / \ | 835 | | 836 |/\/\/\/\| 838 Oops! 839 404 Not Found 841 Figure 14: FIGURE Placeholder PW over X 843 [Editor's note: Where is the text describing how 802.1 TSN Streams 844 are mapping to DetNet services/flows. i.e., EVPN+] 846 6. MPLS-based DetNet data plane solution 848 6.1. DetNet over MPLS Encapsulation Components 850 To carry DetNet over MPLS the following is required: 852 1. A method of identifying the MPLS payload type. 854 2. A method of identifying the DetNet flow group to the processing 855 element. 857 3. A method of distinguishing DetNet OAM packets from DetNet data 858 packets. 860 4. A method of carrying the DetNet sequence number. 862 5. A suitable LSP to deliver the packet to the egress PE. 864 6. A method of carrying queuing and forwarding indication. 866 In this design an MPLS service label (the S-Label), similar to a 867 pseudowire (PW) label [RFC3985], is used to identify both the DetNet 868 flow identity and the payload MPLS payload type satisfying (1) and 869 (2) in the list above. OAM traffic discrimination happens through 870 the use of the Associated Channel method described in [RFC4385]. The 871 sequence number is carried in the DetNet Control word which carries 872 the Data/OAM discriminator. The LSP used to transport the DetNet 873 packet may be of any type (MPLS-LDP, MPLS-TE, MPLS-TP [RFC5921], or 874 MPLS-SR [I-D.ietf-spring-segment-routing-mpls]). The LSP (T-Label) 875 label and/or the S-Label may be used to indicate the queue processing 876 as well as the forwarding parameters. 878 To simplify implementation and to maximize interoperability two 879 sequence number sizes are supported: a 16 bit sequence number and a 880 28 bit sequence number. The 16 bit sequence number is needed to 881 support some types of legacy clients. The 28 bit sequence number is 882 used in situations where it is necessary ensure that in high speed 883 networks the sequence number space does not wrap whilst packets are 884 in flight. In addition it must be possible to send a packet with a 885 zero length sequence number, to support the case where sequence 886 numbers are not required by a particular DetNet flow. 888 Note that the concept of a zero length sequence number is not to be 889 confused with a sequence number of zero. For example, were the 890 sequence number size is 16 bits, the sequence will contain: 65535, 0, 891 1. In this case zero is an ordinary sequence number. Unlike 892 [RFC4448] a sequence number of zero does not indicate that no 893 sequence number is in use. Where sequence numbers are not in use, 894 and thus a zero length sequence number is in used, the sequence 895 number field in the packet is sent as zero. The DetNet packet 896 forwarder knows which of these cases applies through configuration 897 parameters associated with each specific DetNet flow. 899 Note that when the network consists only of DetNet enabled nodes with 900 no aggregation, Penultimate Hop Popping (PHP) means that the only 901 label in the label stack may be the S-label. 903 6.2. MPLS data plane encapsulation 905 Figure 15 illustrates a DetNet data plane MPLS encapsulation. The 906 MPLS-based encapsulation of the DetNet flows is a good fit for the 907 Layer-2 interconnect deployment cases (see Figure 4). Furthermore, 908 end to end DetNet service i.e., native DetNet deployment (see 909 Figure 5) is also possible if DetNet end systems are capable of 910 initiating and termination MPLS encapsulated packets. 912 The MPLS-based DetNet data plane encapsulation consists of: 914 o DetNet control word (d-CW) containing sequencing information for 915 packet replication and duplicate elimination purposes, and the OAM 916 indicator. There MUST be a separate sequence number space for 917 each DetNet flow. 919 o DetNet service Label (S-label) that identifies a DetNet flow to 920 the peer node that is to process it. The S-Label is allocated 921 from the platform label space [RFC3031]. 923 o Zero or more MPLS transport LSP label(s) (T-label) used to direct 924 the packet along the label switched path (LSP) to the next peer 925 node along the path. When Penultimate Hop Popping is in use there 926 may be no label T-label in the protocol stack on the final hop. 928 o The necessary data-link encapsulation is then applied prior to 929 transmission over the physical media. 931 DetNet MPLS-based encapsulation 933 +---------------------------------+ 934 | | 935 | DetNet Flow | 936 | Payload Packet | 937 | | 938 +---------------------------------+ <--\ 939 | DetNet Control Word | | 940 +---------------------------------+ +--> DetNet data plane 941 | S-Label | | MPLS encapsulation 942 +---------------------------------+ <--/ 943 | T-Label(s) | 944 +---------------------------------+ 945 | Data-Link | 946 +---------------------------------+ 947 | Physical | 948 +---------------------------------+ 950 Figure 15: Encapsulation of a DetNet flow in an MPLS(-TP) PSN 952 6.3. DetNet control word 954 A DetNet control word (d-CW) conforms to the Generic PW MPLS Control 955 Word (PWMCW) defined in [RFC4385] and is illustrated in Figure 16. 956 The upper nibble of the d-CW MUST be set to zero (0). Two sequence 957 number sizes are supported: 16 bits and 28 bits. The sequence number 958 size in use for the d-CW associated with a DetNet flow (S-Label) is 959 configured either by a control plane or manually for each DetNet 960 flow. The sequence number is aligned to the right (least significant 961 bits) and unused bits MUST be set to zero (0). Each DetNet flow MUST 962 have its own sequence number counter. The sequence number is 963 incremented by one for each new packet. 965 As discussed in Section 6, zero is an ordinary sequence number with 966 no special meaning. Also as discussed therein, where no sequence 967 number is used by a particular DetNet flow, the sequence number field 968 in the d-CW is set to zero. 970 The d-CW MUST always be present in a packet. In a case where the 971 sequence number is not used (e.g., for DetNet-t-flows) a zero length 972 sequence number is used and the sequence number MUST be set to zero 973 (0). 975 0 1 2 3 976 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 977 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 978 |0 0 0 0| Sequence Number | 979 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 981 Figure 16: DetNet Control Word 983 6.4. Flow Identification 985 DetNet flow identification at a DetNet service layer is realized by 986 an S-label. The S-label is allocated from the platform label space 987 [RFC3031] which means that the DetNet flow is correctly identified 988 and matched to the flow parameters, including the flow history, 989 regardless of which input interface the packet arrives on. The 990 S-label MUST be at the bottom label of the label stack for a DetNet- 991 s- or DetNet-st-flow and MUST precede the d-CW. 993 The S-label for a specific DetNet flow is unique to that DetNet flow 994 on a specific node, but is not required to be identical with the 995 S-label for that DetNet flow in any other node within the DetNet 996 domain. Thus the S-label can only be used to identify the DetNet 997 flow at the intended receiving node. 999 6.5. Indication of the DetNet Payload Type 1001 The only nodes that needs to know the payload type of a flow are the 1002 DetNet ingress node and the DetNet egress nodes. The ingress node 1003 has to know how to process the packet it receives from the ingress AC 1004 or IP flow, and the egress edge node has to know how to prepare the 1005 packet for transmission to the next hop. 1007 On ingress a DetNet edge node has to classify the packets into those 1008 that are for transmission as Detnet packets and those that are for 1009 transmission as "normal" packets at one of more lower priorities. 1010 The packet type is indicated to the egress edge node through the 1011 value of the S-label. Thus, when the egress edge node looks up the 1012 S-label one of the parameters returned is the packet type which in 1013 turn tells the egress edge node how to prepare the packet for 1014 transmission to a next hop. 1016 The consequence of this approach is that if multiple packet 1017 encapsulations are processed on a node pair, each encapsulation will 1018 need its own S-Label. That is not generally a problems, since it is 1019 anticipated that only one encapsulation type will be present for each 1020 DetNet flow. Of course, if for some reason the multiple 1021 encapsulations are needed to support a single DetNet service, 1022 multiple S-labels will be required for that service. Note that in 1023 the unlikely case that Ipv4 and IPv6 will map to the same DetNet 1024 flow, different S-labels will be needed to differentiate between the 1025 versions of IP. 1027 6.6. OAM Indication 1029 OAM follows the procedures set out in [RFC5085] with the restriction 1030 that only Virtual Circuit Connectivity Verification (VCCV) type 1 is 1031 supported. 1033 As shown in Figure 3 of [RFC5085] when the first nibble of the d-CW 1034 is 0x0 the payload following the d-CW is normal user data. However, 1035 when the first nibble of the d-CW is 0X1, the payload that follows 1036 the d-DW is an OAM payload with the OAM type indicated by the value 1037 in the d-CW Channel Type field. 1039 The reader is referred to [RFC5085] for a more detailed description 1040 of the Associated Channel mechanism, and to the DetNet work on OAM 1041 for more information DetNet OAM. 1043 6.7. Flow Aggregation 1045 1. Aggregate at the LSP (Transport) 1047 2. Aggregating DetNet flows as a new DetNet flow 1049 3. Simple Aggregation at the DetNet layer 1051 A further method of using SR to perform aggregation is for further 1052 study. 1054 The resource control and management aspects of aggregation (including 1055 the queuing/shaping/ policing implications) will be covered in other 1056 documents. 1058 The ability to aggregate individual flows, and their associated 1059 resource control, into a larger aggregate is an important technique 1060 for improving scaling of control in the data, management and control 1061 planes. The DetNet data plane allows for the aggregation of DetNet 1062 flows, to improved scaling. There are three methods of introducing 1063 flow aggregation: 1065 The following review comments were received when this section was 1066 committed to github. 1068 General comment: We should points to the major issue of aggregation, 1069 namely the Seq.Num related problem. The aggregated flows have their 1070 own Seq.Num and those are independent. We should consider to group 1071 the aggregation techniques as per their impact on what DetNet 1072 functions they allow on a DetNet flow. (E.g., aggregation without 1073 new Aggregate.Seq.Num would prohibit usage of FR, EF and in-order- 1074 delivery function on the aggregate flow). 1076 SR based aggregation can be treated as a form of H-LSP aggregation. 1077 Should we differentiate them? What are the differences? 1079 What are the issues when aggregating of different payload types? 1080 Should we add an editor note on this? 1082 Simple-aggregation-at-the-detnet-layer: is this not the same as 1083 H-LSP? The A-label can be treated just as an additional T-label. 1085 End of review comment. 1087 6.7.1. Aggregation at the LSP 1089 DetNet flows transported via MPLS can leverage MPLS-TE?s existing 1090 support for hierarchical LSPs (H-LSPs), see [RFC4206]. H-LSPs are 1091 typically used to aggregate control and resources, they may also be 1092 used to provide OAM or protection for the aggregated LSPs. Arbitrary 1093 levels of aggregation naturally falls out of the definition for 1094 hierarchy and the MPLS label stack [RFC3032]. DetNet nodes which 1095 support aggregation (LSP hierarchy) map one or more LSPs (labels) 1096 into and from an H-LSP. Both carried LSPs and H-LSPs may or may not 1097 use the TC field, i.e., L-LSPs or E-LSPs. Such nodes will need to 1098 ensure that traffic from aggregated LSPs are placed (shaped/policed/ 1099 enqueued) onto the H-LSPs in a fashion that ensures the required 1100 DetNet service is preserved. 1102 Additional details of the traffic control capabilities needed at a 1103 DetNet-aware node may be covered in the new service descriptions 1104 mentioned above or in separate future documents. Management and 1105 control plane mechanisms will also need to ensure that the service 1106 required on the aggregate flow (H-LSP or DSCP) are provided, which 1107 may include the discarding or remarking mentioned in the previous 1108 sections. 1110 6.7.2. Aggregating DetNet flows as a new DetNet flow 1112 An aggregate can be built by layering DetNet flows as shown below: 1114 +---------------------------------+ 1115 | | 1116 | DetNet Flow | 1117 | Payload Packet | 1118 | | 1119 +---------------------------------+ <--\ 1120 | DetNet Control Word | | 1121 +=================================+ | 1122 | S-Label | | 1123 +---------------------------------+ +----DetNet data plane 1124 | DetNet Control Word | | MPLS encapsulation 1125 +=================================+ | 1126 | A-Label | | 1127 +---------------------------------+ <--/ 1128 | T-Label(s) | 1129 +---------------------------------+ 1130 | Data-Link | 1131 +---------------------------------+ 1132 | Physical | 1133 +---------------------------------+ 1135 Both the Aggregation (A) label and the S-label have their MPLS S bit 1136 set indicating bottom of stack, and the d-CW allows the PREOF to 1137 work. 1139 It is a property of the A-label that what follows is d-CW followed by 1140 an S-label. A relay node processing the A-label would not know the 1141 underlying payload type. This would only be known to a node that was 1142 a peer of the node imposing the S-label. However there is no real 1143 need for it to know the payload type during aggregation processing. 1145 6.7.3. Simple Aggregation at the DetNet layer 1147 Another approach would be not to include a d-CW for the aggregated 1148 flow. This would be functionally similar to aggregation at the 1149 transport layer using H-LSPs, but would confine knowledge of the 1150 aggregation to the DetNet layer. Such an approach shares the 1151 disadvantage that PREOF operations would not be possible. OAM 1152 operation in this mode is for further study. 1154 +---------------------------------+ 1155 | | 1156 | DetNet Flow | 1157 | Payload Packet | 1158 | | 1159 +---------------------------------+ <--\ 1160 | DetNet Control Word | | 1161 +=================================+ | 1162 | S-Label | +----DetNet data plane 1163 +---------------------------------+ | MPLS encapsulation 1164 | A-Label | | 1165 +---------------------------------+ <--/ 1166 | T-Label(s) | 1167 +---------------------------------+ 1168 | Data-Link | 1169 +---------------------------------+ 1170 | Physical | 1171 +---------------------------------+ 1173 6.8. Service Layer Considerations 1175 The edge and relay node internal procedures related to PREOF are 1176 implementation specific. The order of a packet elimination or 1177 replication is out of scope in this specification. However, care 1178 should be taken that the replication function does not actually 1179 loopback packets as "replicas". Looped back packets include 1180 artificial delay when the node that originally initiated the packet 1181 receives it again. Also, looped back packets may make the network 1182 condition to look healthier than it actually is (in some cases link 1183 failures are not reflected properly because looped back packets make 1184 the situation appear better than it actually is). 1186 It is important that the DetNet layer is configured such that a 1187 DetNet node never receives its own replicated packets. If it were to 1188 receive such packets the replication function would make the loop 1189 more destructive of bandwidth than a conventional unicast loop. 1190 Ultimately the TTL in the S-Label will cause the packet to die during 1191 a transient, but given the sensitivity of applications to packet 1192 latency the impact on the DetNet application would be severe. 1194 6.8.1. Edge node processing 1196 An edge node is resposable for matching ingress packets to the 1197 service they require and encapsulating them accordingly.An edge node 1198 may participate in the packet replication and duplication 1199 elimination. 1201 The DetNet-aware forwarder selects the egress DetNet member flow 1202 segment based on the flow identification. The mapping of ingress 1203 DetNet member flow segment to egress DetNet member flow segment may 1204 be statically or dynamically configured. Additionally the DetNet- 1205 aware forwarder does duplicate frame elimination based on the flow 1206 identification and the sequence number combination. The packet 1207 replication is also done within the DetNet-aware forwarder. During 1208 elimination and the replication process the sequence number of the 1209 DetNet member flow MUST be preserved and copied to the egress DetNet 1210 member flow. 1212 The internal design of a relay node is out of scope of this document. 1213 However the reader's attention is drawn to the need to make any PREOF 1214 state available to the packet processor(s) dealing with packets to 1215 which the PREOF functions must be applied, and to maintain that state 1216 is such as way that it is available to the packet processor operation 1217 on the next packet in the DetNet flow (which may be a duplicate, a 1218 late packet, or the next packet in sequence. 1220 [Editor's note: I think the rest of this section belongs in a new 1221 "802.1 TSN (island Interconnect) over MPLS DetNet" section.] 1223 This may be done in the DetNet layer, or where the native service 1224 processing (NSP) [RFC3985] is IEEE 802.1CB [IEEE8021CB] capable, the 1225 packet replication and duplicate elimination MAY entirely be done in 1226 the NSP, bypassing the DetNet flow encapsulation and logic entirely. 1227 This enables operating over unmodified implementations and 1228 deployments. The NSP approach works only between edge nodes and 1229 cannot make use of relay nodes. 1231 The NSP approach is useful end to end tunnel and for for "island 1232 interconnect" scenarios. However, when there is a need to do PREOF 1233 in a middle of the network, such plain edge to edge operation is not 1234 sufficient. 1236 The extended forwarder MAY copy the sequencing information from the 1237 native DetNet packet into the DetNet sequence number field and vice 1238 versa. If there is no existing sequencing information available in 1239 the native packet or the forwarder chose not to copy it from the 1240 native packet, then the extended forwarder MUST maintain a sequence 1241 number counter for each DetNet flow (indexed by the DetNet flow 1242 identification). 1244 6.8.2. Relay node processing 1246 A DetNet Relay node operates in the DetNet transport layer . This 1247 processing is done within an extended forwarder function. Whether an 1248 ingress DetNet member flow receives DetNet specific processing 1249 depends on how the forwarding is programmed. Some relay nodes may be 1250 DetNet service aware, while others may be unmodified LSRs that only 1251 understand how to swicth MPLS-TE LSPs. 1253 It is also possible to treat the relay node as a transit node, see 1254 Section 6.9.3. Again, this is entirely up to how the forwarding has 1255 been programmed. 1257 6.9. Other DetNet data plane considerations 1259 6.9.1. Class of Service 1261 [Editor's note: this section needs to updated to discuss how DetNet 1262 service is mapped to E- and L-LSPs. Perhaps this gets merged with 1263 the aggregation section or dropped?] 1265 Class and quality of service, i.e., CoS and QoS, are terms that are 1266 often used interchangeably and confused with each other. In the 1267 context of DetNet, CoS is used to refer to mechanisms that provide 1268 traffic forwarding treatment based on aggregate group basis and QoS 1269 is used to refer to mechanisms that provide traffic forwarding 1270 treatment based on a specific DetNet flow basis. Examples of 1271 existing network level CoS mechanisms include DiffServ which is 1272 enabled by IP header differentiated services code point (DSCP) field 1273 [RFC2474] and MPLS label traffic class field [RFC5462], and at Layer- 1274 2, by IEEE 802.1p priority code point (PCP). 1276 CoS for DetNet flows carried in PWs and MPLS is provided using the 1277 existing MPLS Differentiated Services (DiffServ) architecture 1278 [RFC3270]. Both E-LSP and L-LSP MPLS DiffServ modes MAY be used to 1279 support DetNet flows. The Traffic Class field (formerly the EXP 1280 field) of an MPLS label follows the definition of [RFC5462] and 1281 [RFC3270]. The Uniform, Pipe, and Short Pipe DiffServ tunneling and 1282 TTL processing models are described in [RFC3270] and [RFC3443] and 1283 MAY be used for MPLS LSPs supporting DetNet flows. MPLS ECN MAY also 1284 be used as defined in ECN [RFC5129] and updated by [RFC5462]. 1286 CoS for DetNet flows carried in IPv6 is provided using the standard 1287 differentiated services code point (DSCP) field [RFC2474] and related 1288 mechanisms. The 2-bit explicit congestion notification (ECN) 1289 [RFC3168] field MAY also be used. 1291 One additional consideration for DetNet nodes which support CoS 1292 services is that they MUST ensure that the CoS service classes do not 1293 impact the congestion protection and latency control mechanisms used 1294 to provide DetNet QoS. This requirement is similar to requirement 1295 for MPLS LSRs to that CoS LSPs do not impact the resources allocated 1296 to TE LSPs via [RFC3473]. 1298 6.9.2. Quality of Service 1300 Quality of Service (QoS) mechanisms for flow specific traffic 1301 treatment typically includes a guarantee/agreement for the service, 1302 and allocation of resources to support the service. Example QoS 1303 mechanisms include discrete resource allocation, admission control, 1304 flow identification and isolation, and sometimes path control, 1305 traffic protection, shaping, policing and remarking. Example 1306 protocols that support QoS control include Resource ReSerVation 1307 Protocol (RSVP) [RFC2205] (RSVP) and RSVP-TE [RFC3209] and [RFC3473]. 1308 The existing MPLS mechanisms defined to support CoS [RFC3270] can 1309 also be used to reserve resources for specific traffic classes. 1311 In addition to explicit routes, and packet replication and 1312 elimination, described in Section 6 above, DetNet provides zero 1313 congestion loss and bounded latency and jitter. As described in 1314 [I-D.ietf-detnet-architecture], there are different mechanisms that 1315 maybe used separately or in combination to deliver a zero congestion 1316 loss service. These mechanisms are provided by the either the MPLS 1317 or IP layers, and may be combined with the mechanisms defined by the 1318 underlying network layer such as 802.1TSN. 1320 A baseline set of QoS capabilities for DetNet flows carried in PWs 1321 and MPLS can provided by MPLS with Traffic Engineering (MPLS-TE) 1322 [RFC3209] and [RFC3473]. TE LSPs can also support explicit routes 1323 (path pinning). Current service definitions for packet TE LSPs can 1324 be found in "Specification of the Controlled Load Quality of 1325 Service", [RFC2211], "Specification of Guaranteed Quality of 1326 Service", [RFC2212], and "Ethernet Traffic Parameters", [RFC6003]. 1327 Additional service definitions are expected in future documents to 1328 support the full range of DetNet services. In all cases, the 1329 existing label-based marking mechanisms defined for TE-LSPs and even 1330 E-LSPs are use to support the identification of flows requiring 1331 DetNet QoS. 1333 Packets that are marked with a DetNet Class of Service value, but 1334 that have not been the subject of a completed reservation, can 1335 disrupt the QoS offered to properly reserved DetNet flows by using 1336 resources allocated to the reserved flows. Therefore, the network 1337 nodes of a DetNet network: 1339 o MUST defend the DetNet QoS by discarding or remarking (to a non- 1340 DetNet CoS) packets received that are not the subject of a 1341 completed reservation. 1343 o MUST NOT use a DetNet reserved resource, e.g. a queue or shaper 1344 reserved for DetNet flows, for any packet that does not carry a 1345 DetNet Class of Service marker. 1347 6.9.3. Cross-DetNet flow resource aggregation 1349 [Editor's NOTE: keep and extend this section.] 1351 The ability to aggregate individual flows, and their associated 1352 resource control, into a larger aggregate is an important technique 1353 for improving scaling of control in the data, management and control 1354 planes. This document identifies the traffic identification related 1355 aspects of aggregation of DetNet flows. The resource control and 1356 management aspects of aggregation (including the queuing/shaping/ 1357 policing implications) will be covered in other documents. The data 1358 plane implications of aggregation are independent for PW/MPLS and IP 1359 encapsulated DetNet flows. 1361 DetNet flows transported via MPLS can leverage MPLS-TE's existing 1362 support for hierarchical LSPs (H-LSPs), see [RFC4206]. H-LSPs are 1363 typically used to aggregate control and resources, they may also be 1364 used to provide OAM or protection for the aggregated LSPs. Arbitrary 1365 levels of aggregation naturally falls out of the definition for 1366 hierarchy and the MPLS label stack [RFC3032]. DetNet nodes which 1367 support aggregation (LSP hierarchy) map one or more LSPs (labels) 1368 into and from an H-LSP. Both carried LSPs and H-LSPs may or may not 1369 use the TC field, i.e., L-LSPs or E-LSPs. Such nodes will need to 1370 ensure that traffic from aggregated LSPs are placed (shaped/policed/ 1371 enqueued) onto the H-LSPs in a fashion that ensures the required 1372 DetNet service is preserved. 1374 DetNet flows transported via IP have more limited aggregation 1375 options, due to the available traffic flow identification fields of 1376 the IP solution. One available approach is to manage the resources 1377 associated with a DSCP identified traffic class and to map (remark) 1378 individually controlled DetNet flows onto that traffic class. This 1379 approach also requires that nodes support aggregation ensure that 1380 traffic from aggregated LSPs are placed (shaped/policed/enqueued) in 1381 a fashion that ensures the required DetNet service is preserved. 1383 In both the MPLS and IP cases, additional details of the traffic 1384 control capabilities needed at a DetNet-aware node may be covered in 1385 the new service descriptions mentioned above or in separate future 1386 documents. Management and control plane mechanisms will also need to 1387 ensure that the service required on the aggregate flow (H-LSP or 1388 DSCP) are provided, which may include the discarding or remarking 1389 mentioned in the previous sections. 1391 6.9.4. Layer 2 addressing and QoS Considerations 1393 [Editor's NOTE: review and simplify this section.] 1395 The Time-Sensitive Networking (TSN) Task Group of the IEEE 802.1 1396 Working Group have defined (and are defining) a number of amendments 1397 to IEEE 802.1Q [IEEE8021Q] that provide zero congestion loss and 1398 bounded latency in bridged networks. IEEE 802.1CB [IEEE8021CB] 1399 defines packet replication and elimination functions that should 1400 prove both compatible with and useful to, DetNet networks. 1402 As is the case for DetNet, a Layer 2 network node such as a bridge 1403 may need to identify the specific DetNet flow to which a packet 1404 belongs in order to provide the TSN/DetNet QoS for that packet. It 1405 also will likely need a CoS marking, such as the priority field of an 1406 IEEE Std 802.1Q VLAN tag, to give the packet proper service. 1408 Although the flow identification methods described in IEEE 802.1CB 1409 [IEEE8021CB] are flexible, and in fact, include IP 5-tuple 1410 identification methods, the baseline TSN standards assume that every 1411 Ethernet frame belonging to a TSN stream (i.e. DetNet flow) carries 1412 a multicast destination MAC address that is unique to that flow 1413 within the bridged network over which it is carried. Furthermore, 1414 IEEE 802.1CB [IEEE8021CB] describes three methods by which a packet 1415 sequence number can be encoded in an Ethernet frame. 1417 Ensuring that the proper Ethernet VLAN tag priority and destination 1418 MAC address are used on a DetNet/TSN packet may require further 1419 clarification of the customary L2/L3 transformations carried out by 1420 routers and edge label switches. Edge nodes may also have to move 1421 sequence number fields among Layer 2, PW, and IPv6 encapsulations. 1423 6.9.5. Time Synchronization 1425 [Editor's Note: A detailed discussion of time synchronization is 1426 outside the scope of this document, and the production of a 1427 specialist text discussing this topic is encouraged. This section 1428 will be updated/removed if such a document is available before 1429 publication of this text.] 1431 Time synchronization is important both from the perspective of 1432 operating the DetNet network itself and from the perspective of 1433 transferring time across the network between client applications. 1434 Some clients may be able to use the DetNet as their provider of time 1435 and frequency, others may require the DetNet to transfer time between 1436 a client clock source and a client clock user. 1438 The reader's attention is drawn to [RFC8169] which describes a method 1439 of recording the packet queuing time in an MPLS LSR on a packet by 1440 per packet basis and forwarding this information to the egress edge 1441 system. This allows compensation for any variable packet queuing 1442 delay to be applied at the packet receiver. The mechanism described 1443 in [RFC8169] may have wider application than basic time transfer in a 1444 DetNet. 1446 A more detailed discussion of time synchronization is outside the 1447 scope of this document. 1449 7. Management and control considerations 1451 [Editor's note: This section needs to be different for MPLS and IP 1452 solutions. Most solutions are technology dependant. Currently most 1453 text in this section is just a draft and may have bits that are 1454 already moved to other places/documents.] 1456 While management plane and control planes are traditionally 1457 considered separately, from the Data Plane perspective there is no 1458 practical difference based on the origin of flow provisioning 1459 information. This document therefore does not distinguish between 1460 information provided by a control plane protocol, e.g., RSVP-TE 1461 [RFC3209] and [RFC3473], or by a network management mechanisms, e.g., 1462 RestConf [RFC8040] and YANG [RFC7950]. 1464 [Editor's note: This section is a work in progress. discuss here 1465 what kind of enhancements are needed for DetNet and specifically for 1466 PREOF and DetNet zero congest loss and latency control. Need to 1467 cover both traffic control (queuing) and connection control (control 1468 plane).] 1470 7.1. MPLS-based data plane 1472 7.1.1. S-Label assignment and distribution 1474 [Editor's note: Outdated and needs more work.] 1476 The DetNet S-Label distribution follows the same mechanisms specified 1477 for XYZ . The details of the control plane protocol solution required 1478 for the label distribution and the management of the label number 1479 space are out of scope of this document. 1481 7.1.2. Explicit routes 1483 It is necessary to consider explicit routes both at the DetNet layer 1484 and in the MPLS layer. In the DetNet layer the explicit route 1485 consists of the set of Relay Nodes that the DetNet flow must 1486 traverse. In the MPLS layer the explicit route consists of the set 1487 of LSRs, links, and possibly link bundle members and queues that the 1488 DetNet packets of a flow must traverse between nodes in the DetNet 1489 layer (i.e. between a specific Edge Node and the next hop Relay Node, 1490 between specific Relay Nodes, and between a specific Relay node and 1491 the egress Edge Node. This detailed steering is needed to ensure 1492 that packets are routed through the resources that have been reserved 1493 for them, and hence provide the DetNet application with the required 1494 performance. 1496 Whether configuring, calculating and instantiating this is a multi- 1497 stage process, or a single stage process is out of scope of this 1498 document. 1500 The one method of explicitly setting up the explicit path at the 1501 DetNet layer is through the use of the management controller. 1503 [Editor's note: a method of setting up a graph through the DetNet 1504 Nodes using the IGP has been proposed. A reference is needed to 1505 e.g., RFC 7813 IS-IS Path Control and Reservation.] 1507 There are a number of approaches that can be taken to provide 1508 explicit routes/paths in the MPLS layer: 1510 o The path can be explicitly set up by the management controller 1511 calculating the path and explicitly configuring each node along 1512 that path. 1514 o The LSP can be set up using RSVP-TE. Such an approach confines 1515 the packet to the explicit path. 1517 o The path can be implemented using segment routing. 1519 Where the DetNet traffic is carried over IP Section 11 explicit paths 1520 may need to be provided in the IP layer. This is for further study. 1522 7.2. Packet replication and elimination 1524 [Editor's note: Outdated and at the functional level technology 1525 independent.. but needs more work.] 1527 The control plane protocol solution required for managing the PREOF 1528 processing is outside the scope of this document. 1530 7.3. Congestion protection and latency control 1532 [Editor's note: TBD] 1534 7.4. Bidirectional traffic 1536 [Editor's NOTE: this section needs to be updated to have its scope 1537 limited to management and control.] 1539 Some DetNet applications generate bidirectional traffic. Using MPLS 1540 definitions [RFC5654] there are associated bidirectional flows, and 1541 co-routed bidirectional flows. MPLS defines a point-to-point 1542 associated bidirectional LSP as consisting of two unidirectional 1543 point-to-point LSPs, one from A to B and the other from B to A, which 1544 are regarded as providing a single logical bidirectional transport 1545 path. This would be analogous of standard IP routing, or PWs running 1546 over two reciprocal unidirection LSPs. MPLS defines a point-to-point 1547 co-routed bidirectional LSP as an associated bidirectional LSP which 1548 satisfies the additional constraint that its two unidirectional 1549 component LSPs follow the same path (in terms of both nodes and 1550 links) in both directions. An important property of co-routed 1551 bidirectional LSPs is that their unidirectional component LSPs share 1552 fate. In both types of bidirectional LSPs, resource allocations may 1553 differ in each direction. The concepts of associated bidirectional 1554 flows and co-routed bidirectional flows can be applied to DetNet 1555 flows as well whether IPv6 or MPLS is used. 1557 While the IPv6 and MPLS data planes must support bidirectional DetNet 1558 flows, there are no special bidirectional features with respect to 1559 the data plane other than need for the two directions take the same 1560 paths. Fate sharing and associated vs co-routed bidirectional flows 1561 can be managed at the control level. Note, that there is no stated 1562 requirement for bidirectional DetNet flows to be supported using the 1563 same IPv6 Flow Labels or MPLS Labels in each direction. Control 1564 mechanisms will need to support such bidirectional flows for both 1565 IPv6 and MPLS, but such mechanisms are out of scope of this document. 1566 An example control plane solution for MPLS can be found in [RFC7551]. 1568 7.5. Flow aggregation control 1570 [TBD] 1572 8. DetNet IP Operation over DetNet MPLS Service 1574 [Editor's note: this is a place holder section. A standalone section 1575 on operation of IP flows over DetNet MPLS data plane. Includes 1576 RFC2119 Language.] 1578 9. IEEE 802.1 TSN Interconnection over DetNet MPLS Service 1580 [Editor's note: this is a place holder section. A standalone section 1581 on TSN "island" interconnect over DetNet". Includes RFC2119 1582 Language.] 1584 10. DetNet MPLS Transport Layer Operation over IEEE 802.1 TSN Sub- 1585 Networks 1587 [Editor's note: this is a place holder section. A standalone section 1588 on MPLS over IEEE 802.1 TSN. Includes RFC2119 Language.] 1590 11. DetNet MPLS Transport Layer Operation over IP DetNet PSNs 1592 This section specifies the DetNet encapsulation over an IP transport 1593 network. The approach is modeled on the operation of MPLS and 1594 PseudoWires (PW) over an IP Packet Switched Network (PSN) 1595 [RFC3985][RFC4385][RFC7510]. It is also based on the MPLS data plane 1596 encapsulation described in Section 6.2. 1598 To carry DetNet with full functionality at the DetNet layer over an 1599 IP transport network, the following components are required (these 1600 are a subset of the requirements for MPLS encapsulation listed in 1601 Section 6.1): 1603 1. A method of identifying the DetNet flow group to the processing 1604 element. 1606 2. A method of carrying the DetNet sequence number. 1608 3. A method of distinguishing DetNet OAM packets from DetNet data 1609 packets. 1611 4. A method of carrying queuing and forwarding indication. 1613 These requirements are satisfied by the DetNet over MPLS 1614 Encapsulation described in Section 6.2. 1616 To simplify operations and implementations, rather than inventing a 1617 new encapsulation, the IP encapsulation takes advantage of the MPLS 1618 encapsulation. By using the specification of MPLS over UDP and IP in 1619 [RFC7510], the T-Label(s) shown in Figure 15 in Section 6.2 can be 1620 replaced by UDP and IP, resulting in the following encapsulation: 1622 +---------------------------------+ 1623 | | 1624 | DetNet Flow | 1625 | Payload Packet | 1626 | | 1627 +---------------------------------+ <--\ 1628 | DetNet Control Word | | 1629 +---------------------------------+ +--> DetNet data plane 1630 | S-Label | | MPLS encapsulation 1631 +---------------------------------+ <--/ 1632 | UDP Header | 1633 +---------------------------------+ 1634 | IP Header | 1635 +---------------------------------+ 1636 | Data-Link | 1637 +---------------------------------+ 1638 | Physical | 1639 +---------------------------------+ 1641 Figure 17: IP Encapsulation of DetNet 1643 Where the UDP header is used as defined in Section 3 of [RFC7510]. 1645 As in Section 6.2, the S-Label is used to identify a DetNet flow to 1646 the peer node that processes it, in this case the node addressed by 1647 the IP Header in Figure 17. The S-Label is allocated from the 1648 receiving node?s platform label space [RFC3031]. 1650 In ingress Edge Nodes, the encapsulation in Figure 17 will be imposed 1651 on Detnet Flow Payload Packets as received from DetNet End Systems, 1652 and the encapsulation will be removed in egress Edge Nodes as they 1653 transmit the Payload Packets to the End Systems. 1655 Note that this encapsulation works equally well with IPv4 and IPv6. 1657 This encapsulation can also be used in conjunction with segment 1658 routing as specified in [I-D.ietf-spring-segment-routing-mpls]. In 1659 this case, the T-Label(s) in Figure 17 should be retained, and at 1660 each hop, the top T-label is popped and mapped to a corresponding 1661 UDP/IP tunnel, resulting in the following encapsulation: 1663 +---------------------------------+ 1664 | | 1665 | DetNet Flow | 1666 | Payload Packet | 1667 | | 1668 +---------------------------------+ <--\ 1669 | DetNet Control Word | | 1670 +---------------------------------+ +--> DetNet data plane 1671 | S-Label | | MPLS encapsulation 1672 +---------------------------------+ <--/ 1673 | T-Label(s) | 1674 +---------------------------------+ 1675 | UDP Header | 1676 +---------------------------------+ 1677 | IP Header | 1678 +---------------------------------+ 1679 | Data-Link | 1680 +---------------------------------+ 1681 | Physical | 1682 +---------------------------------+ 1684 Figure 18: IP Encapsulation of DetNet with MPLS-SR 1686 Again, the UDP header is used as defined in Section 3 of [RFC7510]. 1688 Note that if required in both the case of IP Encapsulation of DetNet 1689 Figure 17, and of IP Encapsulation of DetNet with MPLS-SR Figure 18, 1690 it is possible to omit the UDP header if required. Operation of MPLS 1691 directly over IP is described in [RFC4023]. In this case DetNet 1692 Service can be provided on a per IP flow basis as described in 1693 [I-D.ietf-detnet-dp-sol-ip]. 1695 12. Security considerations 1697 The security considerations of DetNet in general are discussed in 1698 [I-D.ietf-detnet-architecture] and [I-D.sdt-detnet-security]. Other 1699 security considerations will be added in a future version of this 1700 draft. 1702 13. IANA considerations 1704 This document makes no IANA requests. 1706 14. Contributors 1708 RFC7322 limits the number of authors listed on the front page of a 1709 draft to a maximum of 5, far fewer than the 20 individuals below who 1710 made important contributions to this draft. The editor wishes to 1711 thank and acknowledge each of the following authors for contributing 1712 text to this draft. See also Section 15. 1714 Loa Andersson 1715 Huawei 1716 Email: loa@pi.nu 1718 Yuanlong Jiang 1719 Huawei 1720 Email: jiangyuanlong@huawei.com 1722 Norman Finn 1723 Huawei 1724 3101 Rio Way 1725 Spring Valley, CA 91977 1726 USA 1727 Email: norman.finn@mail01.huawei.com 1729 Janos Farkas 1730 Ericsson 1731 Magyar Tudosok krt. 11. 1732 Budapest 1117 1733 Hungary 1734 Email: janos.farkas@ericsson.com 1736 Carlos J. Bernardos 1737 Universidad Carlos III de Madrid 1738 Av. Universidad, 30 1739 Leganes, Madrid 28911 1740 Spain 1741 Email: cjbc@it.uc3m.es 1743 Tal Mizrahi 1744 Marvell 1745 6 Hamada st. 1746 Yokneam 1747 Israel 1748 Email: talmi@marvell.com 1750 Lou Berger 1751 LabN Consulting, L.L.C. 1752 Email: lberger@labn.net 1754 Stewart Bryant 1755 Huawei Technologies 1756 Email: stewart.bryant@gmail.com 1758 Mach Chen 1759 Huawei Technologies 1760 Email: mach.chen@huawei.com 1762 15. Acknowledgements 1764 The author(s) ACK and NACK. 1766 The following people were part of the DetNet Data Plane Solution 1767 Design Team: 1769 Jouni Korhonen 1771 Janos Farkas 1773 Norman Finn 1775 Balazs Varga 1777 Loa Andersson 1779 Tal Mizrahi 1781 David Mozes 1783 Yuanlong Jiang 1785 Carlos J. Bernardos 1787 The DetNet chairs serving during the DetNet Data Plane Solution 1788 Design Team: 1790 Lou Berger 1792 Pat Thaler 1794 Thanks for Stewart Bryant for his extensive review of the previous 1795 versions of the document. 1797 16. References 1799 16.1. Normative references 1801 [I-D.ietf-spring-segment-routing-mpls] 1802 Bashandy, A., Filsfils, C., Previdi, S., Decraene, B., 1803 Litkowski, S., and R. Shakir, "Segment Routing with MPLS 1804 data plane", draft-ietf-spring-segment-routing-mpls-14 1805 (work in progress), June 2018. 1807 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1808 Requirement Levels", BCP 14, RFC 2119, 1809 DOI 10.17487/RFC2119, March 1997, 1810 . 1812 [RFC2211] Wroclawski, J., "Specification of the Controlled-Load 1813 Network Element Service", RFC 2211, DOI 10.17487/RFC2211, 1814 September 1997, . 1816 [RFC2212] Shenker, S., Partridge, C., and R. Guerin, "Specification 1817 of Guaranteed Quality of Service", RFC 2212, 1818 DOI 10.17487/RFC2212, September 1997, 1819 . 1821 [RFC2474] Nichols, K., Blake, S., Baker, F., and D. Black, 1822 "Definition of the Differentiated Services Field (DS 1823 Field) in the IPv4 and IPv6 Headers", RFC 2474, 1824 DOI 10.17487/RFC2474, December 1998, 1825 . 1827 [RFC3031] Rosen, E., Viswanathan, A., and R. Callon, "Multiprotocol 1828 Label Switching Architecture", RFC 3031, 1829 DOI 10.17487/RFC3031, January 2001, 1830 . 1832 [RFC3032] Rosen, E., Tappan, D., Fedorkow, G., Rekhter, Y., 1833 Farinacci, D., Li, T., and A. Conta, "MPLS Label Stack 1834 Encoding", RFC 3032, DOI 10.17487/RFC3032, January 2001, 1835 . 1837 [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition 1838 of Explicit Congestion Notification (ECN) to IP", 1839 RFC 3168, DOI 10.17487/RFC3168, September 2001, 1840 . 1842 [RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V., 1843 and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP 1844 Tunnels", RFC 3209, DOI 10.17487/RFC3209, December 2001, 1845 . 1847 [RFC3270] Le Faucheur, F., Wu, L., Davie, B., Davari, S., Vaananen, 1848 P., Krishnan, R., Cheval, P., and J. Heinanen, "Multi- 1849 Protocol Label Switching (MPLS) Support of Differentiated 1850 Services", RFC 3270, DOI 10.17487/RFC3270, May 2002, 1851 . 1853 [RFC3443] Agarwal, P. and B. Akyol, "Time To Live (TTL) Processing 1854 in Multi-Protocol Label Switching (MPLS) Networks", 1855 RFC 3443, DOI 10.17487/RFC3443, January 2003, 1856 . 1858 [RFC3473] Berger, L., Ed., "Generalized Multi-Protocol Label 1859 Switching (GMPLS) Signaling Resource ReserVation Protocol- 1860 Traffic Engineering (RSVP-TE) Extensions", RFC 3473, 1861 DOI 10.17487/RFC3473, January 2003, 1862 . 1864 [RFC4023] Worster, T., Rekhter, Y., and E. Rosen, Ed., 1865 "Encapsulating MPLS in IP or Generic Routing Encapsulation 1866 (GRE)", RFC 4023, DOI 10.17487/RFC4023, March 2005, 1867 . 1869 [RFC4206] Kompella, K. and Y. Rekhter, "Label Switched Paths (LSP) 1870 Hierarchy with Generalized Multi-Protocol Label Switching 1871 (GMPLS) Traffic Engineering (TE)", RFC 4206, 1872 DOI 10.17487/RFC4206, October 2005, 1873 . 1875 [RFC4385] Bryant, S., Swallow, G., Martini, L., and D. McPherson, 1876 "Pseudowire Emulation Edge-to-Edge (PWE3) Control Word for 1877 Use over an MPLS PSN", RFC 4385, DOI 10.17487/RFC4385, 1878 February 2006, . 1880 [RFC5085] Nadeau, T., Ed. and C. Pignataro, Ed., "Pseudowire Virtual 1881 Circuit Connectivity Verification (VCCV): A Control 1882 Channel for Pseudowires", RFC 5085, DOI 10.17487/RFC5085, 1883 December 2007, . 1885 [RFC5129] Davie, B., Briscoe, B., and J. Tay, "Explicit Congestion 1886 Marking in MPLS", RFC 5129, DOI 10.17487/RFC5129, January 1887 2008, . 1889 [RFC5462] Andersson, L. and R. Asati, "Multiprotocol Label Switching 1890 (MPLS) Label Stack Entry: "EXP" Field Renamed to "Traffic 1891 Class" Field", RFC 5462, DOI 10.17487/RFC5462, February 1892 2009, . 1894 [RFC6003] Papadimitriou, D., "Ethernet Traffic Parameters", 1895 RFC 6003, DOI 10.17487/RFC6003, October 2010, 1896 . 1898 [RFC7510] Xu, X., Sheth, N., Yong, L., Callon, R., and D. Black, 1899 "Encapsulating MPLS in UDP", RFC 7510, 1900 DOI 10.17487/RFC7510, April 2015, 1901 . 1903 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 1904 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 1905 May 2017, . 1907 16.2. Informative references 1909 [I-D.ietf-detnet-architecture] 1910 Finn, N., Thubert, P., Varga, B., and J. Farkas, 1911 "Deterministic Networking Architecture", draft-ietf- 1912 detnet-architecture-05 (work in progress), May 2018. 1914 [I-D.ietf-detnet-dp-alt] 1915 Korhonen, J., Farkas, J., Mirsky, G., Thubert, P., 1916 Zhuangyan, Z., and L. Berger, "DetNet Data Plane Protocol 1917 and Solution Alternatives", draft-ietf-detnet-dp-alt-00 1918 (work in progress), October 2016. 1920 [I-D.ietf-detnet-dp-sol-ip] 1921 Korhonen, J., Varga, B., "DetNet IP Data Plane 1922 Encapsulation", 2018. 1924 [I-D.sdt-detnet-security] 1925 Mizrahi, T., Grossman, E., Hacker, A., Das, S., 1926 "Deterministic Networking (DetNet) Security 1927 Considerations, draft-sdt-detnet-security, work in 1928 progress", 2017. 1930 [IEEE8021CB] 1931 Finn, N., "Draft Standard for Local and metropolitan area 1932 networks - Seamless Redundancy", IEEE P802.1CB 1933 /D2.1 P802.1CB, December 2015, 1934 . 1937 [IEEE8021Q] 1938 IEEE 802.1, "Standard for Local and metropolitan area 1939 networks--Bridges and Bridged Networks (IEEE Std 802.1Q- 1940 2014)", 2014, . 1942 [RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S. 1943 Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1 1944 Functional Specification", RFC 2205, DOI 10.17487/RFC2205, 1945 September 1997, . 1947 [RFC3985] Bryant, S., Ed. and P. Pate, Ed., "Pseudo Wire Emulation 1948 Edge-to-Edge (PWE3) Architecture", RFC 3985, 1949 DOI 10.17487/RFC3985, March 2005, 1950 . 1952 [RFC4448] Martini, L., Ed., Rosen, E., El-Aawar, N., and G. Heron, 1953 "Encapsulation Methods for Transport of Ethernet over MPLS 1954 Networks", RFC 4448, DOI 10.17487/RFC4448, April 2006, 1955 . 1957 [RFC5654] Niven-Jenkins, B., Ed., Brungard, D., Ed., Betts, M., Ed., 1958 Sprecher, N., and S. Ueno, "Requirements of an MPLS 1959 Transport Profile", RFC 5654, DOI 10.17487/RFC5654, 1960 September 2009, . 1962 [RFC5921] Bocci, M., Ed., Bryant, S., Ed., Frost, D., Ed., Levrau, 1963 L., and L. Berger, "A Framework for MPLS in Transport 1964 Networks", RFC 5921, DOI 10.17487/RFC5921, July 2010, 1965 . 1967 [RFC6073] Martini, L., Metz, C., Nadeau, T., Bocci, M., and M. 1968 Aissaoui, "Segmented Pseudowire", RFC 6073, 1969 DOI 10.17487/RFC6073, January 2011, 1970 . 1972 [RFC7551] Zhang, F., Ed., Jing, R., and R. Gandhi, Ed., "RSVP-TE 1973 Extensions for Associated Bidirectional Label Switched 1974 Paths (LSPs)", RFC 7551, DOI 10.17487/RFC7551, May 2015, 1975 . 1977 [RFC7950] Bjorklund, M., Ed., "The YANG 1.1 Data Modeling Language", 1978 RFC 7950, DOI 10.17487/RFC7950, August 2016, 1979 . 1981 [RFC8040] Bierman, A., Bjorklund, M., and K. Watsen, "RESTCONF 1982 Protocol", RFC 8040, DOI 10.17487/RFC8040, January 2017, 1983 . 1985 [RFC8169] Mirsky, G., Ruffini, S., Gray, E., Drake, J., Bryant, S., 1986 and A. Vainshtein, "Residence Time Measurement in MPLS 1987 Networks", RFC 8169, DOI 10.17487/RFC8169, May 2017, 1988 . 1990 Appendix A. Example of DetNet data plane operation 1992 [Editor's note: Add a simplified example of DetNet data plane and how 1993 labels etc work in the case of MPLS-based PSN and utilizing PREOF. 1995 The figure is subject to change depending on the further DT decisions 1996 on the label handling..] 1998 Authors' Addresses 2000 Jouni Korhonen (editor) 2002 Email: jouni.nospam@gmail.com 2004 Balazs Varga (editor) 2005 Ericsson 2006 Magyar Tudosok krt. 11. 2007 Budapest 1117 2008 Hungary 2010 Email: balazs.a.varga@ericsson.com