idnits 2.17.1 draft-ietf-detnet-mpls-over-tsn-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (May 5, 2019) is 1818 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'Network' is mentioned on line 287, but not defined == Missing Reference: 'TBD-TSN-OVER-DETNET' is mentioned on line 379, but not defined == Unused Reference: 'RFC2211' is defined on line 761, but no explicit reference was found in the text == Unused Reference: 'RFC2212' is defined on line 765, but no explicit reference was found in the text == Unused Reference: 'RFC3032' is defined on line 775, but no explicit reference was found in the text == Unused Reference: 'RFC3270' is defined on line 785, but no explicit reference was found in the text == Unused Reference: 'RFC3443' is defined on line 791, but no explicit reference was found in the text == Unused Reference: 'RFC4206' is defined on line 802, but no explicit reference was found in the text == Unused Reference: 'RFC5085' is defined on line 813, but no explicit reference was found in the text == Unused Reference: 'RFC5129' is defined on line 818, but no explicit reference was found in the text == Unused Reference: 'RFC5462' is defined on line 822, but no explicit reference was found in the text == Unused Reference: 'RFC7510' is defined on line 827, but no explicit reference was found in the text == Unused Reference: 'G.8275.1' is defined on line 838, but no explicit reference was found in the text == Unused Reference: 'G.8275.2' is defined on line 845, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-detnet-flow-information-model' is defined on line 861, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-spring-segment-routing-mpls' is defined on line 873, but no explicit reference was found in the text == Unused Reference: 'IEEE1588' is defined on line 885, but no explicit reference was found in the text == Unused Reference: 'RFC2205' is defined on line 902, but no explicit reference was found in the text == Unused Reference: 'RFC2474' is defined on line 907, but no explicit reference was found in the text == Unused Reference: 'RFC4448' is defined on line 923, but no explicit reference was found in the text == Unused Reference: 'RFC4872' is defined on line 928, but no explicit reference was found in the text == Unused Reference: 'RFC4873' is defined on line 934, but no explicit reference was found in the text == Unused Reference: 'RFC4875' is defined on line 938, but no explicit reference was found in the text == Unused Reference: 'RFC5440' is defined on line 945, but no explicit reference was found in the text == Unused Reference: 'RFC5586' is defined on line 950, but no explicit reference was found in the text == Unused Reference: 'RFC5654' is defined on line 955, but no explicit reference was found in the text == Unused Reference: 'RFC5921' is defined on line 960, but no explicit reference was found in the text == Unused Reference: 'RFC6003' is defined on line 965, but no explicit reference was found in the text == Unused Reference: 'RFC6006' is defined on line 969, but no explicit reference was found in the text == Unused Reference: 'RFC6387' is defined on line 981, but no explicit reference was found in the text == Unused Reference: 'RFC6790' is defined on line 986, but no explicit reference was found in the text == Unused Reference: 'RFC7551' is defined on line 991, but no explicit reference was found in the text == Unused Reference: 'RFC8169' is defined on line 1004, but no explicit reference was found in the text == Outdated reference: A later version (-13) exists of draft-ietf-detnet-architecture-12 == Outdated reference: A later version (-14) exists of draft-ietf-detnet-flow-information-model-03 == Outdated reference: A later version (-14) exists of draft-ietf-pce-pcep-extension-for-pce-controller-01 -- Obsolete informational reference (is this intentional?): RFC 3272 (Obsoleted by RFC 9522) -- Obsolete informational reference (is this intentional?): RFC 6006 (Obsoleted by RFC 8306) Summary: 0 errors (**), 0 flaws (~~), 37 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 DetNet B. Varga, Ed. 3 Internet-Draft J. Farkas 4 Intended status: Standards Track Ericsson 5 Expires: November 6, 2019 A. Malis 6 S. Bryant 7 Huawei Technologies 8 J. Korhonen 9 May 5, 2019 11 DetNet Data Plane: MPLS over IEEE 802.1 Time Sensitive Networking (TSN) 12 draft-ietf-detnet-mpls-over-tsn-00 14 Abstract 16 This document specifies the Deterministic Networking MPLS data plane 17 when operating over a TSN network. 19 Status of This Memo 21 This Internet-Draft is submitted in full conformance with the 22 provisions of BCP 78 and BCP 79. 24 Internet-Drafts are working documents of the Internet Engineering 25 Task Force (IETF). Note that other groups may also distribute 26 working documents as Internet-Drafts. The list of current Internet- 27 Drafts is at https://datatracker.ietf.org/drafts/current/. 29 Internet-Drafts are draft documents valid for a maximum of six months 30 and may be updated, replaced, or obsoleted by other documents at any 31 time. It is inappropriate to use Internet-Drafts as reference 32 material or to cite them other than as "work in progress." 34 This Internet-Draft will expire on November 6, 2019. 36 Copyright Notice 38 Copyright (c) 2019 IETF Trust and the persons identified as the 39 document authors. All rights reserved. 41 This document is subject to BCP 78 and the IETF Trust's Legal 42 Provisions Relating to IETF Documents 43 (https://trustee.ietf.org/license-info) in effect on the date of 44 publication of this document. Please review these documents 45 carefully, as they describe your rights and restrictions with respect 46 to this document. Code Components extracted from this document must 47 include Simplified BSD License text as described in Section 4.e of 48 the Trust Legal Provisions and are provided without warranty as 49 described in the Simplified BSD License. 51 Table of Contents 53 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 54 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 55 2.1. Terms Used in This Document . . . . . . . . . . . . . . . 3 56 2.2. Abbreviations . . . . . . . . . . . . . . . . . . . . . . 4 57 3. Requirements Language . . . . . . . . . . . . . . . . . . . . 5 58 4. DetNet MPLS Data Plane Overview . . . . . . . . . . . . . . . 5 59 4.1. Layers of DetNet Data Plane . . . . . . . . . . . . . . . 5 60 4.2. DetNet MPLS Data Plane Scenarios . . . . . . . . . . . . 6 61 4.3. Packet Flow Example with Service Protection . . . . . . . 9 62 5. DetNet MPLS Data Plane Considerations . . . . . . . . . . . . 11 63 5.1. Sub-Network Considerations . . . . . . . . . . . . . . . 12 64 6. DetNet MPLS Operation Over IEEE 802.1 TSN Sub-Networks . . . 12 65 6.1. Mapping of TSN Stream ID and Sequence Number . . . . . . 14 66 6.2. TSN Usage of FRER . . . . . . . . . . . . . . . . . . . . 15 67 6.3. Procedures . . . . . . . . . . . . . . . . . . . . . . . 16 68 6.4. Layer 2 Addressing and QoS Considerations . . . . . . . . 16 69 7. Management and Control Considerations . . . . . . . . . . . . 16 70 8. Security Considerations . . . . . . . . . . . . . . . . . . . 17 71 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 17 72 10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 17 73 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 17 74 11.1. Normative References . . . . . . . . . . . . . . . . . . 17 75 11.2. Informative References . . . . . . . . . . . . . . . . . 19 76 Appendix A. Example of DetNet Data Plane Operation . . . . . . . 23 77 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 23 79 1. Introduction 81 [Editor's note: Introduction to be made specific to DetNet MPLS over 82 TSN scenario. May be similar to intro of DetNet IP over TSN.]. 84 Deterministic Networking (DetNet) is a service that can be offered by 85 a network to DetNet flows. DetNet provides these flows with a low 86 packet loss rates and assured maximum end-to-end delivery latency. 87 General background and concepts of DetNet can be found in 88 [I-D.ietf-detnet-architecture]. 90 The DetNet Architecture decomposes the DetNet related data plane 91 functions into two sub-layers: a service sub-layer and a forwarding 92 sub-layer. The service sub-layer is used to provide DetNet service 93 protection and reordering. The forwarding sub-layer is used to 94 provides congestion protection (low loss, assured latency, and 95 limited reordering) leveraging MPLS Traffic Engineering mechanisms. 97 This document specifies the DetNet data plane operation and the on- 98 wire encapsulation of DetNet flows over an MPLS-based Packet Switched 99 Network (PSN). The specified encapsulation provides the building 100 blocks to enable the DetNet service and forwarding sub-layer 101 functions and supports flow identification as described in the DetNet 102 Architecture. As part of the service sub-layer functions, this 103 document describes DetNet node data plane operation. It also 104 describes the function and operation of the Packet Replication (PRF) 105 Packet Elimination (PEF) and Packet Ordering (POF) functions with an 106 MPLS data plane. It also describes an MPLS-based DetNet forwarding 107 sub-layer that eliminates (or reduces) contention loss and provides 108 bounded latency for DetNet flows. 110 MPLS encapsulated DetNet flows can be carried over network 111 technologies that can provide the DetNet required level of service. 112 This document defines examples of such, specifically carrying DetNet 113 MPLS flows over IEEE 802.1 TSN sub-networks, and over DetNet IP PSN. 115 The intent is for this document to support different traffic types 116 being mapped over DetNet MPLS, but this is out side the scope of this 117 document. An example of such can be found in 118 [I-D.ietf-detnet-dp-sol-ip]. This document also allows for, but does 119 not define, associated controller plane and Operations, 120 Administration, and Maintenance (OAM) functions. 122 2. Terminology 124 [Editor's note: Needs clean up.]. 126 2.1. Terms Used in This Document 128 This document uses the terminology established in the DetNet 129 architecture [I-D.ietf-detnet-architecture], and the reader is 130 assumed to be familiar with that document and its terminology. 132 The following terminology is introduced in this document: 134 F-Label A Detnet "forwarding" label that identifies the LSP 135 used to forward a DetNet flow across an MPLS PSN, e.g., 136 a hop-by-hop label used between label switching routers 137 (LSR). 139 S-Label A DetNet "service" label that is used between DetNet 140 nodes that implement also the DetNet service sub-layer 141 functions. An S-Label is also used to identify a 142 DetNet flow at DetNet service sub-layer. 144 d-CW A DetNet Control Word (d-CW) is used for sequencing and 145 identifying duplicate packets of a DetNet flow at the 146 DetNet service sub-layer. 148 2.2. Abbreviations 150 The following abbreviations are used in this document: 152 AC Attachment Circuit. 154 CE Customer Edge equipment. 156 CoS Class of Service. 158 CW Control Word. 160 DetNet Deterministic Networking. 162 DF DetNet Flow. 164 DN-IWF DetNet Inter-Working Function. 166 L2 Layer 2. 168 L2VPN Layer 2 Virtual Private Network. 170 L3 Layer 3. 172 LSR Label Switching Router. 174 MPLS Multiprotocol Label Switching. 176 MPLS-TE Multiprotocol Label Switching - Traffic Engineering. 178 MPLS-TP Multiprotocol Label Switching - Transport Profile. 180 MS-PW Multi-Segment PseudoWire (MS-PW). 182 NSP Native Service Processing. 184 OAM Operations, Administration, and Maintenance. 186 PE Provider Edge. 188 PEF Packet Elimination Function. 190 PRF Packet Replication Function. 192 PREOF Packet Replication, Elimination and Ordering Functions. 194 POF Packet Ordering Function. 196 PSN Packet Switched Network. 198 PW PseudoWire. 200 QoS Quality of Service. 202 S-PE Switching Provider Edge. 204 T-PE Terminating Provider Edge. 206 TSN Time-Sensitive Network. 208 3. Requirements Language 210 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 211 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 212 "OPTIONAL" in this document are to be interpreted as described in BCP 213 14 [RFC2119] [RFC8174] when, and only when, they appear in all 214 capitals, as shown here. 216 4. DetNet MPLS Data Plane Overview 218 [Editor's note: simplify this section and highlight DetNet MPLS over 219 subnets scenario being the focus in the remaining part of the 220 document.]. 222 4.1. Layers of DetNet Data Plane 224 This document describes how DetNet flows are carried over MPLS 225 networks. The DetNet Architecture, [I-D.ietf-detnet-architecture], 226 decomposes the DetNet data plane into two sub-layers: a service sub- 227 layer and a forwarding sub-layer. The basic approach defined in this 228 document supports the DetNet service sub-layer based on existing 229 pseudowire (PW) encapsulations and mechanisms, and supports the 230 DetNet forwarding sub-layer based on existing MPLS Traffic 231 Engineering encapsulations and mechanisms. Background on PWs can be 232 found in [RFC3985] and [RFC3031]. Background on MPLS Traffic 233 Engineering can be found in [RFC3272] and [RFC3209]. 235 DetNet MPLS 236 . 237 . 238 +------------+ 239 | Service | d-CW, S-Label 240 +------------+ 241 | Forwarding | F-Label(s) 242 +------------+ 243 . 244 . 246 Figure 1: DetNet Adaptation to MPLS Data Plane 248 The DetNet MPLS data plane approach defined in this document is shown 249 in Figure 1. The service sub-layer is supported by a DetNet control 250 word (d-CW) which conforms to the Generic PW MPLS Control Word 251 (PWMCW) defined in [RFC4385]. A d-CW identifying service label 252 (S-Label) is also used. 254 A node operating on a DetNet flow in the Detnet service sub-layer, 255 i.e. a node processing a DetNet packet which has the S-Label as top 256 of stack uses the local context associated with that S-Label, for 257 example a received F-Label, to determine what local DetNet 258 operation(s) are applied to that packet. An S-Label may be unique 259 when taken from the platform label space [RFC3031], which would 260 enable correct DetNet flow identification regardless of which input 261 interface or LSP the packet arrives on. 263 The DetNet MPLS data plane builds on MPLS Traffic Engineering 264 encapsulations and mechanisms to provide a forwarding sub-layer that 265 is responsible for providing resource allocation and explicit routes. 266 The forwarding sub-layer is supported by one or more forwarding 267 labels (F-Labels). 269 4.2. DetNet MPLS Data Plane Scenarios 271 [Editor's note: simplify this section and highlight DetNet MPLS over 272 subnets scenario being the focus in the remaining part of the 273 document.]. 275 DetNet MPLS Relay Transit Relay DetNet MPLS 276 End System Node Node Node End System 277 (T-PE) (S-PE) (LSR) (S-PE) (T-PE) 278 +----------+ +----------+ 279 | Appl. |<------------ End to End Service ----------->| Appl. | 280 +----------+ +---------+ +---------+ +----------+ 281 | Service |<--| Service |-- DetNet flow --| Service |-->| Service | 282 +----------+ +---------+ +----------+ +---------+ +----------+ 283 |Forwarding| |Fwd| |Fwd| |Forwarding| |Fwd| |Fwd| |Forwarding| 284 +-------.--+ +-.-+ +-.-+ +----.---.-+ +-.-+ +-.-+ +---.------+ 285 : Link : / ,-----. \ : Link : / ,-----. \ 286 +........+ +-[ Sub ]-+ +......+ +-[ Sub ]-+ 287 [Network] [Network] 288 `-----' `-----' 289 |<- LSP -->| |<-------- LSP -----------| |<--- LSP -->| 291 |<----------------- DetNet MPLS --------------------->| 293 Figure 2: A DetNet MPLS Network 295 Figure 2 illustrates a hypothetical DetNet MPLS-only network composed 296 of DetNet aware MPLS enabled end systems, operating over a DetNet 297 aware MPLS network. In this figure, relay nodes sit at MPLS LSP 298 boundaries and transit nodes are LSRs. 300 DetNet end system and relay nodes are DetNet service sub-layer aware, 301 understand the particular needs of DetNet flows and provide both 302 DetNet service and forwarding sub-layer functions. They add, remove 303 and process d-CWs, S-Labels and F-labels as needed. MPLS enabled end 304 system and relay nodes can enhance the reliability of delivery by 305 enabling the replication of packets where multiple copies, possibly 306 over multiple paths, are forwarded through the DetNet domain. They 307 can also eliminate surplus previously replicated copies of DetNet 308 packets. DetNet MPLS nodes provide functionality similar to T-PEs 309 when they sit at the edge of an MPLS domain, and functionality 310 similar to S-PEs when they are in the middle of an MPLS domain, see 311 [RFC6073]. End system and relay nodes also include DetNet forwarding 312 sub-layer functions, support for notably explicit routes, and 313 resources allocation to eliminate (or reduce) congestion loss and 314 jitter. 316 DetNet transit nodes reside wholly within a DetNet domain, and also 317 provide DetNet forwarding sub-layer functions in accordance with the 318 performance required by a DetNet flow carried over an LSP. Unlike 319 other DetNet node types, transit nodes provide no service sub-layer 320 processing. In a DetNet MPLS network, transit nodes may be DetNet 321 service aware or may be DetNet unaware MPLS Label Switching Routers 322 (LSRs). In this latter case, such LSRs would be unaware of the 323 special requirements of the DetNet service sub-layer, but would still 324 provide traffic engineering services and the QoS need to ensure that 325 the (TE) LSPs meet the service requirements of the carried DetNet 326 flows. 328 The LSPs may be provided by any MPLS controller method. For example 329 they may be provisioned via a management plane, RSVP-TE, MPLS-TP, or 330 MPLS Segment Routing (when extended to support resource allocation). 332 [Editor's note: Figure 3. and surrunding text are candidates to 333 delete from this document.]. 335 Figure 3 illustrates how an end to end MPLS-based DetNet service is 336 provided in a more detail. In this figure, the end systems, CE1 and 337 CE2, are able to send and receive MPLS encapsulated DetNet flows, and 338 R1, R2 and R3 are relay nodes as they sit in the middle of a DetNet 339 network. The 'X' in the end systems, and relay nodes represents 340 potential DetNet compound flow packet replication and elimination 341 points. In this example, service protection is supported over four 342 DetNet member flows and TE LSPs. For a unidirectional flow, R1 343 supports PRF, R2 supports PREOF and R3 supports PEF and POF. Note 344 that the relay nodes may change the underlying forwarding sub-layer, 345 for example tunneling MPLS over IEEE 802.1 TSN Section 6, or simply 346 over interconnect network links. 348 DetNet DetNet 349 MPLS Service Transit Transit Service MPLS 350 DetNet | |<-Tnl->| |<-Tnl->| | DetNet 351 End | V 1 V V 2 V | End 352 System | +--------+ +--------+ +--------+ | System 353 +---+ | | R1 |=======| R2 |=======| R3 | | +---+ 354 | X...DFa...|._X_....|..DF1..|.__ ___.|..DF3..|...._X_.|.DFa..|.X | 355 |CE1|========| \ | | X | | / |======|CE2| 356 | | | | \_.|..DF2..|._/ \__.|..DF4..|._/ | | | | 357 +---+ | |=======| |=======| | +---+ 358 ^ +--------+ +--------+ +--------+ ^ 359 | Relay Node Relay Node Relay Node | 360 | (S-PE) (S-PE) (S-PE) | 361 | | 362 |<---------------------- DetNet MPLS --------------------->| 363 | | 364 |<--------------- End to End DetNet Service -------------->| 366 -------------------------- Data Flow -------------------------> 368 X = Optional service protection (none, PRF, PREOF, PEF/POF) 369 DFx = DetNet member flow x over a TE LSP 371 Figure 3: MPLS-Based Native DetNet 373 As previously mentioned, this document specifies how MPLS is used to 374 support DetNet flows using an MPLS data plane as well as how such can 375 be mapped to IEEE 802.1 TSN and IP DetNet PSNs. An equally import 376 scenario is when IP is supported over DetNet MPLS and this is covered 377 in [I-D.ietf-detnet-dp-sol-ip]. Another important scenario is where 378 an Ethernet Layer 2 service is supported over DetNet MPLS and this is 379 covered in [TBD-TSN-OVER-DETNET]. 381 4.3. Packet Flow Example with Service Protection 383 [Editor's note: this text might be relevant for the discussion of 384 FRER within the TSN sub-network. Needs revision.]. 386 An example DetNet MPLS network fragment and packet flow is 387 illustrated in Figure 4. 389 1 1.1 1.1 1.2.1 1.2.1 1.2.2 390 CE1----EN1--------R1-------R2-------R3--------EN2-----CE2 391 \ 1.2.1 / / 392 \1.2 /-----+ / 393 +------R4------------------------+ 394 1.2.2 396 Figure 4: Example Packet Flow in DetNet Enabled MPLS Network 398 In Figure 4 the numbers are used to identify the instance of a 399 packet. Packet 1 is the original packet, and packets 1.1, and 1.2 400 are two first generation copies of packet 1. Packet 1.2.1 is a 401 second generation copy of packet 1.2 etc. Note that these numbers 402 never appear in the packet, and are not to be confused with sequence 403 numbers, labels or any other identifier that appears in the packet. 404 They simply indicate the generation number of the original packet so 405 that its passage through the network fragment can be identified to 406 the reader. 408 Customer Equipment CE1 sends a packet into the DetNet enabled MPLS 409 network. This is packet (1). Edge Node EN1 encapsulates the packet 410 as a DetNet Packet and sends it to Relay node R1 (packet 1.1). EN1 411 makes a copy of the packet (1.2), encapsulates it and sends this copy 412 to Relay node R4. 414 Note that along the MPLS path from EN1 to R1 there may be zero or 415 more LSRs which, for clarity, are not shown. The same is true for 416 any other path between two DetNet entities shown in Figure 4. 418 Relay node R4 has been configured to send one copy of the packet to 419 Relay Node R2 (packet 1.2.1) and one copy to Edge Node EN2 (packet 420 1.2.2). 422 R2 receives packet copy 1.2.1 before packet copy 1.1 arrives, and, 423 having been configured to perform packet elimination on this DetNet 424 flow, forwards packet 1.2.1 to Relay Node R3. Packet copy 1.1 is of 425 no further use and so is discarded by R2. 427 Edge Node EN2 receives packet copy 1.2.2 from R4 before it receives 428 packet copy 1.2.1 from R2 via relay Node R3. EN2 therefore strips 429 any DetNet encapsulation from packet copy 1.2.2 and forwards the 430 packet to CE2. When EN2 receives the later packet copy 1.2.1 this is 431 discarded. 433 The above is of course illustrative of many network scenarios that 434 can be configured. Between a pair of relay nodes there may be one or 435 more transit nodes that simply forward the DetNet traffic, but these 436 are omitted for clarity. 438 5. DetNet MPLS Data Plane Considerations 440 [Editor's note: Sort out what data plane considerations are relevant 441 for sub-net scenarios.]. 443 This section provides informative considerations related to providing 444 DetNet service to flows which are identified based on their header 445 information. At a high level, the following are provided on a per 446 flow basis: 448 Eliminating contention loss and jitter reduction: 450 Use of allocated resources (queuing, policing, shaping) to ensure 451 that the congestion-related loss and latency/jitter requirements 452 of a DetNet flow are met. 454 Explicit routes: 456 Use of a specific path for a flow. This limits misordering and 457 bounds latency. 459 Service protection: 461 Which in the case of this document primarily relates to 462 replication and elimination. Changing the explicit path after a 463 failure is detected in order to restore delivery of the required 464 DetNet service characteristics is also possible. Path changes, 465 even in the case of failure recovery, can lead to the out of order 466 delivery of data. 468 Load sharing: 470 Generally, distributing packets of the same DetNet flow over 471 multiple paths is not recommended. Such load sharing, e.g., via 472 ECMP or UCMP, impacts ordering and possibly jitter. 474 Troubleshooting: 476 For example, to support identification of misbehaving flows. 478 Recognize flow(s) for analytics: 480 For example, increase counters. 482 Correlate events with flows: 484 For example, unexpected loss. 486 The DetNet data plane also allows for the aggregation of DetNet 487 flows, e.g., via MPLS hierarchical LSPs, to improved scaling. When 488 DetNet flows are aggregated, transit nodes provide service to the 489 aggregate and not on a per-DetNet flow basis. In this case, nodes 490 performing aggregation will ensure that per-flow service requirements 491 are achieved. 493 5.1. Sub-Network Considerations 495 As shown in Figure 2, MPLS nodes are interconnected by different sub- 496 network technologies, which may include point-to-point links. Each 497 of these need to provide appropriate service to DetNet flows. In 498 some cases, e.g., on dedicated point-to-point links or TDM 499 technologies, all that is required is for a DetNet node to 500 appropriately queue its output traffic. In other cases, DetNet nodes 501 will need to map DetNet flows to the flow semantics (i.e., 502 identifiers) and mechanisms used by an underlying sub-network 503 technology. Figure 5 shows several examples of header formats that 504 can be used to carry DetNet MPLS flows over different sub-network 505 technologies. L2 represent a generic layer-2 encapsulation that 506 might be used on a point-to-point link. TSN represents the 507 encapsulation used on an IEEE 802.1 TSN network, as described in 508 Section 6. UDP/IP represents the encapsulation used on a DetNet IP 509 PSN. 511 +------+ +------+ +------+ 512 App-Flow | X | | X | | X | 513 +-----+======+--+======+--+======+-----+ 514 DetNet-MPLS | d-CW | | d-CW | | d-CW | 515 +------+ +------+ +------+ 516 |Labels| |Labels| |Labels| 517 +-----+======+--+======+--+======+-----+ 518 Sub-Network | L2 | | TSN | | UDP | 519 +------+ +------+ +------+ 520 | IP | 521 +------+ 522 | L2 | 523 +------+ 525 Figure 5: Example DetNet MPLS Sub-Network Formats 527 6. DetNet MPLS Operation Over IEEE 802.1 TSN Sub-Networks 529 [Editor's note: this is a place holder section. A standalone section 530 on MPLS over IEEE 802.1 TSN. Includes RFC2119 Language.] 531 This section covers how DetNet MPLS flows operate over an IEEE 802.1 532 TSN sub-network. Figure 6 illustrates such a scenario, where two 533 MPLS (DetNet) nodes are interconnected by a TSN sub-network. Node-1 534 is single homed and Node-2 is dual-homed. MPLS nodes can be (1) 535 DetNet MPLS End System, (2) DetNet MPLS Edge or Relay node or (3) 536 MPLS Transit node. 538 Note: in case of MPLS Transit node there is no DetNet Service sub- 539 layer processing. 541 MPLS (DetNet) MPLS (DetNet) 542 Node-1 Node-2 544 +----------+ +----------+ 545 <--| Service* |-- DetNet flow ---| Service* |--> 546 +----------+ +----------+ 547 |Forwarding| |Forwarding| 548 +--------.-+ <-TSN Str-> +-.-----.--+ 549 \ ,-------. / / 550 +----[ TSN-Sub ]---+ / 551 [ Network ]--------+ 552 `-------' 553 <---------------- DetNet MPLS ---------------> 555 Note: * no service sub-layer required for transit nodes 557 Figure 6: DetNet Enabled MPLS Network Over a TSN Sub-Network 559 The Time-Sensitive Networking (TSN) Task Group of the IEEE 802.1 560 Working Group have defined (and are defining) a number of amendments 561 to IEEE 802.1Q [IEEE8021Q] that provide zero congestion loss and 562 bounded latency in bridged networks. Furthermore IEEE 802.1CB 563 [IEEE8021CB] defines frame replication and elimination functions for 564 reliability that should prove both compatible with and useful to, 565 DetNet networks. All these functions have to identify flows those 566 require TSN treatment. 568 As is the case for DetNet, a Layer 2 network node such as a bridge 569 may need to identify the specific DetNet flow to which a packet 570 belongs in order to provide the TSN/DetNet QoS for that packet. It 571 also may need a CoS marking, such as the priority field of an IEEE 572 Std 802.1Q VLAN tag, to give the packet proper service. 574 The challange for MPLS DeNet flows is that the protocol interworking 575 function defined in IEEE 802.1CB [IEEE8021CB] works only for IP 576 flows. The aim of the protocol interworking function is to convert 577 an ingress flow to use a specific multicast destination MAC address 578 and VLAN, for example to direct the packets through a specific path 579 inside the bridged network. A similar interworking pair at the other 580 end of the TSN sub-network would restore the packet to its original 581 destination MAC address and VLAN. 583 As protocol interworking function defined in [IEEE8021CB] does not 584 work for MPLS labeled flows, the DetNet MPLS nodes MUST ensure proper 585 TSN sub-network specific Ethernet encapsulation of the DetNet MPLS 586 packets. For a given TSN Stream (i.e., DetNet flow) an MPLS (DetNet) 587 node MUST behave as a TSN-aware Talker or a Listener inside the TSN 588 sub-network. 590 6.1. Mapping of TSN Stream ID and Sequence Number 592 TSN capable MPLS (DetNet) nodes are TSN-aware Talker/Listener as 593 shown in Figure 7. MPLS (DetNet) node MUST provide the TSN sub- 594 network specific Ethernet encapsulation over the link(s) towards the 595 sub-network. An TSN-aware MPLS (DetNet) node MUST support the 596 following TSN components: 598 1. For recognizing flows: 600 * Stream Identification (MPLS-flow-aware) 602 2. For FRER used inside the TSN domain, additonaly: 604 * Sequencing function (MPLS-flow-aware) 606 * Sequence encode/decode function 608 3. For FRER when the node is a TSN replication or elimination point, 609 additionally: 611 * Stream splitting function 613 * Individual recovery function 615 [Editor's note: Should we added here requirements regarding IEEE 616 802.1Q C-VLAN component?] 618 The Stream Identification and The Sequencing functions are slightly 619 modified for frames passed down the protocol stack from the upper 620 layers. 622 Stream Identification MUST pair MPLS flows and TSN Streams and encode 623 that in data plane formats as well. The packet's stream_handle 624 subparameter (see IEEE 802.1CB [IEEE8021CB]) inside the Talker/ 625 Listener is defined based on the Flow-ID used in the upper DetNet 626 MPLS layer. Stream Identification function MUST encode Ethernet 627 header fields namely (1) the destination MAC-address, (2) the VLAN-ID 628 and (3) priority parameters with TSN sub-network specific values. 629 Encoding is provided for the frame passed down the stack from the 630 upper layers. 632 The sequence generation function resides in the Sequencing function. 633 It generates a sequence_number subparameter for each packet of a 634 Stream passed down to the lower layers. Sequencing function MUST 635 copy sequence information from the MPLS d-CW of the packet to the 636 sequence_number subparameter for the frame passed down the stack from 637 the upper layers. 639 MPLS (DetNet) 640 Node-1 641 <----------> 643 +----------+ 644 <--| Service |-- DetNet flow ------------------ 645 +----------+ 646 |Forwarding| 647 +----------+ +---------------+ 648 | L2 with |<---| L2 Relay with |---- TSN ---- 649 | TSN | | TSN function | Stream 650 +-----.----+ +--.---------.--+ 651 \__________/ \______ 653 TSN-aware 654 Talker / TSN-Bridge 655 Listener Relay 657 <--------- TSN sub-network ------------ 659 Figure 7: MPLS (DetNet) Node with TSN Functions 661 The Sequence encode/decode function MUST support the Redundancy tag 662 (R-TAG) format as per Clause 7.8 of IEEE 802.1CB [IEEE8021CB]. 664 6.2. TSN Usage of FRER 666 TSN Streams supporting DetNet flows may use Frame Replication and 667 Elimination for Redundancy (FRER) [802.1CB] based on the loss service 668 requirements of the TSN Stream, which is derived from the DetNet 669 service requirements of the DetNet mapped flow. The specific 670 operation of FRER is not modified by the use of DetNet and follows 671 IEEE 802.1CB [IEEE8021CB]. 673 FRER function and the provided service recovery is available only 674 within the TSN sub-network however as the Stream-ID and the TSN 675 sequence number are paired with the MPLS flow parameters they can be 676 combined with PREOF functions. 678 6.3. Procedures 680 [Editor's note: This section is TBD - covers required behavior of a 681 TSN-aware DetNet node using a TSN underlay.] 683 6.4. Layer 2 Addressing and QoS Considerations 685 [Editor's NOTE: review and simplify this section. May overlap with 686 previous sections.] 688 The Time-Sensitive Networking (TSN) Task Group of the IEEE 802.1 689 Working Group have defined (and are defining) a number of amendments 690 to IEEE 802.1Q [IEEE8021Q] that provide zero congestion loss and 691 bounded latency in bridged networks. IEEE 802.1CB [IEEE8021CB] 692 defines packet replication and elimination functions that should 693 prove both compatible with and useful to, DetNet networks. 695 As is the case for DetNet, a Layer 2 network node such as a bridge 696 may need to identify the specific DetNet flow to which a packet 697 belongs in order to provide the TSN/DetNet QoS for that packet. It 698 also will likely need a CoS marking, such as the priority field of an 699 IEEE Std 802.1Q VLAN tag, to give the packet proper service. 701 Although the flow identification methods described in IEEE 802.1CB 702 [IEEE8021CB] are flexible, and in fact, include IP 5-tuple 703 identification methods, the baseline TSN standards assume that every 704 Ethernet frame belonging to a TSN stream (i.e. DetNet flow) carries 705 a multicast destination MAC address that is unique to that flow 706 within the bridged network over which it is carried. Furthermore, 707 IEEE 802.1CB [IEEE8021CB] describes three methods by which a packet 708 sequence number can be encoded in an Ethernet frame. 710 Ensuring that the proper Ethernet VLAN tag priority and destination 711 MAC address are used on a DetNet/TSN packet may require further 712 clarification of the customary L2/L3 transformations carried out by 713 routers and edge label switches. Edge nodes may also have to move 714 sequence number fields among Layer 2, PW, and IP encapsulations. 716 7. Management and Control Considerations 718 [Editor's note: This section is TBD Covers Creation, mapping, removal 719 of TSN Stream IDs, related parameters and,when needed, configuration 720 of FRER. Supported by management/control plane. SEE sections in 721 removed text file.] 722 While management plane and control planes are traditionally 723 considered separately, from the Data Plane perspective there is no 724 practical difference based on the origin of flow provisioning 725 information, and the DetNet architecture 726 [I-D.ietf-detnet-architecture] refers to these collectively as the 727 'Controller Plane'. This document therefore does not distinguish 728 between information provided by distributed control plane protocols, 729 e.g., RSVP-TE [RFC3209] and [RFC3473], or by centralized network 730 management mechanisms, e.g., RestConf [RFC8040], YANG [RFC7950], and 731 the Path Computation Element Communication Protocol (PCEP) 732 [I-D.ietf-pce-pcep-extension-for-pce-controller] or any combination 733 thereof. Specific considerations and requirements for the DetNet 734 Controller Plane are discussed below. 736 8. Security Considerations 738 The security considerations of DetNet in general are discussed in 739 [I-D.ietf-detnet-architecture] and [I-D.sdt-detnet-security]. Other 740 security considerations will be added in a future version of this 741 draft. 743 9. IANA Considerations 745 This document makes no IANA requests. 747 10. Acknowledgements 749 Thanks for Norman Finn and Lou Berger for their comments and 750 contributions. 752 11. References 754 11.1. Normative References 756 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 757 Requirement Levels", BCP 14, RFC 2119, 758 DOI 10.17487/RFC2119, March 1997, 759 . 761 [RFC2211] Wroclawski, J., "Specification of the Controlled-Load 762 Network Element Service", RFC 2211, DOI 10.17487/RFC2211, 763 September 1997, . 765 [RFC2212] Shenker, S., Partridge, C., and R. Guerin, "Specification 766 of Guaranteed Quality of Service", RFC 2212, 767 DOI 10.17487/RFC2212, September 1997, 768 . 770 [RFC3031] Rosen, E., Viswanathan, A., and R. Callon, "Multiprotocol 771 Label Switching Architecture", RFC 3031, 772 DOI 10.17487/RFC3031, January 2001, 773 . 775 [RFC3032] Rosen, E., Tappan, D., Fedorkow, G., Rekhter, Y., 776 Farinacci, D., Li, T., and A. Conta, "MPLS Label Stack 777 Encoding", RFC 3032, DOI 10.17487/RFC3032, January 2001, 778 . 780 [RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V., 781 and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP 782 Tunnels", RFC 3209, DOI 10.17487/RFC3209, December 2001, 783 . 785 [RFC3270] Le Faucheur, F., Wu, L., Davie, B., Davari, S., Vaananen, 786 P., Krishnan, R., Cheval, P., and J. Heinanen, "Multi- 787 Protocol Label Switching (MPLS) Support of Differentiated 788 Services", RFC 3270, DOI 10.17487/RFC3270, May 2002, 789 . 791 [RFC3443] Agarwal, P. and B. Akyol, "Time To Live (TTL) Processing 792 in Multi-Protocol Label Switching (MPLS) Networks", 793 RFC 3443, DOI 10.17487/RFC3443, January 2003, 794 . 796 [RFC3473] Berger, L., Ed., "Generalized Multi-Protocol Label 797 Switching (GMPLS) Signaling Resource ReserVation Protocol- 798 Traffic Engineering (RSVP-TE) Extensions", RFC 3473, 799 DOI 10.17487/RFC3473, January 2003, 800 . 802 [RFC4206] Kompella, K. and Y. Rekhter, "Label Switched Paths (LSP) 803 Hierarchy with Generalized Multi-Protocol Label Switching 804 (GMPLS) Traffic Engineering (TE)", RFC 4206, 805 DOI 10.17487/RFC4206, October 2005, 806 . 808 [RFC4385] Bryant, S., Swallow, G., Martini, L., and D. McPherson, 809 "Pseudowire Emulation Edge-to-Edge (PWE3) Control Word for 810 Use over an MPLS PSN", RFC 4385, DOI 10.17487/RFC4385, 811 February 2006, . 813 [RFC5085] Nadeau, T., Ed. and C. Pignataro, Ed., "Pseudowire Virtual 814 Circuit Connectivity Verification (VCCV): A Control 815 Channel for Pseudowires", RFC 5085, DOI 10.17487/RFC5085, 816 December 2007, . 818 [RFC5129] Davie, B., Briscoe, B., and J. Tay, "Explicit Congestion 819 Marking in MPLS", RFC 5129, DOI 10.17487/RFC5129, January 820 2008, . 822 [RFC5462] Andersson, L. and R. Asati, "Multiprotocol Label Switching 823 (MPLS) Label Stack Entry: "EXP" Field Renamed to "Traffic 824 Class" Field", RFC 5462, DOI 10.17487/RFC5462, February 825 2009, . 827 [RFC7510] Xu, X., Sheth, N., Yong, L., Callon, R., and D. Black, 828 "Encapsulating MPLS in UDP", RFC 7510, 829 DOI 10.17487/RFC7510, April 2015, 830 . 832 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 833 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 834 May 2017, . 836 11.2. Informative References 838 [G.8275.1] 839 International Telecommunication Union, "Precision time 840 protocol telecom profile for phase/time synchronization 841 with full timing support from the network", ITU-T 842 G.8275.1/Y.1369.1 G.8275.1, June 2016, 843 . 845 [G.8275.2] 846 International Telecommunication Union, "Precision time 847 protocol telecom profile for phase/time synchronization 848 with partial timing support from the network", ITU-T 849 G.8275.2/Y.1369.2 G.8275.2, June 2016, 850 . 852 [I-D.ietf-detnet-architecture] 853 Finn, N., Thubert, P., Varga, B., and J. Farkas, 854 "Deterministic Networking Architecture", draft-ietf- 855 detnet-architecture-12 (work in progress), March 2019. 857 [I-D.ietf-detnet-dp-sol-ip] 858 Korhonen, J., Varga, B., "DetNet IP Data Plane 859 Encapsulation", 2018. 861 [I-D.ietf-detnet-flow-information-model] 862 Farkas, J., Varga, B., Cummings, R., and Y. Jiang, "DetNet 863 Flow Information Model", draft-ietf-detnet-flow- 864 information-model-03 (work in progress), March 2019. 866 [I-D.ietf-pce-pcep-extension-for-pce-controller] 867 Zhao, Q., Li, Z., Negi, M., and C. Zhou, "PCEP Procedures 868 and Protocol Extensions for Using PCE as a Central 869 Controller (PCECC) of LSPs", draft-ietf-pce-pcep- 870 extension-for-pce-controller-01 (work in progress), 871 February 2019. 873 [I-D.ietf-spring-segment-routing-mpls] 874 Bashandy, A., Filsfils, C., Previdi, S., Decraene, B., 875 Litkowski, S., and R. Shakir, "Segment Routing with MPLS 876 data plane", draft-ietf-spring-segment-routing-mpls-22 877 (work in progress), May 2019. 879 [I-D.sdt-detnet-security] 880 Mizrahi, T., Grossman, E., Hacker, A., Das, S., 881 "Deterministic Networking (DetNet) Security 882 Considerations, draft-sdt-detnet-security, work in 883 progress", 2017. 885 [IEEE1588] 886 IEEE, "IEEE 1588 Standard for a Precision Clock 887 Synchronization Protocol for Networked Measurement and 888 Control Systems Version 2", 2008. 890 [IEEE8021CB] 891 Finn, N., "Draft Standard for Local and metropolitan area 892 networks - Seamless Redundancy", IEEE P802.1CB 893 /D2.1 P802.1CB, December 2015, 894 . 897 [IEEE8021Q] 898 IEEE 802.1, "Standard for Local and metropolitan area 899 networks--Bridges and Bridged Networks (IEEE Std 802.1Q- 900 2014)", 2014, . 902 [RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S. 903 Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1 904 Functional Specification", RFC 2205, DOI 10.17487/RFC2205, 905 September 1997, . 907 [RFC2474] Nichols, K., Blake, S., Baker, F., and D. Black, 908 "Definition of the Differentiated Services Field (DS 909 Field) in the IPv4 and IPv6 Headers", RFC 2474, 910 DOI 10.17487/RFC2474, December 1998, 911 . 913 [RFC3272] Awduche, D., Chiu, A., Elwalid, A., Widjaja, I., and X. 914 Xiao, "Overview and Principles of Internet Traffic 915 Engineering", RFC 3272, DOI 10.17487/RFC3272, May 2002, 916 . 918 [RFC3985] Bryant, S., Ed. and P. Pate, Ed., "Pseudo Wire Emulation 919 Edge-to-Edge (PWE3) Architecture", RFC 3985, 920 DOI 10.17487/RFC3985, March 2005, 921 . 923 [RFC4448] Martini, L., Ed., Rosen, E., El-Aawar, N., and G. Heron, 924 "Encapsulation Methods for Transport of Ethernet over MPLS 925 Networks", RFC 4448, DOI 10.17487/RFC4448, April 2006, 926 . 928 [RFC4872] Lang, J., Ed., Rekhter, Y., Ed., and D. Papadimitriou, 929 Ed., "RSVP-TE Extensions in Support of End-to-End 930 Generalized Multi-Protocol Label Switching (GMPLS) 931 Recovery", RFC 4872, DOI 10.17487/RFC4872, May 2007, 932 . 934 [RFC4873] Berger, L., Bryskin, I., Papadimitriou, D., and A. Farrel, 935 "GMPLS Segment Recovery", RFC 4873, DOI 10.17487/RFC4873, 936 May 2007, . 938 [RFC4875] Aggarwal, R., Ed., Papadimitriou, D., Ed., and S. 939 Yasukawa, Ed., "Extensions to Resource Reservation 940 Protocol - Traffic Engineering (RSVP-TE) for Point-to- 941 Multipoint TE Label Switched Paths (LSPs)", RFC 4875, 942 DOI 10.17487/RFC4875, May 2007, 943 . 945 [RFC5440] Vasseur, JP., Ed. and JL. Le Roux, Ed., "Path Computation 946 Element (PCE) Communication Protocol (PCEP)", RFC 5440, 947 DOI 10.17487/RFC5440, March 2009, 948 . 950 [RFC5586] Bocci, M., Ed., Vigoureux, M., Ed., and S. Bryant, Ed., 951 "MPLS Generic Associated Channel", RFC 5586, 952 DOI 10.17487/RFC5586, June 2009, 953 . 955 [RFC5654] Niven-Jenkins, B., Ed., Brungard, D., Ed., Betts, M., Ed., 956 Sprecher, N., and S. Ueno, "Requirements of an MPLS 957 Transport Profile", RFC 5654, DOI 10.17487/RFC5654, 958 September 2009, . 960 [RFC5921] Bocci, M., Ed., Bryant, S., Ed., Frost, D., Ed., Levrau, 961 L., and L. Berger, "A Framework for MPLS in Transport 962 Networks", RFC 5921, DOI 10.17487/RFC5921, July 2010, 963 . 965 [RFC6003] Papadimitriou, D., "Ethernet Traffic Parameters", 966 RFC 6003, DOI 10.17487/RFC6003, October 2010, 967 . 969 [RFC6006] Zhao, Q., Ed., King, D., Ed., Verhaeghe, F., Takeda, T., 970 Ali, Z., and J. Meuric, "Extensions to the Path 971 Computation Element Communication Protocol (PCEP) for 972 Point-to-Multipoint Traffic Engineering Label Switched 973 Paths", RFC 6006, DOI 10.17487/RFC6006, September 2010, 974 . 976 [RFC6073] Martini, L., Metz, C., Nadeau, T., Bocci, M., and M. 977 Aissaoui, "Segmented Pseudowire", RFC 6073, 978 DOI 10.17487/RFC6073, January 2011, 979 . 981 [RFC6387] Takacs, A., Berger, L., Caviglia, D., Fedyk, D., and J. 982 Meuric, "GMPLS Asymmetric Bandwidth Bidirectional Label 983 Switched Paths (LSPs)", RFC 6387, DOI 10.17487/RFC6387, 984 September 2011, . 986 [RFC6790] Kompella, K., Drake, J., Amante, S., Henderickx, W., and 987 L. Yong, "The Use of Entropy Labels in MPLS Forwarding", 988 RFC 6790, DOI 10.17487/RFC6790, November 2012, 989 . 991 [RFC7551] Zhang, F., Ed., Jing, R., and R. Gandhi, Ed., "RSVP-TE 992 Extensions for Associated Bidirectional Label Switched 993 Paths (LSPs)", RFC 7551, DOI 10.17487/RFC7551, May 2015, 994 . 996 [RFC7950] Bjorklund, M., Ed., "The YANG 1.1 Data Modeling Language", 997 RFC 7950, DOI 10.17487/RFC7950, August 2016, 998 . 1000 [RFC8040] Bierman, A., Bjorklund, M., and K. Watsen, "RESTCONF 1001 Protocol", RFC 8040, DOI 10.17487/RFC8040, January 2017, 1002 . 1004 [RFC8169] Mirsky, G., Ruffini, S., Gray, E., Drake, J., Bryant, S., 1005 and A. Vainshtein, "Residence Time Measurement in MPLS 1006 Networks", RFC 8169, DOI 10.17487/RFC8169, May 2017, 1007 . 1009 Appendix A. Example of DetNet Data Plane Operation 1011 [Editor's note: Add a simplified example of DetNet data plane and how 1012 labels etc work in the case of MPLS-based PSN and utilizing PREOF. 1013 The figure is subject to change depending on the further DT decisions 1014 on the label handling..] 1016 Authors' Addresses 1018 Balazs Varga (editor) 1019 Ericsson 1020 Magyar Tudosok krt. 11. 1021 Budapest 1117 1022 Hungary 1024 Email: balazs.a.varga@ericsson.com 1026 Janos Farkas 1027 Ericsson 1028 Magyar Tudosok krt. 11. 1029 Budapest 1117 1030 Hungary 1032 Email: janos.farkas@ericsson.com 1034 Andrew G. Malis 1035 Huawei Technologies 1037 Email: agmalis@gmail.com 1039 Stewart Bryant 1040 Huawei Technologies 1042 Email: stewart.bryant@gmail.com 1044 Jouni Korhonen 1046 Email: jouni.nospam@gmail.com