idnits 2.17.1 draft-ietf-detnet-mpls-10.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 26, 2020) is 1370 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'Network' is mentioned on line 275, but not defined == Outdated reference: A later version (-09) exists of draft-ietf-detnet-ip-over-mpls-06 == Outdated reference: A later version (-07) exists of draft-ietf-detnet-mpls-over-tsn-03 == Outdated reference: A later version (-16) exists of draft-ietf-detnet-security-10 == Outdated reference: A later version (-20) exists of draft-ietf-detnet-yang-07 -- Obsolete informational reference (is this intentional?): RFC 3272 (Obsoleted by RFC 9522) Summary: 0 errors (**), 0 flaws (~~), 6 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 DetNet B. Varga, Ed. 3 Internet-Draft J. Farkas 4 Intended status: Standards Track Ericsson 5 Expires: January 27, 2021 L. Berger 6 LabN Consulting, L.L.C. 7 A. Malis 8 Malis Consulting 9 S. Bryant 10 Futurewei Technologies 11 J. Korhonen 12 July 26, 2020 14 DetNet Data Plane: MPLS 15 draft-ietf-detnet-mpls-10 17 Abstract 19 This document specifies the Deterministic Networking data plane when 20 operating over an MPLS Packet Switched Network. It leverages 21 existing pseudowire (PW) encapsulations and MPLS Traffic Engineering 22 encapsulations and mechanisms. This document builds on the DetNet 23 Architecture and Data Plane Framework. 25 Status of This Memo 27 This Internet-Draft is submitted in full conformance with the 28 provisions of BCP 78 and BCP 79. 30 Internet-Drafts are working documents of the Internet Engineering 31 Task Force (IETF). Note that other groups may also distribute 32 working documents as Internet-Drafts. The list of current Internet- 33 Drafts is at https://datatracker.ietf.org/drafts/current/. 35 Internet-Drafts are draft documents valid for a maximum of six months 36 and may be updated, replaced, or obsoleted by other documents at any 37 time. It is inappropriate to use Internet-Drafts as reference 38 material or to cite them other than as "work in progress." 40 This Internet-Draft will expire on January 27, 2021. 42 Copyright Notice 44 Copyright (c) 2020 IETF Trust and the persons identified as the 45 document authors. All rights reserved. 47 This document is subject to BCP 78 and the IETF Trust's Legal 48 Provisions Relating to IETF Documents 49 (https://trustee.ietf.org/license-info) in effect on the date of 50 publication of this document. Please review these documents 51 carefully, as they describe your rights and restrictions with respect 52 to this document. Code Components extracted from this document must 53 include Simplified BSD License text as described in Section 4.e of 54 the Trust Legal Provisions and are provided without warranty as 55 described in the Simplified BSD License. 57 Table of Contents 59 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 60 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 61 2.1. Terms Used in This Document . . . . . . . . . . . . . . . 3 62 2.2. Abbreviations . . . . . . . . . . . . . . . . . . . . . . 4 63 2.3. Requirements Language . . . . . . . . . . . . . . . . . . 5 64 3. DetNet MPLS Data Plane Overview . . . . . . . . . . . . . . . 5 65 3.1. Layers of DetNet Data Plane . . . . . . . . . . . . . . . 5 66 3.2. DetNet MPLS Data Plane Scenarios . . . . . . . . . . . . 6 67 4. MPLS-Based DetNet Data Plane Solution . . . . . . . . . . . . 8 68 4.1. DetNet Over MPLS Encapsulation Components . . . . . . . . 8 69 4.2. MPLS Data Plane Encapsulation . . . . . . . . . . . . . . 9 70 4.2.1. DetNet Control Word and the DetNet Sequence Number . 10 71 4.2.2. S-Labels . . . . . . . . . . . . . . . . . . . . . . 11 72 4.2.3. F-Labels . . . . . . . . . . . . . . . . . . . . . . 14 73 4.3. OAM Indication . . . . . . . . . . . . . . . . . . . . . 17 74 4.4. Flow Aggregation . . . . . . . . . . . . . . . . . . . . 17 75 4.4.1. Aggregation Via LSP Hierarchy . . . . . . . . . . . . 17 76 4.4.2. Aggregating DetNet Flows as a new DetNet flow . . . . 18 77 4.5. Service Sub-Layer Considerations . . . . . . . . . . . . 19 78 4.5.1. Edge Node Processing . . . . . . . . . . . . . . . . 19 79 4.5.2. Relay Node Processing . . . . . . . . . . . . . . . . 20 80 4.6. Forwarding Sub-Layer Considerations . . . . . . . . . . . 20 81 4.6.1. Class of Service . . . . . . . . . . . . . . . . . . 20 82 4.6.2. Quality of Service . . . . . . . . . . . . . . . . . 21 83 5. Management and Control Information Summary . . . . . . . . . 21 84 5.1. Service Sub-Layer Information Summary . . . . . . . . . . 22 85 5.1.1. Service Aggregation Information Summary . . . . . . . 23 86 5.2. Forwarding Sub-Layer Information Summary . . . . . . . . 23 87 6. Security Considerations . . . . . . . . . . . . . . . . . . . 24 88 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 25 89 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 25 90 9. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 26 91 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 26 92 10.1. Normative References . . . . . . . . . . . . . . . . . . 26 93 10.2. Informative References . . . . . . . . . . . . . . . . . 28 94 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 30 96 1. Introduction 98 Deterministic Networking (DetNet) is a service that can be offered by 99 a network to DetNet flows. DetNet provides these flows with 100 extremely low packet loss rates and assured maximum end-to-end 101 delivery latency. General background and concepts of DetNet can be 102 found in [RFC8655]. 104 The DetNet Architecture models the DetNet related data plane 105 functions decomposed into two sub-layers: a service sub-layer and a 106 forwarding sub-layer. The service sub-layer is used to provide 107 DetNet service functions such as protection and reordering. The 108 forwarding sub-layer is used to provide forwarding assurance (low 109 loss, assured latency, and limited reordering). 111 This document specifies the DetNet data plane operation and the on- 112 wire encapsulation of DetNet flows over an MPLS-based Packet Switched 113 Network (PSN) using the service reference model. MPLS encapsulation 114 already provides a solid foundation of building blocks to enable the 115 DetNet service and forwarding sub-layer functions. MPLS encapsulated 116 DetNet can be carried over a variety of different network 117 technologies that can provide the DetNet required level of service. 118 However, the specific details of how DetNet MPLS is carried over 119 different network technologies is out of scope of this document. 121 MPLS encapsulated DetNet flows can carry different types of traffic. 122 The details of the types of traffic that are carried in DetNet are 123 also out of scope of this document. An example of IP using DetNet 124 MPLS sub-networks can be found in [I-D.ietf-detnet-ip]. DetNet MPLS 125 may use an associated controller and Operations, Administration, and 126 Maintenance (OAM) functions that are defined outside of this 127 document. 129 Background information common to all data planes for DetNet can be 130 found in the DetNet Data Plane Framework 131 [I-D.ietf-detnet-data-plane-framework]. 133 2. Terminology 135 2.1. Terms Used in This Document 137 This document uses the terminology established in the DetNet 138 architecture [RFC8655] and the DetNet Data Plane Framework 139 [I-D.ietf-detnet-data-plane-framework]. The reader is assumed to be 140 familiar with these documents, any terminology defined therein and 141 basic MPLS related terminologies in [RFC3031]. 143 The following terminology is introduced in this document: 145 F-Label A Detnet "forwarding" label that identifies the LSP 146 used to forward a DetNet flow across an MPLS PSN, e.g., 147 a hop-by-hop label used between label switching routers 148 (LSR). 150 S-Label A DetNet "service" label that is used between DetNet 151 nodes that implement also the DetNet service sub-layer 152 functions. An S-Label is also used to identify a 153 DetNet flow at DetNet service sub-layer at a receiving 154 DetNet node. 156 A-Label A special case of an S-Label, whose aggregation 157 properties are known only at the aggregation and 158 deaggregation end-points. 160 d-CW A DetNet Control Word (d-CW) is used for sequencing 161 information of a DetNet flow at the DetNet service sub- 162 layer. 164 2.2. Abbreviations 166 The following abbreviations are used in this document: 168 CoS Class of Service. 170 CW Control Word. 172 DetNet Deterministic Networking. 174 LSR Label Switching Router. 176 MPLS Multiprotocol Label Switching. 178 MPLS-TE Multiprotocol Label Switching - Traffic Engineering. 180 MPLS-TP Multiprotocol Label Switching - Transport Profile. 182 OAM Operations, Administration, and Maintenance. 184 PE Provider Edge. 186 PEF Packet Elimination Function. 188 PRF Packet Replication Function. 190 PREOF Packet Replication, Elimination and Ordering Functions. 192 POF Packet Ordering Function. 194 PSN Packet Switched Network. 196 PW PseudoWire. 198 QoS Quality of Service. 200 S-PE Switching Provider Edge. 202 T-PE Terminating Provider Edge. 204 TSN Time-Sensitive Network. 206 2.3. Requirements Language 208 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 209 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 210 "OPTIONAL" in this document are to be interpreted as described in BCP 211 14 [RFC2119] [RFC8174] when, and only when, they appear in all 212 capitals, as shown here. 214 3. DetNet MPLS Data Plane Overview 216 3.1. Layers of DetNet Data Plane 218 MPLS provides a wide range of capabilities that can be utilised by 219 DetNet. A straight forward approach utilizing MPLS for a DetNet 220 service sub-layer is based on the existing pseudowire (PW) 221 encapsulations and by utilizing existing MPLS Traffic Engineering 222 encapsulations and mechanisms. Background on PWs can be found in 223 [RFC3985] and [RFC3031]. Background on MPLS Traffic Engineering can 224 be found in [RFC3272] and [RFC3209]. 226 DetNet MPLS 227 . 228 Bottom of Stack . 229 (inner label) +------------+ 230 | Service | d-CW, S-Label (A-Label) 231 +------------+ 232 | Forwarding | F-Label(s) 233 +------------+ 234 Top of Stack . 235 (outer label) . 237 Figure 1: DetNet Adaptation to MPLS Data Plane 239 The DetNet MPLS data plane representation is illustrated in Figure 1. 240 The service sub-layer includes a DetNet control word (d-CW) and an 241 identifying service label (S-Label). The DetNet control word (d-CW) 242 conforms to the Generic PW MPLS Control Word (PWMCW) defined in 243 [RFC4385]. An aggregation label (A-Label) is a special case of 244 S-Label used for aggregation. 246 A node operating on a received DetNet flow at the Detnet service sub- 247 layer uses the local context associated with a received S-Label, 248 i.e., a received F-Label, to determine which local DetNet 249 operation(s) are applied to that packet. An S-Label may be taken 250 from the platform label space [RFC3031], making it unique, enabling 251 DetNet flow identification regardless of which input interface or LSP 252 the packet arrives on. It is important to note that S-Label values 253 are driven by the receiver, not the sender. 255 The DetNet forwarding sub-layer is supported by zero or more 256 forwarding labels (F-Labels). MPLS Traffic Engineering 257 encapsulations and mechanisms can be utilized to provide a forwarding 258 sub-layer that is responsible for providing resource allocation and 259 explicit routes. 261 3.2. DetNet MPLS Data Plane Scenarios 263 DetNet MPLS Relay Transit Relay DetNet MPLS 264 End System Node Node Node End System 265 (T-PE) (S-PE) (LSR) (S-PE) (T-PE) 266 +----------+ +----------+ 267 | Appl. |<------------ End to End Service ----------->| Appl. | 268 +----------+ +---------+ +---------+ +----------+ 269 | Service |<--| Service |-- DetNet flow --| Service |-->| Service | 270 +----------+ +---------+ +----------+ +---------+ +----------+ 271 |Forwarding| |Fwd| |Fwd| |Forwarding| |Fwd| |Fwd| |Forwarding| 272 +-------.--+ +-.-+ +-.-+ +----.---.-+ +-.-+ +-.-+ +---.------+ 273 : Link : / ,-----. \ : Link : / ,-----. \ 274 +........+ +-[ Sub ]-+ +......+ +-[ Sub ]-+ 275 [Network] [Network] 276 `-----' `-----' 277 |<- LSP -->| |<-------- LSP -----------| |<--- LSP -->| 279 |<----------------- DetNet MPLS --------------------->| 281 Figure 2: A DetNet MPLS Network 283 Figure 2 illustrates a hypothetical DetNet MPLS-only network composed 284 of DetNet aware MPLS enabled end systems, operating over a DetNet 285 aware MPLS network. In this figure, the relay nodes are PE devices 286 that define the MPLS LSP boundaries and the transit nodes are LSRs. 288 DetNet end systems and relay nodes understand the particular needs of 289 DetNet flows and provide both DetNet service and forwarding sub-layer 290 functions. In the case of MPLS, DetNet service-aware nodes add, 291 remove and process d-CWs, S-Labels and F-labels as needed. DetNet 292 MPLS nodes provide functionality analogous to T-PEs when they sit at 293 the edge of an MPLS domain, and S-PEs when they are in the middle of 294 an MPLS domain, see [RFC6073]. 296 In a DetNet MPLS network, transit nodes may be DetNet service aware 297 or may be DetNet unaware MPLS Label Switching Routers (LSRs). In 298 this latter case, such LSRs would be unaware of the special 299 requirements of the DetNet service sub-layer, but would still provide 300 traffic engineering functions and the QoS capabilities needed to 301 ensure that the (TE) LSPs meet the service requirements of the 302 carried DetNet flows. 304 The application of DetNet using MPLS supports a number of control 305 plane/management plane types. These types support certain MPLS data 306 plane deployments. For example RSVP-TE, MPLS-TP, or MPLS Segment 307 Routing (when extended to support resource allocation) are all valid 308 MPLS deployment cases. 310 Figure 3 illustrates how an end-to-end MPLS-based DetNet service is 311 provided in a more detail. In this figure, the customer end systems, 312 CE1 and CE2, are able to send and receive MPLS encapsulated DetNet 313 flows, and R1, R2 and R3 are relay nodes in the middle of a DetNet 314 network. The 'X' in the end systems, and relay nodes represents 315 potential DetNet compound flow packet replication and elimination 316 points. In this example, service protection is supported utilizing 317 at least two DetNet member flows and TE LSPs. For a unidirectional 318 flow, R1 supports PRF and R3 supports PEF and POF. Note that the 319 relay nodes may change the underlying forwarding sub-layer, for 320 example tunneling MPLS over IEEE 802.1 TSN 321 [I-D.ietf-detnet-mpls-over-tsn], or simply over interconnect network 322 links. 324 DetNet DetNet 325 MPLS Service Transit Transit Service MPLS 326 DetNet | |<-Tnl->| |<-Tnl->| | DetNet 327 End | V 1 V V 2 V | End 328 System | +--------+ +--------+ +--------+ | System 329 +---+ | | R1 |=======| R2 |=======| R3 | | +---+ 330 | X...DFa...|._X_....|..DF1..|.__ ___.|..DF3..|...._X_.|.DFa..|.X | 331 |CE1|========| \ | | X | | / |======|CE2| 332 | | | | \_.|..DF2..|._/ \__.|..DF4..|._/ | | | | 333 +---+ | |=======| |=======| | +---+ 334 ^ +--------+ +--------+ +--------+ ^ 335 | Relay Node Relay Node Relay Node | 336 | (S-PE) (S-PE) (S-PE) | 337 | | 338 |<---------------------- DetNet MPLS --------------------->| 339 | | 340 |<--------------- End to End DetNet Service -------------->| 342 -------------------------- Data Flow -------------------------> 344 X = Optional service protection (none, PRF, PREOF, PEF/POF) 345 DFx = DetNet member flow x over a TE LSP 347 Figure 3: MPLS-Based Native DetNet 349 4. MPLS-Based DetNet Data Plane Solution 351 4.1. DetNet Over MPLS Encapsulation Components 353 To carry DetNet over MPLS the following is required: 355 1. A method of identifying the MPLS payload type. 357 2. A method of identifying the DetNet flow(s) to the processing 358 element. 360 3. A method of distinguishing DetNet OAM packets from DetNet data 361 packets. 363 4. A method of carrying the DetNet sequence number. 365 5. A suitable LSP to deliver the packet to the egress PE. 367 6. A method of carrying queuing and forwarding indication. 369 In this design an MPLS service label (the S-Label), is similar to a 370 pseudowire (PW) label [RFC3985], and is used to identify both the 371 DetNet flow identity and the payload MPLS payload type satisfying (1) 372 and (2) in the list above. OAM traffic discrimination happens 373 through the use of the Associated Channel method described in 374 [RFC4385]. The DetNet sequence number is carried in the DetNet 375 Control word which carries the Data/OAM discriminator. To simplify 376 implementation and to maximize interoperability two sequence number 377 sizes are supported: a 16 bit sequence number and a 28 bit sequence 378 number. The 16 bit sequence number is needed to support some types 379 of legacy clients. The 28 bit sequence number is used in situations 380 where it is necessary ensure that in high speed networks the sequence 381 number space does not wrap whilst packets are in flight. 383 The LSP used to forward the DetNet packet is not restricted regarding 384 any method used for establishing that LSP (for example, MPLS-LDP, 385 MPLS-TE, MPLS-TP [RFC5921], MPLS-SR [RFC8660], etc.). The LSP 386 (F-Label) label(s) and/or the S-Label may be used to indicate the 387 required queue processing as well as the forwarding parameters. Note 388 that the possible use of Penultimate Hop Popping (PHP) means that the 389 S-Label may be the only label received at the terminating DetNet 390 service. 392 4.2. MPLS Data Plane Encapsulation 394 Figure 4 illustrates a DetNet data plane MPLS encapsulation. The 395 MPLS-based encapsulation of the DetNet flows is well suited for the 396 scenarios described in [I-D.ietf-detnet-data-plane-framework]. 397 Furthermore, an end-to-end DetNet service i.e., native DetNet 398 deployment (see Section 3.2) is also possible if DetNet end systems 399 are capable of initiating and termination MPLS encapsulated packets. 401 The MPLS-based DetNet data plane encapsulation consists of: 403 o DetNet control word (d-CW) containing sequencing information for 404 packet replication and duplicate elimination purposes, and the OAM 405 indicator. 407 o DetNet service Label (S-Label) that identifies a DetNet flow at 408 the receiving DetNet service sub-layer processing node. 410 o Zero or more Detnet MPLS Forwarding label(s) (F-Label) used to 411 direct the packet along the label switched path (LSP) to the next 412 DetNet service sub-layer processing node along the path. When 413 Penultimate Hop Popping is in use there may be no label F-Label in 414 the protocol stack on the final hop. 416 o The necessary data-link encapsulation is then applied prior to 417 transmission over the physical media. 419 DetNet MPLS-based encapsulation 421 +---------------------------------+ 422 | | 423 | DetNet App-Flow | 424 | Payload Packet | 425 | | 426 +---------------------------------+ <--\ 427 | DetNet Control Word | | 428 +---------------------------------+ +--> DetNet data plane 429 | S-Label | | MPLS encapsulation 430 +---------------------------------+ | 431 | [ F-Label(s) ] | | 432 +---------------------------------+ <--/ 433 | Data-Link | 434 +---------------------------------+ 435 | Physical | 436 +---------------------------------+ 438 Figure 4: Encapsulation of a DetNet App-Flow in an MPLS PSN 440 4.2.1. DetNet Control Word and the DetNet Sequence Number 442 A DetNet control word (d-CW) conforms to the Generic PW MPLS Control 443 Word (PWMCW) defined in [RFC4385]. The d-CW formatted as shown in 444 Figure 5 MUST be present in all DetNet packets containing app-flow 445 data. This format of the d-CW was created in order (1) to allow 446 larger S/N space to avoid S/N rollover frequency in some applications 447 and (2) to allow non-skip zero S/N what simplifies implementation. 449 0 1 2 3 450 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 451 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 452 |0 0 0 0| Sequence Number | 453 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 455 Figure 5: DetNet Control Word 457 (bits 0 to 3) 459 Per [RFC4385], MUST be set to zero (0). 461 Sequence Number (bits 4 to 31) 463 An unsigned value implementing the DetNet sequence number. The 464 sequence number space is a circular one. 466 A separate sequence number space MUST be maintained by the node that 467 adds the d-CW for each DetNet app-flow, i.e., DetNet service. The 468 following sequence number field lengths MUST be supported: 470 0 bits 472 16 bits 474 28 bits 476 The sequence number length MUST be provisioned on a per Detnet 477 service basis via configuration, i.e., via the controller plane 478 described in [I-D.ietf-detnet-data-plane-framework]. 480 A 0 bit sequence number field length indicates that there is no 481 DetNet sequence number used for the flow. When the length is zero, 482 the sequence number field MUST be set to zero (0) on all packets sent 483 for the flow. 485 When the sequence number field length is 16 or 28 bits for a flow, 486 the sequence number MUST be incremented by one for each new app-flow 487 packet sent. When the field length is 16 bits, d-CW bits 4 to 15 488 MUST be set to zero (0). The values carried in this field can wrap 489 and it is important to note that zero (0) is a valid field value. 490 For example, were the sequence number size is 16 bits, the sequence 491 will contain: 65535, 0, 1, where zero (0) is an ordinary sequence 492 number. 494 It is important to note that this document differs from [RFC4448] 495 where a sequence number of zero (0) is used to indicate that the 496 sequence number check algorithm is not used. 498 The sequence number is optionally used during receive processing as 499 described below in Section 4.2.2.2 and Section 4.2.2.3. 501 4.2.2. S-Labels 503 A DetNet flow at the DetNet service sub-layer is identified by an 504 S-Label. MPLS-aware DetNet end systems and edge nodes, which are by 505 definition MPLS ingress and egress nodes, MUST add (push) and remove 506 (pop) a DetNet service-specific d-CW and S-Label. Relay nodes MAY 507 swap S-Label values when processing a DetNet flow, i.e., incoming and 508 outgoing S-Labels of a DetNet flow can be different. 510 S-Label values MUST be provisioned per DetNet service via 511 configuration, e.g., via the controller plane described in 512 [I-D.ietf-detnet-data-plane-framework]. Note that S-Labels provide 513 identification at the downstream DetNet service sub-layer receiver, 514 not the sender. As such, S-Labels MUST be allocated by the entity 515 that controls the service sub-layer receiving node's label space, and 516 MAY be allocated from the platform label space [RFC3031]. Because 517 S-Labels are local to each node rather than being a global identifier 518 within a domain, they must be advertised to their upstream DetNet 519 service-aware peer nodes (e.g., a DetNet MPLS End System or a DetNet 520 Relay or Edge Node) and interpreted in the context of their received 521 F-Label(s). In some PREOF topologies, the node performing 522 replication will be sending to multiple nodes performing PEF or POF, 523 and may need to send different S-Label values for the different 524 member flows for the same DetNet service. 526 An S-Label will normally be at the bottom of the label stack once the 527 last F-Label is removed, immediately preceding the d-CW. To support 528 service sub-layer level OAM, an OAM Associated Channel Header (ACH) 529 [RFC4385] together with a Generic Associated Channel Label (GAL) 530 [RFC5586] MAY be used in place of a d-CW. 532 Similarly, an Entropy Label Indicator/Entropy Label (ELI/EL) 533 [RFC6790] MAY be carried below the S-Label in the label stack in 534 networks where DetNet flows would otherwise received ECMP treatment. 535 When ELs are used, the same EL value SHOULD be used for all of the 536 packets sent using a specific S-Label to force the flow to follow the 537 same path. However, as outlines in 538 [I-D.ietf-detnet-data-plane-framework] the use of ECMP for DetNet 539 flows is NOT RECOMMENDED. ECMP MAY be used for non-DetNet flows 540 within a DetNet domain. 542 When receiving a DetNet MPLS packet, an implementation MUST identify 543 the DetNet service associated with the incoming packet based on the 544 S-Label. When a node is using platform labels for S-Labels, no 545 additional information is needed as the S-label uniquely identifies 546 the DetNet service. In the case where platform labels are not used, 547 zero or more F-Labels and optionally, the incoming interface, 548 proceeding the S-Label MUST be used together with the S-Label to 549 uniquely identify the DetNet service associated with a received 550 packet. The incoming interface MAY also be used together with any 551 present F-Label(s) and the S-Label to uniquely identify an incoming 552 DetNet service, for example, in the case where PHP is used. Note 553 that choice to use platform label space for S-Label or S-Label plus 554 one or more F-Labels to identify DetNet services is a local 555 implementation choice, with one caveat. When one or more F-labels, 556 or incoming interface, is needed together with an S-Label to uniquely 557 identify a service, the controller plane must ensure that incoming 558 DetNet MPLS packets arrive with the needed information (F-label(s) 559 and/or incoming interface) and provision the needed information. The 560 provisioned information MUST then be used to identify incoming DetNet 561 service based on the combination of S-Label and F-Label(s) or 562 incoming interface. 564 The use of platform labels for S-Labels matches other pseudowire 565 encapsulations for consistency but there is no hard requirement in 566 this regard. 568 4.2.2.1. Packet Replication Function Processing 570 The Packet Replication Function (PRF) function MAY be supported by an 571 implementation for outgoing DetNet flows. The use of the PRF for a 572 particular DetNet service MUST be provisioned via configuration, 573 e.g., via the controller plane described in 574 [I-D.ietf-detnet-data-plane-framework]. When replication is 575 configure, the same app-flow data will be sent over multiple outgoing 576 DetNet member flows using forwarding sub-layer LSPs. An S-Label 577 value MUST be configured per outgoing member flow. The same d-CW 578 field value MUST be used on all outgoing member flows for each 579 replicated MPLS packet. 581 4.2.2.2. Packet Elimination Function Processing 583 Implementations MAY support the Packet Elimination Function (PEF) for 584 received DetNet MPLS flows. When supported, use of the PEF for a 585 particular DetNet service MUST be provisioned via configuration, 586 e.g., via the controller plane described in 587 [I-D.ietf-detnet-data-plane-framework]. 589 After a DetNet service is identified for a received DetNet MPLS 590 packet, as described above, an implementation MUST check if PEF is 591 configured for that DetNet service. When configured, the 592 implementation MUST track the sequence number contained in received 593 d-CWs and MUST ensure that duplicate (replicated) instances of a 594 particular sequence number are discarded. The specific mechanisms 595 used for an implementation to identify which received packets are 596 duplicates and which are new is an implementation choice. Note that 597 per Section 4.2.1 the sequence number field length may be 16 or 28 598 bits, and the field value can wrap. PEF MUST NOT be used with DetNet 599 flows configured with a d-CW sequence number field length of 0 bits. 601 Note that an implementation MAY wish to constrain the maximum number 602 sequence numbers that are tracked, on platform-wide or per flow 603 basis. Some implementations MAY support the provisioning of the 604 maximum number sequence numbers that are tracked number on either a 605 platform-wide or per flow basis. 607 4.2.2.3. Packet Ordering Function Processing 609 A function that is related to in-order delivery is the Packet 610 Ordering Function (POF). Implementations MAY support POF. When 611 supported, use of the POF for a particular DetNet service MUST be 612 provisioned via configuration, e.g., via the controller plane 613 described by [I-D.ietf-detnet-data-plane-framework]. Implementations 614 MAY required that PEF and POF be used in combination. There is no 615 requirement related to the order of execution of the Packet 616 Elimination and Ordering Functions in an implementation. 618 After a DetNet service is identified for a received DetNet MPLS 619 packet, as described above, an implementation MUST check if POF is 620 configured for that DetNet service. When configured, the 621 implementation MUST track the sequence number contained in received 622 d-CWs and MUST ensure that packets are processed in the order 623 indicated in the received d-CW sequence number field, which may not 624 be in the order the packets are received. As defined in 625 Section 4.2.1 the sequence number field length may be 16 or 28 bits, 626 is incremented by one (1) for each new MPLS packet sent for a 627 particular DetNet service, and the field value can wrap. The 628 specific mechanisms used for an implementation to identify the order 629 of received packets is an implementation choice. 631 Note that an implementation MAY wish to constrain the maximum number 632 of out of order packets that can be processed, on platform-wide or 633 per flow basis. Some implementations MAY support the provisioning of 634 this number on either a platform-wide or per flow basis. The number 635 of out of order packets that can be processed also impacts the 636 latency of a flow. 638 4.2.3. F-Labels 640 F-Labels are supported the DetNet forwarding sub-layer. F-Labels are 641 used to provide LSP-based connectivity between DetNet service sub- 642 layer processing nodes. 644 4.2.3.1. Service Sub-Layer Related Processing 646 DetNet MPLS end systems, edge nodes and relay nodes may operate at 647 the DetNet service sub-layer with understanding of DetNet services 648 and their requirements. As mentioned earlier, when operating at this 649 layer such nodes can push, pop or swap (pop then push) S-Labels. In 650 all cases, the F-Label(s) used for a DetNet service are always 651 replaced and the following procedures apply. 653 When sending a DetNet flow, zero or more F-Labels MAY be pushed on 654 top of an S-Label by the node pushing an S-Label. The F-Label(s) to 655 be pushed when sending a particular DetNet service MUST be 656 provisioned per outgoing S-Label via configuration, e.g., via the 657 controller plane discussed in [I-D.ietf-detnet-data-plane-framework]. 658 F-Label(s) can also provide context for an S-Label. To allow for the 659 omission of F-Label(s), an implementation SHOULD also allow an 660 outgoing interface to be configured per S-Label. 662 Note, when PRF is supported, the same app-flow data will be sent over 663 multiple outgoing DetNet member flows using forwarding sub-layer 664 LSPs. This means that implementation may be sending different sets 665 of F-Labels per DetNet member flow, each with a different S-Label. 667 When a single set of F-Labels is provisioned for a particular 668 outgoing S-Label, that set of F-labels MUST be pushed after the 669 S-Label is pushed. The outgoing packet is then forwarded as 670 described below in Section 4.2.3.2. When a single outgoing interface 671 is provisioned, the outgoing packet is then forwarded as described 672 below in Section 4.2.3.2. 674 When multiple sets of outgoing F-Labels or interfaces are provisioned 675 for a particular DetNet service, a copy of the outgoing packet, 676 including the pushed member flow-specific S-Label, MUST be made per 677 F-label set and outgoing interface. Each set of provisioned F-Labels 678 are then pushed onto a copy of the packet. Each copy is then 679 forwarded as described below in Section 4.2.3.2. 681 As described in the previous section, when receiving a DetNet MPLS 682 flow, an implementation identifies the DetNet service associated with 683 the incoming packet based on the S-Label. When a node is using 684 platform labels for S-Labels, any F-Labels can be popped and the 685 S-label uniquely identifies the DetNet service. In the case where 686 platform labels are not used, incoming F-Label(s) and, optionally, 687 the incoming interface MUST also be provisioned for a DetNet service. 689 4.2.3.2. Common F-Label Processing 691 All DetNet aware MPLS nodes process F-Labels as needed to meet the 692 service requirements of the DetNet flow or flows carried in the LSPs 693 represented by the F-Labels. This includes normal push, pop and swap 694 operations. Such processing is essentially the same type of 695 processing provided for TE LSPs, although the specific service 696 parameters, or traffic specification, can differ. When the DetNet 697 service parameters of the DetNet flow or flows carried in an LSP 698 represented by an F-Label can be met by an existing TE mechanism, the 699 forwarding sub-layer processing node MAY be a DetNet unaware, i.e., 700 standard, MPLS LSR. Such TE LSPs may provide LSP forwarding service 701 as defined in, but not limited to, [RFC3209], [RFC3270], [RFC3272], 702 [RFC3473], [RFC4875], [RFC5440], and [RFC8306]. 704 More specifically, as mentioned above, the DetNet forwarding sub- 705 layer provides explicit routes and allocated resources, and F-Labels 706 are used to map to each. Explicit routes are supported based on the 707 topmost (outermost) F-Label that is pushed or swapped and the LSP 708 that corresponds to this label. This topmost (outgoing) label MUST 709 be associated with a provisioned outgoing interface and, for non- 710 point-to-point outgoing interfaces, a next hop LSR. Note that this 711 information MUST be provisioned via configuration or the controller 712 plane. In the previously mentioned special case where there are no 713 added F-labels and the outgoing interface is not a point-to-point 714 interface, the outgoing interface MUST also be associated with a next 715 hop LSR. 717 Resources may be allocated in a hierarchical fashion per LSP that is 718 represented by each F-Label. Each LSP MAY be provisioned with a 719 service parameters that dictates the specific traffic treatment to be 720 received by the traffic carried over that LSP. Implementations of 721 this document MUST ensure that traffic carried over each LSP 722 represented by one or more F-Labels receives the traffic treatment 723 provisioned for that LSP. Typical mechanisms used to provide 724 different treatment to different flows includes the allocation of 725 system resources (such as queues and buffers) and provisioning or 726 related parameters (such as shaping, and policing) that may be found 727 in implementations of Resource ReSerVation Protocol (RSVP) [RFC2205] 728 (RSVP) and RSVP-TE [RFC3209] and [RFC3473]. Support can also be 729 provided via an underlying network technology such IEEE802.1 TSN 730 [I-D.ietf-detnet-mpls-over-tsn]. The specific mechanisms selected by 731 a DetNet node to ensure DetNet service delivery requirements are met 732 for supported DetNet flows is outside the scope of this document. 734 Packets that are marked in a way that do not correspond to allocated 735 resources, e.g., lack a provisioned F-Label, can disrupt the QoS 736 offered to properly reserved DetNet flows by using resources 737 allocated to the reserved flows. Therefore, the network nodes of a 738 DetNet network: 740 o MUST defend the DetNet QoS by discarding or remarking (to an 741 allocated DetNet flow or non-competing non-DetNet flow) packets 742 received that are not associated with a completed resource 743 allocation. 745 o MUST NOT use a DetNet allocated resource, e.g. a queue or shaper 746 reserved for DetNet flows, for any packet that does match the 747 corresponding DetNet flow. 749 o MUST ensure a QoS flow does not exceed its allocated resources or 750 provisioned service level, 752 o MUST ensure a CoS flow or service class does not impact the 753 service delivered to other flows. This requirement is similar to 754 requirement for MPLS LSRs to that CoS LSPs do not impact the 755 resources allocated to TE LSPs, e.g., via [RFC3473]. 757 Subsequent sections provide additional considerations related to CoS 758 (Section 4.6.1), QoS (Section 4.6.2) and aggregation (Section 4.4). 760 4.3. OAM Indication 762 OAM follows the procedures set out in [RFC5085] with the restriction 763 that only Virtual Circuit Connectivity Verification (VCCV) type 1 is 764 supported. 766 As shown in Figure 3 of [RFC5085] when the first nibble of the d-CW 767 is 0x0 the payload following the d-CW is normal user data. However, 768 when the first nibble of the d-CW is 0X1, the payload that follows 769 the d-CW is an OAM payload with the OAM type indicated by the value 770 in the d-CW Channel Type field. 772 The reader is referred to [RFC5085] for a more detailed description 773 of the Associated Channel mechanism, and to the DetNet work on OAM 774 for more information DetNet OAM. 776 Additional considerations on DetNet-specific OAM are subjects for 777 further study. 779 4.4. Flow Aggregation 781 The ability to aggregate individual flows, and their associated 782 resource control, into a larger aggregate is an important technique 783 for improving scaling of control in the data, management and control 784 planes. The DetNet data plane allows for the aggregation of DetNet 785 flows, to improved scaling. There are two methods of supporting flow 786 aggregation covered in this section. 788 The resource control and management aspects of aggregation (including 789 the configuration of queuing, shaping, and policing) are the 790 responsibility of the DetNet controller plane and is out of scope of 791 this documents. It is also the responsibility of the controller 792 plane to ensure that consistent aggregation methods are used. 794 4.4.1. Aggregation Via LSP Hierarchy 796 DetNet flows forwarded via MPLS can leverage MPLS-TE's existing 797 support for hierarchical LSPs (H-LSPs), see [RFC4206]. H-LSPs are 798 typically used to aggregate control and resources, they may also be 799 used to provide OAM or protection for the aggregated LSPs. Arbitrary 800 levels of aggregation naturally falls out of the definition for 801 hierarchy and the MPLS label stack [RFC3032]. DetNet nodes which 802 support aggregation (LSP hierarchy) map one or more LSPs (labels) 803 into and from an H-LSP. Both carried LSPs and H-LSPs may or may not 804 use the TC field, i.e., L-LSPs or E-LSPs. Such nodes will need to 805 ensure that individual LSPs and H-LSPs receive the traffic treatment 806 required to ensure the required DetNet service is preserved. 808 Additional details of the traffic control capabilities needed at a 809 DetNet-aware node may be covered in the new service definitions 810 mentioned above or in separate future documents. Controller plane 811 mechanisms will also need to ensure that the service required on the 812 aggregate flow are provided, which may include the discarding or 813 remarking mentioned in the previous sections. 815 4.4.2. Aggregating DetNet Flows as a new DetNet flow 817 An aggregate can be built by layering DetNet flows, including both 818 their S-Label and, when present, F-Labels as shown below: 820 +---------------------------------+ 821 | | 822 | DetNet Flow | 823 | Payload Packet | 824 | | 825 +---------------------------------+ <--\ 826 | DetNet Control Word | | 827 +=================================+ | 828 | S-Label | | 829 +---------------------------------+ | 830 | [ F-Label(s) ] | +----DetNet data plane 831 +---------------------------------+ | 832 | DetNet Control Word | | 833 +=================================+ | 834 | A-Label | | 835 +---------------------------------+ | 836 | F-Label(s) | <--/ 837 +---------------------------------+ 838 | Data-Link | 839 +---------------------------------+ 840 | Physical | 841 +---------------------------------+ 843 Figure 6: DetNet Aggregation as a new DetNet Flow 845 Both the aggregation label, which is referred to as an A-Label, and 846 the individual flow's S-Label have their MPLS S bit set indicating 847 bottom of stack, and the d-CW allows the PREOF to work. An A-Label 848 is a special case of an S-Label, whose properties are known only at 849 the aggregation and deaggregation end-points. 851 It is a property of the A-Label that what follows is a d-CW followed 852 by an MPLS label stack. A relay node processing the A-Label would 853 not know the underlying payload type, and the A-Label would be 854 processed as a normal S-Label. This would only be known to a node 855 that was a peer of the node imposing the S-Label. However there is 856 no real need for it to know the payload type during aggregation 857 processing. 859 As in the previous section, nodes supporting this type of aggregation 860 will need to ensure that individual and aggregated flows receive the 861 traffic treatment required to ensure the required DetNet service is 862 preserved. Also, it is the controller plane's responsibility to to 863 ensure that the service required on the aggregate flow are properly 864 provisioned. 866 4.5. Service Sub-Layer Considerations 868 The edge and relay node internal procedures related to PREOF are 869 implementation specific. The order of a packet elimination or 870 replication is out of scope in this specification. 872 It is important that the DetNet layer is configured such that a 873 DetNet node never receives its own replicated packets. If it were to 874 receive such packets the replication function would make the loop 875 more destructive of bandwidth than a conventional unicast loop. 876 Ultimately the TTL in the S-Label will cause the packet to die during 877 a transient loop, but given the sensitivity of applications to packet 878 latency the impact on the DetNet application would be severe. To 879 avoid the problem of a transient forwarding loop, changes to an LSP 880 supporting DetNet MUST be loop-free. 882 4.5.1. Edge Node Processing 884 A DetNet Edge node operates in the DetNet forwarding sub-layer and 885 service sub-layer. An edge node is responsible for matching incoming 886 packets to the service they require and encapsulating them 887 accordingly. An edge node may perform PRF, PEF, and or POF. Details 888 on encapsulation can be found in Section 4.2; details on PRF can be 889 found in Section 4.2.2.1; details on PEF can be found in 890 Section 4.2.2.2; and details on POF can be found in Section 4.2.2.3. 892 4.5.2. Relay Node Processing 894 A DetNet Relay node operates in the DetNet forwarding sub-layer and 895 service sub-layer. For DetNet using MPLS forwarding related 896 processing is performed on the F-Label. This processing is done 897 within an extended forwarder function. Whether an incoming DetNet 898 flow receives DetNet specific processing depends on how the 899 forwarding is programmed. Some relay nodes may be DetNet service 900 aware for certain DetNet services, while for other DetNet services 901 these nodes may perform as unmodified LSRs that only understand how 902 to switch MPLS-TE LSPs, i.e., as a transit node, see Section 4.4. 903 Again, this is entirely up to how the forwarding has been programmed. 905 During the elimination and replication process the sequence number of 906 an incoming DetNet packet MUST be preserved and carried in the 907 corresponding outgoing DetNet packet. For example, a relay node that 908 performs both PEF and PRF first performs PEF on incoming packets to 909 create a compound flow. It then performs PRF and copies the app-flow 910 data and the d-CW into packets for each outgoing DetNet member flow. 912 The internal design of a relay node is out of scope of this document. 913 However the reader's attention is drawn to the need to make any PREOF 914 state available to the packet processor(s) dealing with packets to 915 which the PREOF functions must be applied, and to maintain that state 916 is such a way that it is available to the packet processor operation 917 on the next packet in the DetNet flow (which may be a duplicate, a 918 late packet, or the next packet in sequence). 920 4.6. Forwarding Sub-Layer Considerations 922 4.6.1. Class of Service 924 Class and quality of service, i.e., CoS and QoS, are terms that are 925 often used interchangeably and confused with each other. In the 926 context of DetNet, CoS is used to refer to mechanisms that provide 927 traffic forwarding treatment based on aggregate group basis and QoS 928 is used to refer to mechanisms that provide traffic forwarding 929 treatment based on a specific DetNet flow basis. Examples of 930 existing network level CoS mechanisms include DiffServ which is 931 enabled by IP header differentiated services code point (DSCP) field 932 [RFC2474] and MPLS label traffic class field [RFC5462], and at Layer- 933 2, by IEEE 802.1p priority code point (PCP). 935 CoS for DetNet flows carried in PWs and MPLS is provided using the 936 existing MPLS Differentiated Services (DiffServ) architecture 937 [RFC3270]. Both E-LSP and L-LSP MPLS DiffServ modes MAY be used to 938 support DetNet flows. The Traffic Class field (formerly the EXP 939 field) of an MPLS label follows the definition of [RFC5462] and 941 [RFC3270]. The Uniform, Pipe, and Short Pipe DiffServ tunneling and 942 TTL processing models are described in [RFC3270] and [RFC3443] and 943 MAY be used for MPLS LSPs supporting DetNet flows. MPLS ECN MAY also 944 be used as defined in ECN [RFC5129] and updated by [RFC5462]. 946 4.6.2. Quality of Service 948 In addition to explicit routes, and packet replication and 949 elimination, described in Section 4 above, DetNet provides zero 950 congestion loss and bounded latency and jitter. As described in 951 [RFC8655], there are different mechanisms that maybe used separately 952 or in combination to deliver a zero congestion loss service. This 953 includes Quality of Service (QoS) mechanisms at the MPLS layer, that 954 may be combined with the mechanisms defined by the underlying network 955 layer such as 802.1TSN. 957 Quality of Service (QoS) mechanisms for flow specific traffic 958 treatment typically includes a guarantee/agreement for the service, 959 and allocation of resources to support the service. Example QoS 960 mechanisms include discrete resource allocation, admission control, 961 flow identification and isolation, and sometimes path control, 962 traffic protection, shaping, policing and remarking. Example 963 protocols that support QoS control include Resource ReSerVation 964 Protocol (RSVP) [RFC2205] (RSVP) and RSVP-TE [RFC3209] and [RFC3473]. 965 The existing MPLS mechanisms defined to support CoS [RFC3270] can 966 also be used to reserve resources for specific traffic classes. 968 A baseline set of QoS capabilities for DetNet flows carried in PWs 969 and MPLS can provided by MPLS with Traffic Engineering (MPLS-TE) 970 [RFC3209] and [RFC3473]. TE LSPs can also support explicit routes 971 (path pinning). Current service definitions for packet TE LSPs can 972 be found in "Specification of the Controlled Load Quality of 973 Service", [RFC2211], "Specification of Guaranteed Quality of 974 Service", [RFC2212], and "Ethernet Traffic Parameters", [RFC6003]. 975 Additional service definitions are expected in future documents to 976 support the full range of DetNet services. In all cases, the 977 existing label-based marking mechanisms defined for TE-LSPs and even 978 E-LSPs are use to support the identification of flows requiring 979 DetNet QoS. 981 5. Management and Control Information Summary 983 The specific information needed for the processing of each DetNet 984 service depends on the DetNet node type and the functions being 985 provided on the node. This section summarizes based on the DetNet 986 sub-layer and if the DetNet traffic is being sent or received. All 987 DetNet node types are DetNet forwarding sub-layer aware, while all 988 but transit nodes are service sub-layer aware. This is shown in 989 Figure 2. 991 Much like other MPLS labels, there are a number of alternatives 992 available for DetNet S-Label and F-Label advertisement to an upstream 993 peer node. These include distributed signaling protocols such as 994 RSVP-TE, centralized label distribution via a controller that manages 995 both the sender and the receiver using NETCONF/YANG, BGP, PCEP, etc., 996 and hybrid combinations of the two. The details of the controller 997 plane solution required for the label distribution and the management 998 of the label number space are out of scope of this document. There 999 are particular DetNet considerations and requirements that are 1000 discussed in [I-D.ietf-detnet-data-plane-framework]. Conformance 1001 language is not used in the summary since it applies to future 1002 mechanisms such as those that may be provided in signaling and YANG 1003 models, e.g., [I-D.ietf-detnet-yang]. 1005 5.1. Service Sub-Layer Information Summary 1007 The following summarizes the information that is needed on service 1008 sub-layer aware nodes that transmit DetNet MPLS traffic, on a per 1009 service basis: 1011 o App-Flow identification information, e.g., IP information as 1012 defined in [I-D.ietf-detnet-ip-over-mpls]. Note, this information 1013 is not needed on DetNet relay nodes. 1015 o The sequence number length to be used for the service. Valid 1016 values included 0, 16 and 28 bits. 0 bits cannot be used when PEF 1017 or POF is configured for the service. 1019 o If PRF is to be provided for the service. 1021 o The outgoing S-Label for each of the service's outgoing DetNet 1022 (member) flow. 1024 o The forwarding sub-layer information associated with the output of 1025 the service sub-layer. Note that when the PRF function is 1026 provisioned, this information is per DetNet member flow. 1027 Logically the forwarding sub-layer information is a pointer to 1028 further details of transmission of Detnet flows at the forwarding 1029 sub-layer. 1031 The following summarizes the information that is needed on service 1032 sub-layer aware nodes that receive DetNet MPLS traffic, on a per 1033 service basis: 1035 o The forwarding sub-layer information associated with the input of 1036 the service sub-layer. Note that when the PEF function is 1037 provisioned, this information is per DetNet member flow. 1038 Logically the forwarding sub-layer information is a pointer to 1039 further details of the reception of Detnet flows at the forwarding 1040 sub-layer or A-Label. 1042 o The incoming S-Label for the service. 1044 o If PEF or POF is to be provided for the service. 1046 o The sequence number length to be used for the service. Valid 1047 values included 0, 16 and 28 bits. 0 bits cannot be used when PEF 1048 or POF are configured for the service. 1050 o App-Flow identification information, e.g., IP information as 1051 defined in [I-D.ietf-detnet-ip-over-mpls]. Note, this information 1052 is not needed on DetNet relay nodes. 1054 5.1.1. Service Aggregation Information Summary 1056 Nodes performing aggregation using A-Labels, per 1057 Section Section 4.4.2, require the additional information summarized 1058 in this section. 1060 The following summarizes the additional information that is needed on 1061 a node that sends aggregated flows using A-Labels: 1063 o The S-Labels or F-Labels that are to be carried over each 1064 aggregated service. 1066 o The A-Label associated with each aggregated service. 1068 o The other S-Label information summarized above. 1070 On the receiving node, the A-Label provides the forwarding context of 1071 an incoming interface or an F-Label and is used in subsequent service 1072 or forwarding sub-layer receive processing, as appropriated. The 1073 related addition configuration that may be provided discussed 1074 elsewhere in this section. 1076 5.2. Forwarding Sub-Layer Information Summary 1078 The following summarizes the information that is needed on forwarding 1079 sub-layer aware nodes that send DetNet MPLS traffic, on a per 1080 forwarding sub-layer flow basis: 1082 o The outgoing F-Label stack to be pushed. The stack may include 1083 H-LSP labels. 1085 o The traffic parameters associated with a specific label in the 1086 stack. Note that there may be multiple sets of traffic paramters 1087 associated with specific labels in the stack, e.g., when H-LSPs 1088 are used. 1090 o Outgoing interface and, for unicast traffic, the next hop 1091 information. 1093 o Sub-network specific parameters on a technology specific basis. 1094 For example, see [I-D.ietf-detnet-mpls-over-tsn]. 1096 The following summarizes the information that is needed on forwarding 1097 sub-layer aware nodes that receive DetNet MPLS traffic, on a per 1098 forwarding sub-layer flow basis: 1100 o The incoming interface. For some implementations and deployment 1101 scenarios, this information may not be needed. 1103 o The incoming F-Label stack to be popped. The stack may include 1104 H-LSP labels. 1106 o How the incoming forwarding sub-layer flow is to be handled, i.e., 1107 forwarded as a transit node, or provided to the service sub-layer. 1109 It is the responsibility of the DetNet controller plane to properly 1110 provision both flow identification information and the flow specific 1111 resources needed to provided the traffic treatment needed to meet 1112 each flow's service requirements. This applies for aggregated and 1113 individual flows. 1115 6. Security Considerations 1117 Detailed security considerations for DetNet are cataloged in 1118 [I-D.ietf-detnet-security], and more general security considerations 1119 are described in [RFC8655]. This section considers exclusively 1120 security considerations which are specific to the DetNet MPLS data 1121 plane. The considerations raised related to MPLS networks in general 1122 in [RFC5920] are equally applicable to the the DetNet MPLS data 1123 plane. 1125 Security aspects which are unique to DetNet are those whose aim is to 1126 provide the specific quality of service aspects of DetNet, which are 1127 primarily to deliver data flows with extremely low packet loss rates 1128 and bounded end-to-end delivery latency. Achieving such loss rates 1129 and bounded latency may not be possible in the face of a highly 1130 capable adversary, such as the one envisioned by the Internet Threat 1131 Model of BCP 72 that can arbitrarily drop or delay any or all 1132 traffic. In order to present meaningful security considerations, we 1133 consider a somewhat weaker attacker who does not control the physical 1134 links of the DetNet domain, but may have the ability to control a 1135 network node within the boundary of the DetNet domain. 1137 The primary consideration for the DetNet data plane is to maintain 1138 integrity of data and delivery of the associated DetNet service 1139 traversing the DetNet network. Application flows can be protected 1140 through whatever means are provided by the underlying technology. 1141 For example, encryption may be used, such as that provided by IPsec 1142 [RFC4301] for IP flows and/or by an underlying sub-net using MACSec 1143 [IEEE802.1AE-2018] for IP over Ethernet (Layer-2) flows. 1145 From a data plane perspective this document does not add or modify 1146 any header information. 1148 At the management and control level DetNet flows are identified on a 1149 per-flow basis, which may provide controller plane attackers with 1150 additional information about the data flows (when compared to 1151 controller planes that do not include per-flow identification). This 1152 is an inherent property of DetNet which has security implications 1153 that should be considered when determining if DetNet is a suitable 1154 technology for any given use case. 1156 To provide uninterrupted availability of the DetNet service, 1157 provisions can be made against DOS attacks and delay attacks. To 1158 protect against DOS attacks, excess traffic due to malicious or 1159 malfunctioning devices is prevented or mitigated through the use of 1160 existing mechanisms, for example by policing and shaping incoming 1161 traffic. To prevent DetNet packets from being delayed by an entity 1162 external to a DetNet domain, DetNet technology definition can allow 1163 for the mitigation of Man-In-The-Middle attacks, for example through 1164 use of authentication and authorization of devices within the DetNet 1165 domain. 1167 7. IANA Considerations 1169 This document makes no IANA requests. 1171 8. Acknowledgements 1173 The authors wish to thank Pat Thaler, Norman Finn, Loa Anderson, 1174 David Black, Rodney Cummings, Ethan Grossman, Tal Mizrahi, David 1175 Mozes, Craig Gunther, George Swallow, Yuanlong Jiang, Jeong-dong Ryoo 1176 and Carlos J. Bernardos for their various contributions to this 1177 work. 1179 9. Contributors 1181 RFC7322 limits the number of authors listed on the front page of a 1182 draft to a maximum of 5. The editor wishes to thank and acknowledge 1183 the follow author for contributing text to this draft. 1185 Don Fedyk 1186 LabN Consulting, L.L.C. 1187 Email: dfedyk@labn.net 1189 10. References 1191 10.1. Normative References 1193 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1194 Requirement Levels", BCP 14, RFC 2119, 1195 DOI 10.17487/RFC2119, March 1997, 1196 . 1198 [RFC2211] Wroclawski, J., "Specification of the Controlled-Load 1199 Network Element Service", RFC 2211, DOI 10.17487/RFC2211, 1200 September 1997, . 1202 [RFC2212] Shenker, S., Partridge, C., and R. Guerin, "Specification 1203 of Guaranteed Quality of Service", RFC 2212, 1204 DOI 10.17487/RFC2212, September 1997, 1205 . 1207 [RFC3031] Rosen, E., Viswanathan, A., and R. Callon, "Multiprotocol 1208 Label Switching Architecture", RFC 3031, 1209 DOI 10.17487/RFC3031, January 2001, 1210 . 1212 [RFC3032] Rosen, E., Tappan, D., Fedorkow, G., Rekhter, Y., 1213 Farinacci, D., Li, T., and A. Conta, "MPLS Label Stack 1214 Encoding", RFC 3032, DOI 10.17487/RFC3032, January 2001, 1215 . 1217 [RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V., 1218 and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP 1219 Tunnels", RFC 3209, DOI 10.17487/RFC3209, December 2001, 1220 . 1222 [RFC3270] Le Faucheur, F., Wu, L., Davie, B., Davari, S., Vaananen, 1223 P., Krishnan, R., Cheval, P., and J. Heinanen, "Multi- 1224 Protocol Label Switching (MPLS) Support of Differentiated 1225 Services", RFC 3270, DOI 10.17487/RFC3270, May 2002, 1226 . 1228 [RFC3443] Agarwal, P. and B. Akyol, "Time To Live (TTL) Processing 1229 in Multi-Protocol Label Switching (MPLS) Networks", 1230 RFC 3443, DOI 10.17487/RFC3443, January 2003, 1231 . 1233 [RFC3473] Berger, L., Ed., "Generalized Multi-Protocol Label 1234 Switching (GMPLS) Signaling Resource ReserVation Protocol- 1235 Traffic Engineering (RSVP-TE) Extensions", RFC 3473, 1236 DOI 10.17487/RFC3473, January 2003, 1237 . 1239 [RFC4206] Kompella, K. and Y. Rekhter, "Label Switched Paths (LSP) 1240 Hierarchy with Generalized Multi-Protocol Label Switching 1241 (GMPLS) Traffic Engineering (TE)", RFC 4206, 1242 DOI 10.17487/RFC4206, October 2005, 1243 . 1245 [RFC4385] Bryant, S., Swallow, G., Martini, L., and D. McPherson, 1246 "Pseudowire Emulation Edge-to-Edge (PWE3) Control Word for 1247 Use over an MPLS PSN", RFC 4385, DOI 10.17487/RFC4385, 1248 February 2006, . 1250 [RFC5085] Nadeau, T., Ed. and C. Pignataro, Ed., "Pseudowire Virtual 1251 Circuit Connectivity Verification (VCCV): A Control 1252 Channel for Pseudowires", RFC 5085, DOI 10.17487/RFC5085, 1253 December 2007, . 1255 [RFC5129] Davie, B., Briscoe, B., and J. Tay, "Explicit Congestion 1256 Marking in MPLS", RFC 5129, DOI 10.17487/RFC5129, January 1257 2008, . 1259 [RFC5462] Andersson, L. and R. Asati, "Multiprotocol Label Switching 1260 (MPLS) Label Stack Entry: "EXP" Field Renamed to "Traffic 1261 Class" Field", RFC 5462, DOI 10.17487/RFC5462, February 1262 2009, . 1264 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 1265 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 1266 May 2017, . 1268 [RFC8655] Finn, N., Thubert, P., Varga, B., and J. Farkas, 1269 "Deterministic Networking Architecture", RFC 8655, 1270 DOI 10.17487/RFC8655, October 2019, 1271 . 1273 10.2. Informative References 1275 [I-D.ietf-detnet-data-plane-framework] 1276 Varga, B., Farkas, J., Berger, L., Malis, A., and S. 1277 Bryant, "DetNet Data Plane Framework", draft-ietf-detnet- 1278 data-plane-framework-06 (work in progress), May 2020. 1280 [I-D.ietf-detnet-ip] 1281 Varga, B., Farkas, J., Berger, L., Fedyk, D., and S. 1282 Bryant, "DetNet Data Plane: IP", draft-ietf-detnet-ip-07 1283 (work in progress), July 2020. 1285 [I-D.ietf-detnet-ip-over-mpls] 1286 Varga, B., Berger, L., Fedyk, D., Bryant, S., and J. 1287 Korhonen, "DetNet Data Plane: IP over MPLS", draft-ietf- 1288 detnet-ip-over-mpls-06 (work in progress), May 2020. 1290 [I-D.ietf-detnet-mpls-over-tsn] 1291 Varga, B., Farkas, J., Malis, A., and S. Bryant, "DetNet 1292 Data Plane: MPLS over IEEE 802.1 Time Sensitive Networking 1293 (TSN)", draft-ietf-detnet-mpls-over-tsn-03 (work in 1294 progress), June 2020. 1296 [I-D.ietf-detnet-security] 1297 Mizrahi, T. and E. Grossman, "Deterministic Networking 1298 (DetNet) Security Considerations", draft-ietf-detnet- 1299 security-10 (work in progress), May 2020. 1301 [I-D.ietf-detnet-yang] 1302 Geng, X., Chen, M., Ryoo, Y., Fedyk, D., Li, Z., and R. 1303 Rahman, "Deterministic Networking (DetNet) Configuration 1304 YANG Model", draft-ietf-detnet-yang-07 (work in progress), 1305 July 2020. 1307 [IEEE802.1AE-2018] 1308 IEEE Standards Association, "IEEE Std 802.1AE-2018 MAC 1309 Security (MACsec)", 2018, 1310 . 1312 [RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S. 1313 Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1 1314 Functional Specification", RFC 2205, DOI 10.17487/RFC2205, 1315 September 1997, . 1317 [RFC2474] Nichols, K., Blake, S., Baker, F., and D. Black, 1318 "Definition of the Differentiated Services Field (DS 1319 Field) in the IPv4 and IPv6 Headers", RFC 2474, 1320 DOI 10.17487/RFC2474, December 1998, 1321 . 1323 [RFC3272] Awduche, D., Chiu, A., Elwalid, A., Widjaja, I., and X. 1324 Xiao, "Overview and Principles of Internet Traffic 1325 Engineering", RFC 3272, DOI 10.17487/RFC3272, May 2002, 1326 . 1328 [RFC3985] Bryant, S., Ed. and P. Pate, Ed., "Pseudo Wire Emulation 1329 Edge-to-Edge (PWE3) Architecture", RFC 3985, 1330 DOI 10.17487/RFC3985, March 2005, 1331 . 1333 [RFC4301] Kent, S. and K. Seo, "Security Architecture for the 1334 Internet Protocol", RFC 4301, DOI 10.17487/RFC4301, 1335 December 2005, . 1337 [RFC4448] Martini, L., Ed., Rosen, E., El-Aawar, N., and G. Heron, 1338 "Encapsulation Methods for Transport of Ethernet over MPLS 1339 Networks", RFC 4448, DOI 10.17487/RFC4448, April 2006, 1340 . 1342 [RFC4875] Aggarwal, R., Ed., Papadimitriou, D., Ed., and S. 1343 Yasukawa, Ed., "Extensions to Resource Reservation 1344 Protocol - Traffic Engineering (RSVP-TE) for Point-to- 1345 Multipoint TE Label Switched Paths (LSPs)", RFC 4875, 1346 DOI 10.17487/RFC4875, May 2007, 1347 . 1349 [RFC5440] Vasseur, JP., Ed. and JL. Le Roux, Ed., "Path Computation 1350 Element (PCE) Communication Protocol (PCEP)", RFC 5440, 1351 DOI 10.17487/RFC5440, March 2009, 1352 . 1354 [RFC5586] Bocci, M., Ed., Vigoureux, M., Ed., and S. Bryant, Ed., 1355 "MPLS Generic Associated Channel", RFC 5586, 1356 DOI 10.17487/RFC5586, June 2009, 1357 . 1359 [RFC5920] Fang, L., Ed., "Security Framework for MPLS and GMPLS 1360 Networks", RFC 5920, DOI 10.17487/RFC5920, July 2010, 1361 . 1363 [RFC5921] Bocci, M., Ed., Bryant, S., Ed., Frost, D., Ed., Levrau, 1364 L., and L. Berger, "A Framework for MPLS in Transport 1365 Networks", RFC 5921, DOI 10.17487/RFC5921, July 2010, 1366 . 1368 [RFC6003] Papadimitriou, D., "Ethernet Traffic Parameters", 1369 RFC 6003, DOI 10.17487/RFC6003, October 2010, 1370 . 1372 [RFC6073] Martini, L., Metz, C., Nadeau, T., Bocci, M., and M. 1373 Aissaoui, "Segmented Pseudowire", RFC 6073, 1374 DOI 10.17487/RFC6073, January 2011, 1375 . 1377 [RFC6790] Kompella, K., Drake, J., Amante, S., Henderickx, W., and 1378 L. Yong, "The Use of Entropy Labels in MPLS Forwarding", 1379 RFC 6790, DOI 10.17487/RFC6790, November 2012, 1380 . 1382 [RFC8306] Zhao, Q., Dhody, D., Ed., Palleti, R., and D. King, 1383 "Extensions to the Path Computation Element Communication 1384 Protocol (PCEP) for Point-to-Multipoint Traffic 1385 Engineering Label Switched Paths", RFC 8306, 1386 DOI 10.17487/RFC8306, November 2017, 1387 . 1389 [RFC8660] Bashandy, A., Ed., Filsfils, C., Ed., Previdi, S., 1390 Decraene, B., Litkowski, S., and R. Shakir, "Segment 1391 Routing with the MPLS Data Plane", RFC 8660, 1392 DOI 10.17487/RFC8660, December 2019, 1393 . 1395 Authors' Addresses 1397 Balazs Varga (editor) 1398 Ericsson 1399 Magyar Tudosok krt. 11. 1400 Budapest 1117 1401 Hungary 1403 Email: balazs.a.varga@ericsson.com 1404 Janos Farkas 1405 Ericsson 1406 Magyar Tudosok krt. 11. 1407 Budapest 1117 1408 Hungary 1410 Email: janos.farkas@ericsson.com 1412 Lou Berger 1413 LabN Consulting, L.L.C. 1415 Email: lberger@labn.net 1417 Andrew G. Malis 1418 Malis Consulting 1420 Email: agmalis@gmail.com 1422 Stewart Bryant 1423 Futurewei Technologies 1425 Email: stewart.bryant@gmail.com 1427 Jouni Korhonen 1429 Email: jouni.nospam@gmail.com