idnits 2.17.1 draft-ietf-detnet-mpls-13.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (October 11, 2020) is 1294 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'Network' is mentioned on line 281, but not defined ** Downref: Normative reference to an Informational draft: draft-ietf-detnet-data-plane-framework (ref. 'I-D.ietf-detnet-data-plane-framework') == Outdated reference: A later version (-09) exists of draft-ietf-detnet-ip-over-mpls-08 == Outdated reference: A later version (-07) exists of draft-ietf-detnet-mpls-over-tsn-03 == Outdated reference: A later version (-16) exists of draft-ietf-detnet-security-12 == Outdated reference: A later version (-20) exists of draft-ietf-detnet-yang-07 -- Obsolete informational reference (is this intentional?): RFC 3272 (Obsoleted by RFC 9522) Summary: 1 error (**), 0 flaws (~~), 6 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 DetNet B. Varga, Ed. 3 Internet-Draft J. Farkas 4 Intended status: Standards Track Ericsson 5 Expires: April 14, 2021 L. Berger 6 LabN Consulting, L.L.C. 7 A. Malis 8 Malis Consulting 9 S. Bryant 10 Futurewei Technologies 11 J. Korhonen 12 October 11, 2020 14 DetNet Data Plane: MPLS 15 draft-ietf-detnet-mpls-13 17 Abstract 19 This document specifies the Deterministic Networking data plane when 20 operating over an MPLS Packet Switched Network. It leverages 21 existing pseudowire (PW) encapsulations and MPLS Traffic Engineering 22 encapsulations and mechanisms. This document builds on the DetNet 23 Architecture and Data Plane Framework. 25 Status of This Memo 27 This Internet-Draft is submitted in full conformance with the 28 provisions of BCP 78 and BCP 79. 30 Internet-Drafts are working documents of the Internet Engineering 31 Task Force (IETF). Note that other groups may also distribute 32 working documents as Internet-Drafts. The list of current Internet- 33 Drafts is at https://datatracker.ietf.org/drafts/current/. 35 Internet-Drafts are draft documents valid for a maximum of six months 36 and may be updated, replaced, or obsoleted by other documents at any 37 time. It is inappropriate to use Internet-Drafts as reference 38 material or to cite them other than as "work in progress." 40 This Internet-Draft will expire on April 14, 2021. 42 Copyright Notice 44 Copyright (c) 2020 IETF Trust and the persons identified as the 45 document authors. All rights reserved. 47 This document is subject to BCP 78 and the IETF Trust's Legal 48 Provisions Relating to IETF Documents 49 (https://trustee.ietf.org/license-info) in effect on the date of 50 publication of this document. Please review these documents 51 carefully, as they describe your rights and restrictions with respect 52 to this document. Code Components extracted from this document must 53 include Simplified BSD License text as described in Section 4.e of 54 the Trust Legal Provisions and are provided without warranty as 55 described in the Simplified BSD License. 57 Table of Contents 59 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 60 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 61 2.1. Terms Used in This Document . . . . . . . . . . . . . . . 4 62 2.2. Abbreviations . . . . . . . . . . . . . . . . . . . . . . 4 63 2.3. Requirements Language . . . . . . . . . . . . . . . . . . 5 64 3. DetNet MPLS Data Plane Overview . . . . . . . . . . . . . . . 5 65 3.1. Layers of DetNet Data Plane . . . . . . . . . . . . . . . 5 66 3.2. DetNet MPLS Data Plane Scenarios . . . . . . . . . . . . 6 67 4. MPLS-Based DetNet Data Plane Solution . . . . . . . . . . . . 8 68 4.1. DetNet Over MPLS Encapsulation Components . . . . . . . . 8 69 4.2. MPLS Data Plane Encapsulation . . . . . . . . . . . . . . 9 70 4.2.1. DetNet Control Word and the DetNet Sequence Number . 10 71 4.2.2. S-Labels . . . . . . . . . . . . . . . . . . . . . . 12 72 4.2.3. F-Labels . . . . . . . . . . . . . . . . . . . . . . 15 73 4.3. OAM Indication . . . . . . . . . . . . . . . . . . . . . 17 74 4.4. Flow Aggregation . . . . . . . . . . . . . . . . . . . . 18 75 4.4.1. Aggregation Via LSP Hierarchy . . . . . . . . . . . . 18 76 4.4.2. Aggregating DetNet Flows as a new DetNet flow . . . . 19 77 4.5. Service Sub-Layer Considerations . . . . . . . . . . . . 20 78 4.5.1. Edge Node Processing . . . . . . . . . . . . . . . . 20 79 4.5.2. Relay Node Processing . . . . . . . . . . . . . . . . 20 80 4.6. Forwarding Sub-Layer Considerations . . . . . . . . . . . 21 81 4.6.1. Class of Service . . . . . . . . . . . . . . . . . . 21 82 4.6.2. Quality of Service . . . . . . . . . . . . . . . . . 21 83 5. Management and Control Information Summary . . . . . . . . . 22 84 5.1. Service Sub-Layer Information Summary . . . . . . . . . . 23 85 5.1.1. Service Aggregation Information Summary . . . . . . . 24 86 5.2. Forwarding Sub-Layer Information Summary . . . . . . . . 24 87 6. Security Considerations . . . . . . . . . . . . . . . . . . . 25 88 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 26 89 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 26 90 9. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 26 91 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 27 92 10.1. Normative References . . . . . . . . . . . . . . . . . . 27 93 10.2. Informative References . . . . . . . . . . . . . . . . . 29 94 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 31 96 1. Introduction 98 Deterministic Networking (DetNet) is a service that can be offered by 99 a network to DetNet flows. DetNet provides a capability for the 100 delivery of data flows with extremely low packet loss rates and 101 bounded end-to-end delivery latency. General background and concepts 102 of DetNet can be found in the DetNet Architecture [RFC8655]. 104 The purpose of this document is to describe the use of the MPLS data 105 plane to establish and support DetNet flows. The DetNet Architecture 106 models the DetNet related data plane functions decomposed into two 107 sub-layers: a service sub-layer and a forwarding sub-layer. The 108 service sub-layer is used to provide DetNet service functions such as 109 protection and reordering. At the DetNet data plane a new set of 110 functions (PREOF) provide the service sub-layer specific tasks. The 111 forwarding sub-layer is used to provide forwarding assurance (low 112 loss, assured latency, and limited out-of-order delivery). The use 113 of the functionalities of the DetNet service sub-layer and the DetNet 114 forwarding sub-layer require careful design and control by the 115 controller plane in addition to the DetNet specific use of MPLS 116 encapsulation as specified by this document. 118 This document specifies the DetNet data plane operation and the on- 119 wire encapsulation of DetNet flows over an MPLS-based Packet Switched 120 Network (PSN) using the service reference model. MPLS encapsulation 121 already provides a solid foundation of building blocks to enable the 122 DetNet service and forwarding sub-layer functions. MPLS encapsulated 123 DetNet can be carried over a variety of different network 124 technologies that can provide the DetNet required level of service. 125 However, the specific details of how DetNet MPLS is carried over 126 different network technologies are out of scope of this document. 128 MPLS encapsulated DetNet flows can carry different types of traffic. 129 The details of the types of traffic that are carried in DetNet are 130 also out of scope of this document. An example of IP using DetNet 131 MPLS sub-networks can be found in [I-D.ietf-detnet-ip]. DetNet MPLS 132 may use an associated controller and Operations, Administration, and 133 Maintenance (OAM) functions that are defined outside of this 134 document. 136 Background information common to all data planes for DetNet can be 137 found in the DetNet Data Plane Framework 138 [I-D.ietf-detnet-data-plane-framework]. 140 2. Terminology 142 2.1. Terms Used in This Document 144 This document uses the terminology established in the DetNet 145 architecture [RFC8655] and the DetNet Data Plane Framework 146 [I-D.ietf-detnet-data-plane-framework]. The reader is assumed to be 147 familiar with these documents, any terminology defined therein and 148 basic MPLS related terminologies in [RFC3031]. 150 The following terminology is introduced in this document: 152 F-Label A Detnet "forwarding" label that identifies the LSP 153 used to forward a DetNet flow across an MPLS PSN, i.e., 154 a hop-by-hop label used between label switching routers 155 (LSR). 157 S-Label A DetNet "service" label that is used between DetNet 158 nodes that implement the DetNet service sub-layer 159 functions. An S-Label is used to identify a DetNet 160 flow at DetNet service sub-layer at a receiving DetNet 161 node. 163 A-Label A special case of an S-Label, whose aggregation 164 properties are known only at the aggregation and 165 deaggregation end-points. 167 d-CW A DetNet Control Word (d-CW) is used for sequencing 168 information of a DetNet flow at the DetNet service sub- 169 layer. 171 2.2. Abbreviations 173 The following abbreviations are used in this document: 175 CoS Class of Service. 177 CW Control Word. 179 DetNet Deterministic Networking. 181 LSR Label Switching Router. 183 MPLS Multiprotocol Label Switching. 185 MPLS-TE Multiprotocol Label Switching - Traffic Engineering. 187 MPLS-TP Multiprotocol Label Switching - Transport Profile. 189 OAM Operations, Administration, and Maintenance. 191 PE Provider Edge. 193 PEF Packet Elimination Function. 195 PRF Packet Replication Function. 197 PREOF Packet Replication, Elimination and Ordering Functions. 199 POF Packet Ordering Function. 201 PSN Packet Switched Network. 203 PW PseudoWire. 205 QoS Quality of Service. 207 S-PE Switching Provider Edge. 209 T-PE Terminating Provider Edge. 211 TSN Time-Sensitive Network. 213 2.3. Requirements Language 215 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 216 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 217 "OPTIONAL" in this document are to be interpreted as described in BCP 218 14 [RFC2119] [RFC8174] when, and only when, they appear in all 219 capitals, as shown here. 221 3. DetNet MPLS Data Plane Overview 223 3.1. Layers of DetNet Data Plane 225 MPLS provides a wide range of capabilities that can be utilised by 226 DetNet. A straight forward approach utilizing MPLS for a DetNet 227 service sub-layer is based on the existing pseudowire (PW) 228 encapsulations and by utilizing existing MPLS Traffic Engineering 229 encapsulations and mechanisms. Background on PWs can be found in 230 [RFC3985] and [RFC3031]. Background on MPLS Traffic Engineering can 231 be found in [RFC3272] and [RFC3209]. 233 DetNet MPLS 234 . 235 Bottom of Stack . 236 (inner label) +------------+ 237 | Service | d-CW, S-Label (A-Label) 238 +------------+ 239 | Forwarding | F-Label(s) 240 +------------+ 241 Top of Stack . 242 (outer label) . 244 Figure 1: DetNet Adaptation to MPLS Data Plane 246 The DetNet MPLS data plane representation is illustrated in Figure 1. 247 The service sub-layer includes a DetNet control word (d-CW) and an 248 identifying service label (S-Label). The DetNet control word (d-CW) 249 conforms to the Generic PW MPLS Control Word (PWMCW) defined in 250 [RFC4385]. An aggregation label (A-Label) is a special case of 251 S-Label used for aggregation. 253 A node operating on a received DetNet flow at the Detnet service sub- 254 layer uses the local context associated with a received S-Label, 255 i.e., a received F-Label, to determine which local DetNet 256 operation(s) are applied to that packet. An S-Label may be taken 257 from the platform label space [RFC3031], making it unique, enabling 258 DetNet flow identification regardless of which input interface or LSP 259 the packet arrives on. It is important to note that S-Label values 260 are driven by the receiver, not the sender. 262 The DetNet forwarding sub-layer is supported by zero or more 263 forwarding labels (F-Labels). MPLS Traffic Engineering 264 encapsulations and mechanisms can be utilized to provide a forwarding 265 sub-layer that is responsible for providing resource allocation and 266 explicit routes. 268 3.2. DetNet MPLS Data Plane Scenarios 269 DetNet MPLS Relay Transit Relay DetNet MPLS 270 End System Node Node Node End System 271 (T-PE) (S-PE) (LSR) (S-PE) (T-PE) 272 +----------+ +----------+ 273 | Appl. |<------------ End to End Service ----------->| Appl. | 274 +----------+ +---------+ +---------+ +----------+ 275 | Service |<--| Service |-- DetNet flow --| Service |-->| Service | 276 +----------+ +---------+ +----------+ +---------+ +----------+ 277 |Forwarding| |Fwd| |Fwd| |Forwarding| |Fwd| |Fwd| |Forwarding| 278 +-------.--+ +-.-+ +-.-+ +----.---.-+ +-.-+ +-.-+ +---.------+ 279 : Link : / ,-----. \ : Link : / ,-----. \ 280 +........+ +-[ Sub ]-+ +......+ +-[ Sub ]-+ 281 [Network] [Network] 282 `-----' `-----' 283 |<- LSP -->| |<-------- LSP -----------| |<--- LSP -->| 285 |<----------------- DetNet MPLS --------------------->| 287 Figure 2: A DetNet MPLS Network 289 Figure 2 illustrates a hypothetical DetNet MPLS-only network composed 290 of DetNet aware MPLS enabled end systems, operating over a DetNet 291 aware MPLS network. In this figure, the relay nodes are PE devices 292 that define the MPLS LSP boundaries and the transit nodes are LSRs. 294 DetNet end systems and relay nodes understand the particular needs of 295 DetNet flows and provide both DetNet service and forwarding sub-layer 296 functions. In the case of MPLS, DetNet service-aware nodes add, 297 remove and process d-CWs, S-Labels and F-labels as needed. DetNet 298 MPLS nodes provide functionality analogous to T-PEs when they sit at 299 the edge of an MPLS domain, and S-PEs when they are in the middle of 300 an MPLS domain, see [RFC6073]. 302 In a DetNet MPLS network, transit nodes may be DetNet service aware 303 or may be DetNet unaware MPLS Label Switching Routers (LSRs). In 304 this latter case, such LSRs would be unaware of the special 305 requirements of the DetNet service sub-layer, but would still provide 306 traffic engineering functions and the QoS capabilities needed to 307 ensure that the (TE) LSPs meet the service requirements of the 308 carried DetNet flows. 310 The application of DetNet using MPLS supports a number of control 311 plane/management plane types. These types support certain MPLS data 312 plane deployments. For example RSVP-TE, MPLS-TP, or MPLS Segment 313 Routing (when extended to support resource allocation) are all valid 314 MPLS deployment cases. 316 Figure 3 illustrates how an end-to-end MPLS-based DetNet service is 317 provided in a more detail. In this figure, the customer end systems, 318 CE1 and CE2, are able to send and receive MPLS encapsulated DetNet 319 flows, and R1, R2 and R3 are relay nodes in the middle of a DetNet 320 network. The 'X' in the end systems, and relay nodes represents 321 potential DetNet compound flow packet replication and elimination 322 points. In this example, service protection is supported utilizing 323 at least two DetNet member flows and TE LSPs. For a unidirectional 324 flow, R1 supports PRF and R3 supports PEF and POF. Note that the 325 relay nodes may change the underlying forwarding sub-layer, for 326 example tunneling MPLS over IEEE 802.1 TSN 327 [I-D.ietf-detnet-mpls-over-tsn], or simply over interconnect network 328 links. 330 DetNet DetNet 331 DetNet Service Transit Transit Service DetNet 332 MPLS | |<-Tnl->| |<-Tnl->| | MPLS 333 End | V 1 V V 2 V | End 334 System | +--------+ +--------+ +--------+ | System 335 +---+ | | R1 |=======| R2 |=======| R3 | | +---+ 336 | X...DFa...|._X_....|..DF1..|.__ ___.|..DF3..|...._X_.|.DFa..|.X | 337 |CE1|========| \ | | X | | / |======|CE2| 338 | | | | \_.|..DF2..|._/ \__.|..DF4..|._/ | | | | 339 +---+ | |=======| |=======| | +---+ 340 ^ +--------+ +--------+ +--------+ ^ 341 | Relay Node Relay Node Relay Node | 342 | (S-PE) (S-PE) (S-PE) | 343 | | 344 |<---------------------- DetNet MPLS --------------------->| 345 | | 346 |<--------------- End to End DetNet Service -------------->| 348 -------------------------- Data Flow -------------------------> 350 X = Optional service protection (none, PRF, PREOF, PEF/POF) 351 DFx = DetNet member flow x over a TE LSP 353 Figure 3: MPLS-Based Native DetNet 355 4. MPLS-Based DetNet Data Plane Solution 357 4.1. DetNet Over MPLS Encapsulation Components 359 To carry DetNet over MPLS the following is required: 361 1. A method of identifying the MPLS payload type. 363 2. A method of identifying the DetNet flow(s) to the processing 364 element. 366 3. A method of distinguishing DetNet OAM packets from DetNet data 367 packets. 369 4. A method of carrying the DetNet sequence number. 371 5. A suitable LSP to deliver the packet to the egress PE. 373 6. A method of carrying queuing and forwarding indication. 375 In this design an MPLS service label (the S-Label), is similar to a 376 pseudowire (PW) label [RFC3985], and is used to identify both the 377 DetNet flow identity and the payload MPLS payload type satisfying (1) 378 and (2) in the list above. OAM traffic discrimination happens 379 through the use of the Associated Channel method described in 380 [RFC4385]. The DetNet sequence number is carried in the DetNet 381 Control word which carries the Data/OAM discriminator. To simplify 382 implementation and to maximize interoperability two sequence number 383 sizes are supported: a 16 bit sequence number and a 28 bit sequence 384 number. The 16 bit sequence number is needed to support some types 385 of legacy clients. The 28 bit sequence number is used in situations 386 where it is necessary ensure that in high speed networks the sequence 387 number space does not wrap whilst packets are in flight. 389 The LSP used to forward the DetNet packet is not restricted regarding 390 any method used for establishing that LSP (for example, MPLS-LDP, 391 MPLS-TE, MPLS-TP [RFC5921], MPLS-SR [RFC8660], etc.). The LSP 392 (F-Label) label(s) and/or the S-Label may be used to indicate the 393 required queue processing as well as the forwarding parameters. Note 394 that the possible use of Penultimate Hop Popping (PHP) means that the 395 S-Label may be the only label received at the terminating DetNet 396 service. 398 4.2. MPLS Data Plane Encapsulation 400 Figure 4 illustrates a DetNet data plane MPLS encapsulation. The 401 MPLS-based encapsulation of the DetNet flows is well suited for the 402 scenarios described in [I-D.ietf-detnet-data-plane-framework]. 403 Furthermore, an end-to-end DetNet service i.e., native DetNet 404 deployment (see Section 3.2) is also possible if DetNet end systems 405 are capable of initiating and termination MPLS encapsulated packets. 407 The MPLS-based DetNet data plane encapsulation consists of: 409 o DetNet control word (d-CW) containing sequencing information for 410 packet replication and duplicate elimination purposes, and the OAM 411 indicator. 413 o DetNet service Label (S-Label) that identifies a DetNet flow at 414 the receiving DetNet service sub-layer processing node. 416 o Zero or more Detnet MPLS Forwarding label(s) (F-Label) used to 417 direct the packet along the label switched path (LSP) to the next 418 DetNet service sub-layer processing node along the path. When 419 Penultimate Hop Popping is in use there may be no label F-Label in 420 the protocol stack on the final hop. 422 o The necessary data-link encapsulation is then applied prior to 423 transmission over the physical media. 425 DetNet MPLS-based encapsulation 427 +---------------------------------+ 428 | | 429 | DetNet App-Flow | 430 | Payload Packet | 431 | | 432 +---------------------------------+ <--\ 433 | DetNet Control Word | | 434 +---------------------------------+ +--> DetNet data plane 435 | S-Label | | MPLS encapsulation 436 +---------------------------------+ | 437 | [ F-Label(s) ] | | 438 +---------------------------------+ <--/ 439 | Data-Link | 440 +---------------------------------+ 441 | Physical | 442 +---------------------------------+ 444 Figure 4: Encapsulation of a DetNet App-Flow in an MPLS PSN 446 4.2.1. DetNet Control Word and the DetNet Sequence Number 448 A DetNet control word (d-CW) conforms to the Generic PW MPLS Control 449 Word (PWMCW) defined in [RFC4385]. The d-CW formatted as shown in 450 Figure 5 MUST be present in all DetNet packets containing app-flow 451 data. This format of the d-CW was created in order (1) to allow 452 larger sequence number space to avoid sequence number rollover 453 frequency in some applications and (2) to allow sequence numbering 454 systems that include the value zero as a valid sequence number, which 455 simplifies implementation. 457 0 1 2 3 458 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 459 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 460 |0 0 0 0| Sequence Number | 461 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 463 Figure 5: DetNet Control Word 465 (bits 0 to 3) 467 Per [RFC4385], MUST be set to zero (0). 469 Sequence Number (bits 4 to 31) 471 An unsigned value implementing the DetNet sequence number. The 472 sequence number space is a circular one with no restriction on 473 initial value. 475 A separate sequence number space MUST be maintained by the node that 476 adds the d-CW for each DetNet app-flow, i.e., DetNet service. The 477 following sequence number field lengths MUST be supported: 479 0 bits 481 16 bits 483 28 bits 485 The sequence number length MUST be provisioned on a per Detnet 486 service basis via configuration, i.e., via the controller plane 487 described in [I-D.ietf-detnet-data-plane-framework]. 489 A 0 bit sequence number field length indicates that there is no 490 DetNet sequence number used for the flow. When the length is zero, 491 the sequence number field MUST be set to zero (0) on all packets sent 492 for the flow. 494 When the sequence number field length is 16 or 28 bits for a flow, 495 the sequence number MUST be incremented by one for each new app-flow 496 packet sent. When the field length is 16 bits, d-CW bits 4 to 15 497 MUST be set to zero (0). The values carried in this field can wrap 498 and it is important to note that zero (0) is a valid field value. 499 For example, where the sequence number size is 16 bits, the sequence 500 will contain: 65535, 0, 1, where zero (0) is an ordinary sequence 501 number. 503 It is important to note that this document differs from [RFC4448] 504 where a sequence number of zero (0) is used to indicate that the 505 sequence number check algorithm is not used. 507 The sequence number is optionally used during receive processing as 508 described below in Section 4.2.2.2 and Section 4.2.2.3. 510 4.2.2. S-Labels 512 A DetNet flow at the DetNet service sub-layer is identified by an 513 S-Label. MPLS-aware DetNet end systems and edge nodes, which are by 514 definition MPLS ingress and egress nodes, MUST add (push) and remove 515 (pop) a DetNet service-specific d-CW and S-Label. Relay nodes MAY 516 swap S-Label values when processing a DetNet flow, i.e., incoming and 517 outgoing S-Labels of a DetNet flow can be different. 519 S-Label values MUST be provisioned per DetNet service via 520 configuration, i.e., via the controller plane described in 521 [I-D.ietf-detnet-data-plane-framework]. Note that S-Labels provide 522 identification at the downstream DetNet service sub-layer receiver, 523 not the sender. As such, S-Labels MUST be allocated by the entity 524 that controls the service sub-layer receiving node's label space, and 525 MAY be allocated from the platform label space [RFC3031]. Because 526 S-Labels are local to each node rather than being a global identifier 527 within a domain, they must be advertised to their upstream DetNet 528 service-aware peer nodes (i.e., a DetNet MPLS End System or a DetNet 529 Relay or Edge Node) and interpreted in the context of their received 530 F-Label(s). In some PREOF topologies, the node performing 531 replication will be sending to multiple nodes performing PEF or POF, 532 and may need to send different S-Label values for the different 533 member flows for the same DetNet service. 535 An S-Label will normally be at the bottom of the label stack once the 536 last F-Label is removed, immediately preceding the d-CW. To support 537 service sub-layer level OAM, an OAM Associated Channel Header (ACH) 538 [RFC4385] together with a Generic Associated Channel Label (GAL) 539 [RFC5586] MAY be used in place of a d-CW. 541 Similarly, an Entropy Label Indicator/Entropy Label (ELI/EL) 542 [RFC6790] MAY be carried below the S-Label in the label stack in 543 networks where DetNet flows would otherwise receive ECMP treatment. 544 When ELs are used, the same EL value SHOULD be used for all of the 545 packets sent using a specific S-Label to force the flow to follow the 546 same path. However, as outlined in 547 [I-D.ietf-detnet-data-plane-framework] the use of ECMP for DetNet 548 flows is NOT RECOMMENDED. ECMP MAY be used for non-DetNet flows 549 within a DetNet domain. 551 When receiving a DetNet MPLS packet, an implementation MUST identify 552 the DetNet service associated with the incoming packet based on the 553 S-Label. When a node is using platform labels for S-Labels, no 554 additional information is needed as the S-label uniquely identifies 555 the DetNet service. In the case where platform labels are not used, 556 zero or more F-Labels proceeding the S-Label MUST be used together 557 with the S-Label to uniquely identify the DetNet service associated 558 with a received packet. The incoming interface MAY also be used 559 together with any present F-Label(s) and the S-Label to uniquely 560 identify an incoming DetNet service, for example, in the case where 561 PHP is used. Note that choice to use platform label space for 562 S-Label or S-Label plus one or more F-Labels to identify DetNet 563 services is a local implementation choice, with one caveat. When one 564 or more F-labels, or incoming interface, is needed together with an 565 S-Label to uniquely identify a service, the controller plane must 566 ensure that incoming DetNet MPLS packets arrive with the needed 567 information (F-label(s) and/or incoming interface) and provision the 568 needed information. The provisioned information MUST then be used to 569 identify incoming DetNet service based on the combination of S-Label 570 and F-Label(s) or incoming interface. 572 The use of platform labels for S-Labels matches other pseudowire 573 encapsulations for consistency but there is no hard requirement in 574 this regard. 576 Implementation details of PREOF functions are out of scope for this 577 document. [IEEE802.1CB-2017] defines equivalent replication and 578 elimination specific aspects, which can be applied to PRF and PEF. 580 4.2.2.1. Packet Replication Function Processing 582 The Packet Replication Function (PRF) function MAY be supported by an 583 implementation for outgoing DetNet flows. The use of the PRF for a 584 particular DetNet service MUST be provisioned via configuration, 585 i.e., via the controller plane described in 586 [I-D.ietf-detnet-data-plane-framework]. When replication is 587 configured, the same app-flow data will be sent over multiple 588 outgoing DetNet member flows using forwarding sub-layer LSPs. An 589 S-Label value MUST be configured per outgoing member flow. The same 590 d-CW field value MUST be used on all outgoing member flows for each 591 replicated MPLS packet. 593 4.2.2.2. Packet Elimination Function Processing 595 Implementations MAY support the Packet Elimination Function (PEF) for 596 received DetNet MPLS flows. When supported, use of the PEF for a 597 particular DetNet service MUST be provisioned via configuration, 598 i.e., via the controller plane described in 599 [I-D.ietf-detnet-data-plane-framework]. 601 After a DetNet service is identified for a received DetNet MPLS 602 packet, as described above, if PEF is configured for that DetNet 603 service, duplicate (replicated) instances of a particular sequence 604 number MUST be discarded. The specific mechanisms used for an 605 implementation to identify which received packets are duplicates and 606 which are new is an implementation choice. Note that per 607 Section 4.2.1 the sequence number field length may be 16 or 28 bits, 608 and the field value can wrap. PEF MUST NOT be used with DetNet flows 609 configured with a d-CW sequence number field length of 0 bits. 611 An implementation MAY constrain the maximum number of sequence 612 numbers that are tracked on either a platform-wide or per flow basis. 613 Some implementations MAY support the provisioning of the maximum 614 number of sequence numbers that are tracked on either a platform-wide 615 or per flow basis. 617 4.2.2.3. Packet Ordering Function Processing 619 A function that is related to in-order delivery is the Packet 620 Ordering Function (POF). Implementations MAY support POF. When 621 supported, use of the POF for a particular DetNet service MUST be 622 provisioned via configuration, i.e., via the controller plane 623 described by [I-D.ietf-detnet-data-plane-framework]. Implementations 624 MAY require that PEF and POF be used in combination. There is no 625 requirement related to the order of execution of the Packet 626 Elimination and Ordering Functions in an implementation. 628 After a DetNet service is identified for a received DetNet MPLS 629 packet, as described above, if POF is configured for that DetNet 630 service, packets MUST be processed in the order indicated in the 631 received d-CW sequence number field, which may not be in the order 632 the packets are received. As defined in Section 4.2.1 the sequence 633 number field length may be 16 or 28 bits, is incremented by one (1) 634 for each new MPLS packet sent for a particular DetNet service, and 635 the field value can wrap. The specific mechanisms used for an 636 implementation to identify the order of received packets is an 637 implementation choice. 639 An implementation MAY constrain the maximum number of out of order 640 packets that can be processed, on either a platform-wide or per flow 641 basis. The number of out of order packets that can be processed also 642 impacts the latency of a flow. 644 The latency impact on the system resources needed to support a 645 specific DetNet flow will need to be evaluated by the controller 646 plane based on that flow's traffic specification. An example traffic 647 specification that can be used with MPLS with Traffic Engineering 648 (MPLS-TE) can be found in [RFC2212]. 650 DetNet implementations can use flow-specific requirements (e.g., 651 maximum number of out-of-order packets, maximum latency of the flow) 652 for configuration of POF-related buffers. POF implementation details 653 are out-of-scope for this document and POF configuration parameters 654 are implementation specific. The Controller Plane identifies and 655 sets the POF configuration parameters. 657 4.2.3. F-Labels 659 F-Labels support the DetNet forwarding sub-layer. F-Labels are used 660 to provide LSP-based connectivity between DetNet service sub-layer 661 processing nodes. 663 4.2.3.1. Service Sub-Layer Related Processing 665 DetNet MPLS end systems, edge nodes and relay nodes may operate at 666 the DetNet service sub-layer with understanding of DetNet services 667 and their requirements. As mentioned earlier, when operating at this 668 layer such nodes can push, pop or swap (pop then push) S-Labels. In 669 all cases, the F-Label(s) used for a DetNet service are always 670 replaced and the following procedures apply. 672 When sending a DetNet flow, zero or more F-Labels MAY be pushed on 673 top of an S-Label by the node pushing an S-Label. The F-Label(s) to 674 be pushed when sending a particular DetNet service MUST be 675 provisioned per outgoing S-Label via configuration, i.e., via the 676 controller plane discussed in [I-D.ietf-detnet-data-plane-framework]. 677 F-Label(s) can also provide context for an S-Label. To allow for the 678 omission of F-Label(s), an implementation SHOULD also allow an 679 outgoing interface to be configured per S-Label. 681 Note, when PRF is supported, the same app-flow data will be sent over 682 multiple outgoing DetNet member flows using forwarding sub-layer 683 LSPs. This means that implementation may be sending different sets 684 of F-Labels per DetNet member flow, each with a different S-Label. 686 When a single set of F-Labels is provisioned for a particular 687 outgoing S-Label, that set of F-labels MUST be pushed after the 688 S-Label is pushed. The outgoing packet is then forwarded as 689 described below in Section 4.2.3.2. When a single outgoing interface 690 is provisioned, the outgoing packet is then forwarded as described 691 below in Section 4.2.3.2. 693 When multiple sets of outgoing F-Labels or interfaces are provisioned 694 for a particular DetNet service (i.e., for PRF), a copy of the 695 outgoing packet, including the pushed member flow-specific S-Label, 696 MUST be made per F-label set and outgoing interface. Each set of 697 provisioned F-Labels are then pushed onto a copy of the packet. Each 698 copy is then forwarded as described below in Section 4.2.3.2. 700 As described in the previous section, when receiving a DetNet MPLS 701 flow, an implementation identifies the DetNet service associated with 702 the incoming packet based on the S-Label. When a node is using 703 platform labels for S-Labels, any F-Labels can be popped and the 704 S-label uniquely identifies the DetNet service. In the case where 705 platform labels are not used, incoming F-Label(s) and, optionally, 706 the incoming interface MUST also be provisioned for a DetNet service. 708 4.2.3.2. Common F-Label Processing 710 All DetNet aware MPLS nodes process F-Labels as needed to meet the 711 service requirements of the DetNet flow or flows carried in the LSPs 712 represented by the F-Labels. This includes normal push, pop and swap 713 operations. Such processing is essentially the same type of 714 processing provided for TE LSPs, although the specific service 715 parameters, or traffic specification, can differ. When the DetNet 716 service parameters of the DetNet flow or flows carried in an LSP 717 represented by an F-Label can be met by an existing TE mechanism, the 718 forwarding sub-layer processing node MAY be a DetNet unaware, i.e., 719 standard, MPLS LSR. Such TE LSPs may provide LSP forwarding service 720 as defined in, but not limited to, [RFC3209], [RFC3270], [RFC3272], 721 [RFC3473], [RFC4875], [RFC5440], and [RFC8306]. 723 More specifically, as mentioned above, the DetNet forwarding sub- 724 layer provides explicit routes and allocated resources, and F-Labels 725 are used to map to each. Explicit routes are supported based on the 726 topmost (outermost) F-Label that is pushed or swapped and the LSP 727 that corresponds to this label. This topmost (outgoing) label MUST 728 be associated with a provisioned outgoing interface and, for non- 729 point-to-point outgoing interfaces, a next hop LSR. Note that this 730 information MUST be provisioned via configuration or the controller 731 plane. In the previously mentioned special case where there are no 732 added F-labels and the outgoing interface is not a point-to-point 733 interface, the outgoing interface MUST also be associated with a next 734 hop LSR. 736 Resources may be allocated in a hierarchical fashion per LSP that is 737 represented by each F-Label. Each LSP MAY be provisioned with a 738 service parameter that dictates the specific traffic treatment to be 739 received by the traffic carried over that LSP. Implementations of 740 this document MUST ensure that traffic carried over each LSP 741 represented by one or more F-Labels receives the traffic treatment 742 provisioned for that LSP. Typical mechanisms used to provide 743 different treatment to different flows includes the allocation of 744 system resources (such as queues and buffers) and provisioning of 745 related parameters (such as shaping, and policing) that may be found 746 in implementations of Resource ReSerVation Protocol (RSVP) [RFC2205] 747 (RSVP) and RSVP-TE [RFC3209] and [RFC3473]. Support can also be 748 provided via an underlying network technology such IEEE802.1 TSN 749 [I-D.ietf-detnet-mpls-over-tsn]. The specific mechanisms selected by 750 a DetNet node to ensure DetNet service delivery requirements are met 751 for supported DetNet flows is outside the scope of this document. 753 Packets that are marked in a way that do not correspond to allocated 754 resources, e.g., lack a provisioned F-Label, can disrupt the QoS 755 offered to properly reserved DetNet flows by using resources 756 allocated to the reserved flows. Therefore, the network nodes of a 757 DetNet network: 759 o MUST defend the DetNet QoS by discarding or remarking (to an 760 allocated DetNet flow or non-competing non-DetNet flow) packets 761 received that are not associated with a completed resource 762 allocation. 764 o MUST NOT use a DetNet allocated resource, e.g. a queue or shaper 765 reserved for DetNet flows, for any packet that does match the 766 corresponding DetNet flow. 768 o MUST ensure a QoS flow does not exceed its allocated resources or 769 provisioned service level, 771 o MUST ensure a CoS flow or service class does not impact the 772 service delivered to other flows. This requirement is similar to 773 the requirement for MPLS LSRs that CoS LSPs do not impact the 774 resources allocated to TE LSPs, e.g., via [RFC3473]. 776 Subsequent sections provide additional considerations related to CoS 777 (Section 4.6.1), QoS (Section 4.6.2) and aggregation (Section 4.4). 779 4.3. OAM Indication 781 OAM follows the procedures set out in [RFC5085] with the restriction 782 that only Virtual Circuit Connectivity Verification (VCCV) type 1 is 783 supported. 785 As shown in Figure 3 of [RFC5085] when the first nibble of the d-CW 786 is 0x0 the payload following the d-CW is normal user data. However, 787 when the first nibble of the d-CW is 0x1, the payload that follows 788 the d-CW is an OAM payload with the OAM type indicated by the value 789 in the d-CW Channel Type field. 791 The reader is referred to [RFC5085] for a more detailed description 792 of the Associated Channel mechanism, and to the DetNet work on OAM 793 for more information DetNet OAM. 795 Additional considerations on DetNet-specific OAM are subjects for 796 further study. 798 4.4. Flow Aggregation 800 The ability to aggregate individual flows, and their associated 801 resource control, into a larger aggregate is an important technique 802 for improving scaling of control in the data, management and control 803 planes. The DetNet data plane allows for the aggregation of DetNet 804 flows, to improved scaling. There are two methods of supporting flow 805 aggregation covered in this section. 807 The resource control and management aspects of aggregation (including 808 the configuration of queuing, shaping, and policing) are the 809 responsibility of the DetNet controller plane and is out of scope of 810 this documents. It is also the responsibility of the controller 811 plane to ensure that consistent aggregation methods are used. 813 4.4.1. Aggregation Via LSP Hierarchy 815 DetNet flows forwarded via MPLS can leverage MPLS-TE's existing 816 support for hierarchical LSPs (H-LSPs), see [RFC4206]. H-LSPs are 817 typically used to aggregate control and resources, they may also be 818 used to provide OAM or protection for the aggregated LSPs. Arbitrary 819 levels of aggregation naturally falls out of the definition for 820 hierarchy and the MPLS label stack [RFC3032]. DetNet nodes which 821 support aggregation (LSP hierarchy) map one or more LSPs (labels) 822 into and from an H-LSP. Both carried LSPs and H-LSPs may or may not 823 use the TC field, i.e., L-LSPs or E-LSPs [RFC3270]. Such nodes will 824 need to ensure that individual LSPs and H-LSPs receive the traffic 825 treatment required to ensure the required DetNet service is 826 preserved. 828 Additional details of the traffic control capabilities needed at a 829 DetNet-aware node may be covered in the new service definitions 830 mentioned above or in separate future documents. Controller plane 831 mechanisms will also need to ensure that the service required on the 832 aggregate flow are provided, which may include the discarding or 833 remarking mentioned in the previous sections. 835 4.4.2. Aggregating DetNet Flows as a new DetNet flow 837 An aggregate can be built by layering DetNet flows, including both 838 their S-Label and, when present, F-Labels as shown below: 840 +---------------------------------+ 841 | | 842 | DetNet Flow | 843 | Payload Packet | 844 | | 845 +---------------------------------+ <--\ 846 | DetNet Control Word | | 847 +=================================+ | 848 | S-Label | | 849 +---------------------------------+ | 850 | [ F-Label(s) ] | +----DetNet data plane 851 +---------------------------------+ | 852 | DetNet Control Word | | 853 +=================================+ | 854 | A-Label | | 855 +---------------------------------+ | 856 | F-Label(s) | <--/ 857 +---------------------------------+ 858 | Data-Link | 859 +---------------------------------+ 860 | Physical | 861 +---------------------------------+ 863 Figure 6: DetNet Aggregation as a new DetNet Flow 865 Both the aggregation label, which is referred to as an A-Label, and 866 the individual flow's S-Label have their MPLS S bit set indicating 867 bottom of stack, and the d-CW allows the PREOF to work. An A-Label 868 is a special case of an S-Label, whose properties are known only at 869 the aggregation and deaggregation end-points. 871 It is a property of the A-Label that what follows is a d-CW followed 872 by an MPLS label stack. A relay node processing the A-Label would 873 not know the underlying payload type, and the A-Label would be 874 processed as a normal S-Label. This would only be known to a node 875 that was a peer of the node imposing the S-Label. However there is 876 no real need for it to know the payload type during aggregation 877 processing. 879 As in the previous section, nodes supporting this type of aggregation 880 will need to ensure that individual and aggregated flows receive the 881 traffic treatment required to ensure the required DetNet service is 882 preserved. Also, it is the controller plane's responsibility to 883 ensure that the service required on the aggregate flow are properly 884 provisioned. 886 4.5. Service Sub-Layer Considerations 888 The edge and relay node internal procedures related to PREOF are 889 implementation specific. The order of a packet elimination or 890 replication is out of scope in this specification. 892 It is important that the DetNet layer is configured such that a 893 DetNet node never receives its own replicated packets. If it were to 894 receive such packets the replication function would make the loop 895 more destructive of bandwidth than a conventional unicast loop. 896 Ultimately the TTL in the S-Label will cause the packet to die during 897 a transient loop, but given the sensitivity of applications to packet 898 latency the impact on the DetNet application would be severe. To 899 avoid the problem of a transient forwarding loop, changes to an LSP 900 supporting DetNet MUST be loop-free. 902 4.5.1. Edge Node Processing 904 A DetNet Edge node operates in the DetNet forwarding sub-layer and 905 service sub-layer. An edge node is responsible for matching incoming 906 packets to the service they require and encapsulating them 907 accordingly. An edge node may perform PRF, PEF, and or POF. Details 908 on encapsulation can be found in Section 4.2; details on PRF can be 909 found in Section 4.2.2.1; details on PEF can be found in 910 Section 4.2.2.2; and details on POF can be found in Section 4.2.2.3. 912 4.5.2. Relay Node Processing 914 A DetNet Relay node operates in the DetNet forwarding sub-layer and 915 service sub-layer. For DetNet using MPLS forwarding related 916 processing is performed on the F-Label. This processing is done 917 within an extended forwarder function. Whether an incoming DetNet 918 flow receives DetNet specific processing depends on how the 919 forwarding is programmed. Some relay nodes may be DetNet service 920 aware for certain DetNet services, while for other DetNet services 921 these nodes may perform as unmodified LSRs that only understand how 922 to switch MPLS-TE LSPs, i.e., as a transit node, see Section 4.4. 923 Again, this is entirely up to how the forwarding has been programmed. 925 During the elimination and replication process the sequence number of 926 an incoming DetNet packet MUST be preserved and carried in the 927 corresponding outgoing DetNet packet. For example, a relay node that 928 performs both PEF and PRF first performs PEF on incoming packets to 929 create a compound flow. It then performs PRF and copies the app-flow 930 data and the d-CW into packets for each outgoing DetNet member flow. 932 The internal design of a relay node is out of scope of this document. 933 However the reader's attention is drawn to the need to make any PREOF 934 state available to the packet processor(s) dealing with packets to 935 which the PREOF functions must be applied, and to maintain that state 936 is such a way that it is available to the packet processor operation 937 on the next packet in the DetNet flow (which may be a duplicate, a 938 late packet, or the next packet in sequence). 940 4.6. Forwarding Sub-Layer Considerations 942 4.6.1. Class of Service 944 Class and quality of service, i.e., CoS and QoS, are terms that are 945 often used interchangeably and confused with each other. In the 946 context of DetNet, CoS is used to refer to mechanisms that provide 947 traffic forwarding treatment based on aggregate group basis and QoS 948 is used to refer to mechanisms that provide traffic forwarding 949 treatment based on a specific DetNet flow basis. Examples of 950 existing network level CoS mechanisms include DiffServ which is 951 enabled by IP header differentiated services code point (DSCP) field 952 [RFC2474] and MPLS label traffic class field [RFC5462], and at Layer- 953 2, by IEEE 802.1p priority code point (PCP). 955 CoS for DetNet flows carried in PWs and MPLS is provided using the 956 existing MPLS Differentiated Services (DiffServ) architecture 957 [RFC3270]. Both E-LSP and L-LSP MPLS DiffServ modes MAY be used to 958 support DetNet flows. The Traffic Class field (formerly the EXP 959 field) of an MPLS label follows the definition of [RFC5462] and 960 [RFC3270]. The Uniform, Pipe, and Short Pipe DiffServ tunneling and 961 TTL processing models are described in [RFC3270] and [RFC3443] and 962 MAY be used for MPLS LSPs supporting DetNet flows. MPLS Explicit 963 Congestion Notification (ECN) MAY also be used as defined in ECN 964 [RFC5129] and updated by [RFC5462]. 966 4.6.2. Quality of Service 968 In addition to explicit routes, and packet replication and 969 elimination, described in Section 4 above, DetNet provides zero 970 congestion loss and bounded latency and jitter. As described in 971 [RFC8655], there are different mechanisms that may be used separately 972 or in combination to deliver a zero congestion loss service. This 973 includes Quality of Service (QoS) mechanisms at the MPLS layer, that 974 may be combined with the mechanisms defined by the underlying network 975 layer such as 802.1TSN. 977 Quality of Service (QoS) mechanisms for flow specific traffic 978 treatment typically includes a guarantee/agreement for the service, 979 and allocation of resources to support the service. Example QoS 980 mechanisms include discrete resource allocation, admission control, 981 flow identification and isolation, and sometimes path control, 982 traffic protection, shaping, policing and remarking. Example 983 protocols that support QoS control include Resource ReSerVation 984 Protocol (RSVP) [RFC2205] (RSVP) and RSVP-TE [RFC3209] and [RFC3473]. 985 The existing MPLS mechanisms defined to support CoS [RFC3270] can 986 also be used to reserve resources for specific traffic classes. 988 A baseline set of QoS capabilities for DetNet flows carried in PWs 989 and MPLS can be provided by MPLS with Traffic Engineering (MPLS-TE) 990 [RFC3209] and [RFC3473]. TE LSPs can also support explicit routes 991 (path pinning). Current service definitions for packet TE LSPs can 992 be found in "Specification of the Controlled Load Quality of 993 Service", [RFC2211], "Specification of Guaranteed Quality of 994 Service", [RFC2212], and "Ethernet Traffic Parameters", [RFC6003]. 995 Additional service definitions are expected in future documents to 996 support the full range of DetNet services. In all cases, the 997 existing label-based marking mechanisms defined for TE-LSPs and even 998 E-LSPs are use to support the identification of flows requiring 999 DetNet QoS. 1001 5. Management and Control Information Summary 1003 The specific information needed for the processing of each DetNet 1004 service depends on the DetNet node type and the functions being 1005 provided on the node. This section summarizes based on the DetNet 1006 sub-layer and if the DetNet traffic is being sent or received. All 1007 DetNet node types are DetNet forwarding sub-layer aware, while all 1008 but transit nodes are service sub-layer aware. This is shown in 1009 Figure 2. 1011 Much like other MPLS labels, there are a number of alternatives 1012 available for DetNet S-Label and F-Label advertisement to an upstream 1013 peer node. These include distributed signaling protocols such as 1014 RSVP-TE, centralized label distribution via a controller that manages 1015 both the sender and the receiver using NETCONF/YANG, BGP, PCEP, etc., 1016 and hybrid combinations of the two. The details of the controller 1017 plane solution required for the label distribution and the management 1018 of the label number space are out of scope of this document. There 1019 are particular DetNet considerations and requirements that are 1020 discussed in [I-D.ietf-detnet-data-plane-framework]. Conformance 1021 language is not used in the summary since it applies to future 1022 mechanisms such as those that may be provided in signaling and YANG 1023 models, e.g., [I-D.ietf-detnet-yang]. 1025 5.1. Service Sub-Layer Information Summary 1027 The following summarizes the information that is needed on service 1028 sub-layer aware nodes that transmit DetNet MPLS traffic, on a per 1029 service basis: 1031 o App-Flow identification information, e.g., IP information as 1032 defined in [I-D.ietf-detnet-ip-over-mpls]. Note, this information 1033 is not needed on DetNet relay nodes. 1035 o The sequence number length to be used for the service. Valid 1036 values include 0, 16 and 28 bits. 0 bits cannot be used when PEF 1037 or POF is configured for the service. 1039 o If PRF is to be provided for the service. 1041 o The outgoing S-Label for each of the service's outgoing DetNet 1042 (member) flows. 1044 o The forwarding sub-layer information associated with the output of 1045 the service sub-layer. Note that when the PRF function is 1046 provisioned, this information is per DetNet member flow. 1047 Logically the forwarding sub-layer information is a pointer to 1048 further details of transmission of Detnet flows at the forwarding 1049 sub-layer. 1051 The following summarizes the information that is needed on service 1052 sub-layer aware nodes that receive DetNet MPLS traffic, on a per 1053 service basis: 1055 o The forwarding sub-layer information associated with the input of 1056 the service sub-layer. Note that when the PEF function is 1057 provisioned, this information is per DetNet member flow. 1058 Logically the forwarding sub-layer information is a pointer to 1059 further details of the reception of Detnet flows at the forwarding 1060 sub-layer or A-Label. 1062 o The incoming S-Label for the service. 1064 o If PEF or POF is to be provided for the service. 1066 o The sequence number length to be used for the service. Valid 1067 values included 0, 16 and 28 bits. 0 bits cannot be used when PEF 1068 or POF are configured for the service. 1070 o App-Flow identification information, e.g., IP information as 1071 defined in [I-D.ietf-detnet-ip-over-mpls]. Note, this information 1072 is not needed on DetNet relay nodes. 1074 5.1.1. Service Aggregation Information Summary 1076 Nodes performing aggregation using A-Labels, per 1077 Section Section 4.4.2, require the additional information summarized 1078 in this section. 1080 The following summarizes the additional information that is needed on 1081 a node that sends aggregated flows using A-Labels: 1083 o The S-Labels or F-Labels that are to be carried over each 1084 aggregated service. 1086 o The A-Label associated with each aggregated service. 1088 o The other S-Label information summarized above. 1090 On the receiving node, the A-Label provides the forwarding context of 1091 an incoming interface or an F-Label and is used in subsequent service 1092 or forwarding sub-layer receive processing, as appropriated. The 1093 related additional configuration that may be provided is discussed 1094 elsewhere in this section. 1096 5.2. Forwarding Sub-Layer Information Summary 1098 The following summarizes the information that is needed on forwarding 1099 sub-layer aware nodes that send DetNet MPLS traffic, on a per 1100 forwarding sub-layer flow basis: 1102 o The outgoing F-Label stack to be pushed. The stack may include 1103 H-LSP labels. 1105 o The traffic parameters associated with a specific label in the 1106 stack. Note that there may be multiple sets of traffic parameters 1107 associated with specific labels in the stack, e.g., when H-LSPs 1108 are used. 1110 o Outgoing interface and, for unicast traffic, the next hop 1111 information. 1113 o Sub-network specific parameters on a technology specific basis. 1114 For example, see [I-D.ietf-detnet-mpls-over-tsn]. 1116 The following summarizes the information that is needed on forwarding 1117 sub-layer aware nodes that receive DetNet MPLS traffic, on a per 1118 forwarding sub-layer flow basis: 1120 o The incoming interface. For some implementations and deployment 1121 scenarios, this information may not be needed. 1123 o The incoming F-Label stack to be popped. The stack may include 1124 H-LSP labels. 1126 o How the incoming forwarding sub-layer flow is to be handled, i.e., 1127 forwarded as a transit node, or provided to the service sub-layer. 1129 It is the responsibility of the DetNet controller plane to properly 1130 provision both flow identification information and the flow-specific 1131 resources needed to provided the traffic treatment needed to meet 1132 each flow's service requirements. This applies for aggregated and 1133 individual flows. 1135 6. Security Considerations 1137 Detailed security considerations for DetNet are cataloged in 1138 [I-D.ietf-detnet-security], and more general security considerations 1139 are described in [RFC8655]. This section considers exclusively 1140 security considerations which are specific to the DetNet MPLS data 1141 plane. The considerations raised related to MPLS networks in general 1142 in [RFC5920] are equally applicable to the the DetNet MPLS data 1143 plane. 1145 Security aspects which are unique to DetNet are those whose aim is to 1146 protect the support of specific quality of service aspects of DetNet, 1147 which are primarily to deliver data flows with extremely low packet 1148 loss rates and bounded end-to-end delivery latency. Achieving such 1149 loss rates and bounded latency may not be possible in the face of a 1150 highly capable adversary, such as the one envisioned by the Internet 1151 Threat Model of BCP 72 that can arbitrarily drop or delay any or all 1152 traffic. In order to present meaningful security considerations, we 1153 consider a somewhat weaker attacker who does not control the physical 1154 links of the DetNet domain, but may have the ability to control a 1155 network node within the boundary of the DetNet domain. 1157 An additional consideration for the DetNet data plane is to maintain 1158 integrity of data and delivery of the associated DetNet service 1159 traversing the DetNet network. Application flows can be protected 1160 through whatever means are provided by the underlying technology. 1161 For example, encryption may be used, such as that provided by IPsec 1162 [RFC4301] for IP flows and/or by an underlying sub-net using MACSec 1163 [IEEE802.1AE-2018] for IP over Ethernet (Layer-2) flows. MPLS 1164 doesn't provide any native security services to account for 1165 confidentiality and integrity. 1167 From a data plane perspective this document does not add or modify 1168 any application header information. 1170 At the management and control level DetNet flows are identified on a 1171 per-flow basis, which may provide controller plane attackers with 1172 additional information about the data flows (when compared to 1173 controller planes that do not include per-flow identification). This 1174 is an inherent property of DetNet which has security implications 1175 that should be considered when determining if DetNet is a suitable 1176 technology for any given use case. 1178 To provide uninterrupted availability of the DetNet service, 1179 provisions can be made against DOS attacks and delay attacks. To 1180 protect against DOS attacks, excess traffic due to malicious or 1181 malfunctioning devices is prevented or mitigated through the use of 1182 existing mechanisms, for example by policing and shaping incoming 1183 traffic. To prevent DetNet packets having their delay manipulated by 1184 an external entity, precautions need to be taken to ensure that all 1185 devices on an LSP are those intended to be there by the network 1186 operator and that they are well behaved. In addition to physical 1187 security, technical methods such as authentication and authorization 1188 of network equipment and the instrumentation and monitoring of the 1189 LSP packet delay may be used. If a delay attack is suspected, 1190 traffic may be moved to an alternate path, for example through 1191 changing the LSP or management of the PREOF configuration. 1193 7. IANA Considerations 1195 This document makes no IANA requests. 1197 8. Acknowledgements 1199 The authors wish to thank Pat Thaler, Norman Finn, Loa Anderson, 1200 David Black, Rodney Cummings, Ethan Grossman, Tal Mizrahi, David 1201 Mozes, Craig Gunther, George Swallow, Yuanlong Jiang, Jeong-dong Ryoo 1202 and Carlos J. Bernardos for their various contributions to this 1203 work. 1205 9. Contributors 1207 RFC7322 limits the number of authors listed on the front page of a 1208 draft to a maximum of 5. The editor wishes to thank and acknowledge 1209 the following author for contributing text to this draft. 1211 Don Fedyk 1212 LabN Consulting, L.L.C. 1213 Email: dfedyk@labn.net 1215 10. References 1217 10.1. Normative References 1219 [I-D.ietf-detnet-data-plane-framework] 1220 Varga, B., Farkas, J., Berger, L., Malis, A., and S. 1221 Bryant, "DetNet Data Plane Framework", draft-ietf-detnet- 1222 data-plane-framework-06 (work in progress), May 2020. 1224 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1225 Requirement Levels", BCP 14, RFC 2119, 1226 DOI 10.17487/RFC2119, March 1997, 1227 . 1229 [RFC2211] Wroclawski, J., "Specification of the Controlled-Load 1230 Network Element Service", RFC 2211, DOI 10.17487/RFC2211, 1231 September 1997, . 1233 [RFC2212] Shenker, S., Partridge, C., and R. Guerin, "Specification 1234 of Guaranteed Quality of Service", RFC 2212, 1235 DOI 10.17487/RFC2212, September 1997, 1236 . 1238 [RFC3031] Rosen, E., Viswanathan, A., and R. Callon, "Multiprotocol 1239 Label Switching Architecture", RFC 3031, 1240 DOI 10.17487/RFC3031, January 2001, 1241 . 1243 [RFC3032] Rosen, E., Tappan, D., Fedorkow, G., Rekhter, Y., 1244 Farinacci, D., Li, T., and A. Conta, "MPLS Label Stack 1245 Encoding", RFC 3032, DOI 10.17487/RFC3032, January 2001, 1246 . 1248 [RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V., 1249 and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP 1250 Tunnels", RFC 3209, DOI 10.17487/RFC3209, December 2001, 1251 . 1253 [RFC3270] Le Faucheur, F., Wu, L., Davie, B., Davari, S., Vaananen, 1254 P., Krishnan, R., Cheval, P., and J. Heinanen, "Multi- 1255 Protocol Label Switching (MPLS) Support of Differentiated 1256 Services", RFC 3270, DOI 10.17487/RFC3270, May 2002, 1257 . 1259 [RFC3443] Agarwal, P. and B. Akyol, "Time To Live (TTL) Processing 1260 in Multi-Protocol Label Switching (MPLS) Networks", 1261 RFC 3443, DOI 10.17487/RFC3443, January 2003, 1262 . 1264 [RFC3473] Berger, L., Ed., "Generalized Multi-Protocol Label 1265 Switching (GMPLS) Signaling Resource ReserVation Protocol- 1266 Traffic Engineering (RSVP-TE) Extensions", RFC 3473, 1267 DOI 10.17487/RFC3473, January 2003, 1268 . 1270 [RFC4206] Kompella, K. and Y. Rekhter, "Label Switched Paths (LSP) 1271 Hierarchy with Generalized Multi-Protocol Label Switching 1272 (GMPLS) Traffic Engineering (TE)", RFC 4206, 1273 DOI 10.17487/RFC4206, October 2005, 1274 . 1276 [RFC4385] Bryant, S., Swallow, G., Martini, L., and D. McPherson, 1277 "Pseudowire Emulation Edge-to-Edge (PWE3) Control Word for 1278 Use over an MPLS PSN", RFC 4385, DOI 10.17487/RFC4385, 1279 February 2006, . 1281 [RFC5085] Nadeau, T., Ed. and C. Pignataro, Ed., "Pseudowire Virtual 1282 Circuit Connectivity Verification (VCCV): A Control 1283 Channel for Pseudowires", RFC 5085, DOI 10.17487/RFC5085, 1284 December 2007, . 1286 [RFC5129] Davie, B., Briscoe, B., and J. Tay, "Explicit Congestion 1287 Marking in MPLS", RFC 5129, DOI 10.17487/RFC5129, January 1288 2008, . 1290 [RFC5462] Andersson, L. and R. Asati, "Multiprotocol Label Switching 1291 (MPLS) Label Stack Entry: "EXP" Field Renamed to "Traffic 1292 Class" Field", RFC 5462, DOI 10.17487/RFC5462, February 1293 2009, . 1295 [RFC5586] Bocci, M., Ed., Vigoureux, M., Ed., and S. Bryant, Ed., 1296 "MPLS Generic Associated Channel", RFC 5586, 1297 DOI 10.17487/RFC5586, June 2009, 1298 . 1300 [RFC6790] Kompella, K., Drake, J., Amante, S., Henderickx, W., and 1301 L. Yong, "The Use of Entropy Labels in MPLS Forwarding", 1302 RFC 6790, DOI 10.17487/RFC6790, November 2012, 1303 . 1305 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 1306 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 1307 May 2017, . 1309 [RFC8655] Finn, N., Thubert, P., Varga, B., and J. Farkas, 1310 "Deterministic Networking Architecture", RFC 8655, 1311 DOI 10.17487/RFC8655, October 2019, 1312 . 1314 10.2. Informative References 1316 [I-D.ietf-detnet-ip] 1317 Varga, B., Farkas, J., Berger, L., Fedyk, D., and S. 1318 Bryant, "DetNet Data Plane: IP", draft-ietf-detnet-ip-07 1319 (work in progress), July 2020. 1321 [I-D.ietf-detnet-ip-over-mpls] 1322 Varga, B., Berger, L., Fedyk, D., Bryant, S., and J. 1323 Korhonen, "DetNet Data Plane: IP over MPLS", draft-ietf- 1324 detnet-ip-over-mpls-08 (work in progress), September 2020. 1326 [I-D.ietf-detnet-mpls-over-tsn] 1327 Varga, B., Farkas, J., Malis, A., and S. Bryant, "DetNet 1328 Data Plane: MPLS over IEEE 802.1 Time Sensitive Networking 1329 (TSN)", draft-ietf-detnet-mpls-over-tsn-03 (work in 1330 progress), June 2020. 1332 [I-D.ietf-detnet-security] 1333 Grossman, E., Mizrahi, T., and A. Hacker, "Deterministic 1334 Networking (DetNet) Security Considerations", draft-ietf- 1335 detnet-security-12 (work in progress), October 2020. 1337 [I-D.ietf-detnet-yang] 1338 Geng, X., Chen, M., Ryoo, Y., Fedyk, D., Li, Z., and R. 1339 Rahman, "Deterministic Networking (DetNet) Configuration 1340 YANG Model", draft-ietf-detnet-yang-07 (work in progress), 1341 July 2020. 1343 [IEEE802.1AE-2018] 1344 IEEE Standards Association, "IEEE Std 802.1AE-2018 MAC 1345 Security (MACsec)", 2018, 1346 . 1348 [IEEE802.1CB-2017] 1349 IEEE Standards Association, "IEEE Std 802.1CB-2017 Frame 1350 Replication and Elimination for Reliability", 2017, 1351 . 1353 [RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S. 1354 Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1 1355 Functional Specification", RFC 2205, DOI 10.17487/RFC2205, 1356 September 1997, . 1358 [RFC2474] Nichols, K., Blake, S., Baker, F., and D. Black, 1359 "Definition of the Differentiated Services Field (DS 1360 Field) in the IPv4 and IPv6 Headers", RFC 2474, 1361 DOI 10.17487/RFC2474, December 1998, 1362 . 1364 [RFC3272] Awduche, D., Chiu, A., Elwalid, A., Widjaja, I., and X. 1365 Xiao, "Overview and Principles of Internet Traffic 1366 Engineering", RFC 3272, DOI 10.17487/RFC3272, May 2002, 1367 . 1369 [RFC3985] Bryant, S., Ed. and P. Pate, Ed., "Pseudo Wire Emulation 1370 Edge-to-Edge (PWE3) Architecture", RFC 3985, 1371 DOI 10.17487/RFC3985, March 2005, 1372 . 1374 [RFC4301] Kent, S. and K. Seo, "Security Architecture for the 1375 Internet Protocol", RFC 4301, DOI 10.17487/RFC4301, 1376 December 2005, . 1378 [RFC4448] Martini, L., Ed., Rosen, E., El-Aawar, N., and G. Heron, 1379 "Encapsulation Methods for Transport of Ethernet over MPLS 1380 Networks", RFC 4448, DOI 10.17487/RFC4448, April 2006, 1381 . 1383 [RFC4875] Aggarwal, R., Ed., Papadimitriou, D., Ed., and S. 1384 Yasukawa, Ed., "Extensions to Resource Reservation 1385 Protocol - Traffic Engineering (RSVP-TE) for Point-to- 1386 Multipoint TE Label Switched Paths (LSPs)", RFC 4875, 1387 DOI 10.17487/RFC4875, May 2007, 1388 . 1390 [RFC5440] Vasseur, JP., Ed. and JL. Le Roux, Ed., "Path Computation 1391 Element (PCE) Communication Protocol (PCEP)", RFC 5440, 1392 DOI 10.17487/RFC5440, March 2009, 1393 . 1395 [RFC5920] Fang, L., Ed., "Security Framework for MPLS and GMPLS 1396 Networks", RFC 5920, DOI 10.17487/RFC5920, July 2010, 1397 . 1399 [RFC5921] Bocci, M., Ed., Bryant, S., Ed., Frost, D., Ed., Levrau, 1400 L., and L. Berger, "A Framework for MPLS in Transport 1401 Networks", RFC 5921, DOI 10.17487/RFC5921, July 2010, 1402 . 1404 [RFC6003] Papadimitriou, D., "Ethernet Traffic Parameters", 1405 RFC 6003, DOI 10.17487/RFC6003, October 2010, 1406 . 1408 [RFC6073] Martini, L., Metz, C., Nadeau, T., Bocci, M., and M. 1409 Aissaoui, "Segmented Pseudowire", RFC 6073, 1410 DOI 10.17487/RFC6073, January 2011, 1411 . 1413 [RFC8306] Zhao, Q., Dhody, D., Ed., Palleti, R., and D. King, 1414 "Extensions to the Path Computation Element Communication 1415 Protocol (PCEP) for Point-to-Multipoint Traffic 1416 Engineering Label Switched Paths", RFC 8306, 1417 DOI 10.17487/RFC8306, November 2017, 1418 . 1420 [RFC8660] Bashandy, A., Ed., Filsfils, C., Ed., Previdi, S., 1421 Decraene, B., Litkowski, S., and R. Shakir, "Segment 1422 Routing with the MPLS Data Plane", RFC 8660, 1423 DOI 10.17487/RFC8660, December 2019, 1424 . 1426 Authors' Addresses 1428 Balazs Varga (editor) 1429 Ericsson 1430 Magyar Tudosok krt. 11. 1431 Budapest 1117 1432 Hungary 1434 Email: balazs.a.varga@ericsson.com 1436 Janos Farkas 1437 Ericsson 1438 Magyar Tudosok krt. 11. 1439 Budapest 1117 1440 Hungary 1442 Email: janos.farkas@ericsson.com 1444 Lou Berger 1445 LabN Consulting, L.L.C. 1447 Email: lberger@labn.net 1448 Andrew G. Malis 1449 Malis Consulting 1451 Email: agmalis@gmail.com 1453 Stewart Bryant 1454 Futurewei Technologies 1456 Email: stewart.bryant@gmail.com 1458 Jouni Korhonen 1460 Email: jouni.nospam@gmail.com