idnits 2.17.1 draft-ietf-detnet-mpls-07.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (June 8, 2020) is 1417 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'Network' is mentioned on line 269, but not defined == Outdated reference: A later version (-07) exists of draft-ietf-detnet-ip-06 == Outdated reference: A later version (-09) exists of draft-ietf-detnet-ip-over-mpls-06 == Outdated reference: A later version (-07) exists of draft-ietf-detnet-mpls-over-tsn-02 == Outdated reference: A later version (-16) exists of draft-ietf-detnet-security-10 -- Obsolete informational reference (is this intentional?): RFC 3272 (Obsoleted by RFC 9522) Summary: 0 errors (**), 0 flaws (~~), 6 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 DetNet B. Varga, Ed. 3 Internet-Draft J. Farkas 4 Intended status: Standards Track Ericsson 5 Expires: December 10, 2020 L. Berger 6 LabN Consulting, L.L.C. 7 A. Malis 8 Malis Consulting 9 S. Bryant 10 Futurewei Technologies 11 J. Korhonen 12 June 8, 2020 14 DetNet Data Plane: MPLS 15 draft-ietf-detnet-mpls-07 17 Abstract 19 This document specifies the Deterministic Networking data plane when 20 operating over an MPLS Packet Switched Networks. 22 Status of This Memo 24 This Internet-Draft is submitted in full conformance with the 25 provisions of BCP 78 and BCP 79. 27 Internet-Drafts are working documents of the Internet Engineering 28 Task Force (IETF). Note that other groups may also distribute 29 working documents as Internet-Drafts. The list of current Internet- 30 Drafts is at https://datatracker.ietf.org/drafts/current/. 32 Internet-Drafts are draft documents valid for a maximum of six months 33 and may be updated, replaced, or obsoleted by other documents at any 34 time. It is inappropriate to use Internet-Drafts as reference 35 material or to cite them other than as "work in progress." 37 This Internet-Draft will expire on December 10, 2020. 39 Copyright Notice 41 Copyright (c) 2020 IETF Trust and the persons identified as the 42 document authors. All rights reserved. 44 This document is subject to BCP 78 and the IETF Trust's Legal 45 Provisions Relating to IETF Documents 46 (https://trustee.ietf.org/license-info) in effect on the date of 47 publication of this document. Please review these documents 48 carefully, as they describe your rights and restrictions with respect 49 to this document. Code Components extracted from this document must 50 include Simplified BSD License text as described in Section 4.e of 51 the Trust Legal Provisions and are provided without warranty as 52 described in the Simplified BSD License. 54 Table of Contents 56 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 57 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 58 2.1. Terms Used in This Document . . . . . . . . . . . . . . . 3 59 2.2. Abbreviations . . . . . . . . . . . . . . . . . . . . . . 4 60 2.3. Requirements Language . . . . . . . . . . . . . . . . . . 5 61 3. DetNet MPLS Data Plane Overview . . . . . . . . . . . . . . . 5 62 3.1. Layers of DetNet Data Plane . . . . . . . . . . . . . . . 5 63 3.2. DetNet MPLS Data Plane Scenarios . . . . . . . . . . . . 6 64 4. MPLS-Based DetNet Data Plane Solution . . . . . . . . . . . . 8 65 4.1. DetNet Over MPLS Encapsulation Components . . . . . . . . 8 66 4.2. MPLS Data Plane Encapsulation . . . . . . . . . . . . . . 9 67 4.2.1. DetNet Control Word and the DetNet Sequence Number . 10 68 4.2.2. S-Labels . . . . . . . . . . . . . . . . . . . . . . 11 69 4.2.3. F-Labels . . . . . . . . . . . . . . . . . . . . . . 14 70 4.3. OAM Indication . . . . . . . . . . . . . . . . . . . . . 16 71 4.4. Flow Aggregation . . . . . . . . . . . . . . . . . . . . 17 72 4.4.1. Aggregation Via LSP Hierarchy . . . . . . . . . . . . 17 73 4.4.2. Aggregating DetNet Flows as a new DetNet flow . . . . 18 74 4.5. Service Sub-Layer Considerations . . . . . . . . . . . . 19 75 4.5.1. Edge Node Processing . . . . . . . . . . . . . . . . 19 76 4.5.2. Relay Node Processing . . . . . . . . . . . . . . . . 19 77 4.6. Forwarding Sub-Layer Considerations . . . . . . . . . . . 20 78 4.6.1. Class of Service . . . . . . . . . . . . . . . . . . 20 79 4.6.2. Quality of Service . . . . . . . . . . . . . . . . . 20 80 5. Management and Control Information Summary . . . . . . . . . 21 81 5.1. Service Sub-Layer Information Summary . . . . . . . . . . 22 82 5.1.1. Service Aggregation Information Summary . . . . . . . 23 83 5.2. Forwarding Sub-Layer Information Summary . . . . . . . . 23 84 6. Security Considerations . . . . . . . . . . . . . . . . . . . 24 85 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 25 86 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 25 87 9. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 25 88 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 25 89 10.1. Normative References . . . . . . . . . . . . . . . . . . 25 90 10.2. Informative References . . . . . . . . . . . . . . . . . 27 91 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 29 93 1. Introduction 95 Deterministic Networking (DetNet) is a service that can be offered by 96 a network to DetNet flows. DetNet provides these flows extremely low 97 packet loss rates and assured maximum end-to-end delivery latency. 98 General background and concepts of DetNet can be found in [RFC8655]. 100 The DetNet Architecture models the DetNet related data plane 101 functions decomposed into two sub-layers: a service sub-layer and a 102 forwarding sub-layer. The service sub-layer is used to provide 103 DetNet service functions such as protection and reordering. The 104 forwarding sub-layer is used to provide forwarding assurance (low 105 loss, assured latency, and limited reordering). 107 This document specifies the DetNet data plane operation and the on- 108 wire encapsulation of DetNet flows over an MPLS-based Packet Switched 109 Network (PSN) using the service sub-layer reference model. MPLS 110 encapsulation already provides a solid foundation of building blocks 111 to enable the DetNet service and forwarding sub-layer functions. 112 MPLS encapsulated DetNet can be carried over a variety of different 113 network technologies that can provide the DetNet required level of 114 service. However, the specific details of how DetNet MPLS is carried 115 over different network technologies is out of scope of this document. 117 MPLS encapsulated DetNet flows can carry different types of traffic. 118 The details of the types of traffic that are carried in DetNet are 119 also out of scope of this document. An example of IP using DetNet 120 MPLS sub-networks can be found in [I-D.ietf-detnet-ip]. DetNet MPLS 121 may use an associated controller and Operations, Administration, and 122 Maintenance (OAM) functions that are defined outside of this 123 document. 125 Background information common to all data planes for DetNet can be 126 found in the DetNet Data Plane Framework 127 [I-D.ietf-detnet-data-plane-framework]. 129 2. Terminology 131 2.1. Terms Used in This Document 133 This document uses the terminology established in the DetNet 134 architecture [RFC8655] and the the DetNet Data Plane Framework 135 [I-D.ietf-detnet-data-plane-framework]. The reader is assumed to be 136 familiar with these documents, any terminology defined therein and 137 basic MPLS related terminologies in [RFC3031]. 139 The following terminology is introduced in this document: 141 F-Label A Detnet "forwarding" label that identifies the LSP 142 used to forward a DetNet flow across an MPLS PSN, e.g., 143 a hop-by-hop label used between label switching routers 144 (LSR). 146 S-Label A DetNet "service" label that is used between DetNet 147 nodes that implement also the DetNet service sub-layer 148 functions. An S-Label is also used to identify a 149 DetNet flow at DetNet service sub-layer. 151 A-Label A special case of an S-Label, whose aggregation 152 properties are known only at the aggregation and 153 deaggregation end-points. 155 d-CW A DetNet Control Word (d-CW) is used for sequencing 156 information of a DetNet flow at the DetNet service sub- 157 layer. 159 2.2. Abbreviations 161 The following abbreviations are used in this document: 163 CoS Class of Service. 165 CW Control Word. 167 DetNet Deterministic Networking. 169 LSR Label Switching Router. 171 MPLS Multiprotocol Label Switching. 173 MPLS-TE Multiprotocol Label Switching - Traffic Engineering. 175 MPLS-TP Multiprotocol Label Switching - Transport Profile. 177 OAM Operations, Administration, and Maintenance. 179 PE Provider Edge. 181 PEF Packet Elimination Function. 183 PRF Packet Replication Function. 185 PREOF Packet Replication, Elimination and Ordering Functions. 187 POF Packet Ordering Function. 189 PSN Packet Switched Network. 191 PW PseudoWire. 193 QoS Quality of Service. 195 S-PE Switching Provider Edge. 197 T-PE Terminating Provider Edge. 199 TSN Time-Sensitive Network. 201 2.3. Requirements Language 203 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 204 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 205 "OPTIONAL" in this document are to be interpreted as described in BCP 206 14 [RFC2119] [RFC8174] when, and only when, they appear in all 207 capitals, as shown here. 209 3. DetNet MPLS Data Plane Overview 211 3.1. Layers of DetNet Data Plane 213 MPLS provides a wide range of capabilities that can be utilised by 214 DetNet. A straight forward approach utilizing MPLS for a DetNet 215 service sub-layer is based on the existing pseudowire (PW) 216 encapsulations and by utilizing existing MPLS Traffic Engineering 217 encapsulations and mechanisms. Background on PWs can be found in 218 [RFC3985] and [RFC3031]. Background on MPLS Traffic Engineering can 219 be found in [RFC3272] and [RFC3209]. 221 DetNet MPLS 222 . 223 Bottom of Stack . 224 (inner label) +------------+ 225 | Service | d-CW, S-Label (A-Label) 226 +------------+ 227 | Forwarding | F-Label(s) 228 +------------+ 229 Top of Stack . 230 (outer label) . 232 Figure 1: DetNet Adaptation to MPLS Data Plane 234 The DetNet MPLS data plane representation is illustrated in Figure 1. 235 The service sub-layer includes a DetNet control word (d-CW) and a 236 identifying service label (S-Label). The DetNet control word (d-CW) 237 conforms to the Generic PW MPLS Control Word (PWMCW) defined in 238 [RFC4385]. An aggregation label (A-Label) is a special case of 239 S-Label used for aggregation. 241 A node operating on a DetNet flow in the Detnet service sub-layer, 242 uses the local context associated with that S-Label, provided by a 243 received F-Label, to determine what local DetNet operation(s) are 244 applied to that packet. An S-Label may be taken from the platform 245 label space [RFC3031], making it unique, enabling DetNet flow 246 identification regardless of which input interface or LSP the packet 247 arrives on. 249 The DetNet forwarding sub-layer is supported by zero or more 250 forwarding labels (F-Labels). MPLS Traffic Engineering 251 encapsulations and mechanisms can be utilized to provide a forwarding 252 sub-layer that is responsible for providing resource allocation and 253 explicit routes. 255 3.2. DetNet MPLS Data Plane Scenarios 257 DetNet MPLS Relay Transit Relay DetNet MPLS 258 End System Node Node Node End System 259 (T-PE) (S-PE) (LSR) (S-PE) (T-PE) 260 +----------+ +----------+ 261 | Appl. |<------------ End to End Service ----------->| Appl. | 262 +----------+ +---------+ +---------+ +----------+ 263 | Service |<--| Service |-- DetNet flow --| Service |-->| Service | 264 +----------+ +---------+ +----------+ +---------+ +----------+ 265 |Forwarding| |Fwd| |Fwd| |Forwarding| |Fwd| |Fwd| |Forwarding| 266 +-------.--+ +-.-+ +-.-+ +----.---.-+ +-.-+ +-.-+ +---.------+ 267 : Link : / ,-----. \ : Link : / ,-----. \ 268 +........+ +-[ Sub ]-+ +......+ +-[ Sub ]-+ 269 [Network] [Network] 270 `-----' `-----' 271 |<- LSP -->| |<-------- LSP -----------| |<--- LSP -->| 273 |<----------------- DetNet MPLS --------------------->| 275 Figure 2: A DetNet MPLS Network 277 Figure 2 illustrates a hypothetical DetNet MPLS-only network composed 278 of DetNet aware MPLS enabled end systems, operating over a DetNet 279 aware MPLS network. In this figure, the relay nodes are PE devices 280 that define the MPLS LSP boundaries and the transit nodes are LSRs. 282 DetNet end system and relay nodes understand the particular needs of 283 DetNet flows and provide both DetNet service and forwarding sub-layer 284 functions. In the case of MPLS, DetNet service-aware nodes add, 285 remove and process d-CWs, S-Labels and F-labels as needed. DetNet 286 MPLS nodes provide functionality analogous to T-PEs when they sit at 287 the edge of an MPLS domain, and S-PEs when they are in the middle of 288 an MPLS domain, see [RFC6073]. 290 In a DetNet MPLS network, transit nodes may be DetNet service aware 291 or may be DetNet unaware MPLS Label Switching Routers (LSRs). In 292 this latter case, such LSRs would be unaware of the special 293 requirements of the DetNet service sub-layer, but would still provide 294 traffic engineering functions and the QoS capabilities needed to 295 ensure that the (TE) LSPs meet the service requirements of the 296 carried DetNet flows. 298 The application of DetNet using MPLS supports a number of control 299 plane/management plane types. These types support certain MPLS data 300 plane deployments. For example RSVP-TE, MPLS-TP, or MPLS Segment 301 Routing (when extended to support resource allocation) are all valid 302 MPLS deployment cases. 304 Figure 3 illustrates how an end-to-end MPLS-based DetNet service is 305 provided in a more detail. In this figure, the customer end systems, 306 CE1 and CE2, are able to send and receive MPLS encapsulated DetNet 307 flows, and R1, R2 and R3 are relay nodes in the middle of a DetNet 308 network. The 'X' in the end systems, and relay nodes represents 309 potential DetNet compound flow packet replication and elimination 310 points. In this example, service protection is supported utilizing 311 at least two DetNet member flows and TE LSPs. For a unidirectional 312 flow, R1 supports PRF and R3 supports PEF and POF. Note that the 313 relay nodes may change the underlying forwarding sub-layer, for 314 example tunneling MPLS over IEEE 802.1 TSN 315 [I-D.ietf-detnet-mpls-over-tsn], or simply over interconnect network 316 links. 318 DetNet DetNet 319 MPLS Service Transit Transit Service MPLS 320 DetNet | |<-Tnl->| |<-Tnl->| | DetNet 321 End | V 1 V V 2 V | End 322 System | +--------+ +--------+ +--------+ | System 323 +---+ | | R1 |=======| R2 |=======| R3 | | +---+ 324 | X...DFa...|._X_....|..DF1..|.__ ___.|..DF3..|...._X_.|.DFa..|.X | 325 |CE1|========| \ | | X | | / |======|CE2| 326 | | | | \_.|..DF2..|._/ \__.|..DF4..|._/ | | | | 327 +---+ | |=======| |=======| | +---+ 328 ^ +--------+ +--------+ +--------+ ^ 329 | Relay Node Relay Node Relay Node | 330 | (S-PE) (S-PE) (S-PE) | 331 | | 332 |<---------------------- DetNet MPLS --------------------->| 333 | | 334 |<--------------- End to End DetNet Service -------------->| 336 -------------------------- Data Flow -------------------------> 338 X = Optional service protection (none, PRF, PREOF, PEF/POF) 339 DFx = DetNet member flow x over a TE LSP 341 Figure 3: MPLS-Based Native DetNet 343 4. MPLS-Based DetNet Data Plane Solution 345 4.1. DetNet Over MPLS Encapsulation Components 347 To carry DetNet over MPLS the following is required: 349 1. A method of identifying the MPLS payload type. 351 2. A method of identifying the DetNet flow group to the processing 352 element. 354 3. A method of distinguishing DetNet OAM packets from DetNet data 355 packets. 357 4. A method of carrying the DetNet sequence number. 359 5. A suitable LSP to deliver the packet to the egress PE. 361 6. A method of carrying queuing and forwarding indication. 363 In this design an MPLS service label (the S-Label), similar to a 364 pseudowire (PW) label [RFC3985], is used to identify both the DetNet 365 flow identity and the payload MPLS payload type satisfying (1) and 366 (2) in the list above. OAM traffic discrimination happens through 367 the use of the Associated Channel method described in [RFC4385]. The 368 DetNet sequence number is carried in the DetNet Control word which 369 carries the Data/OAM discriminator. To simplify implementation and 370 to maximize interoperability two sequence number sizes are supported: 371 a 16 bit sequence number and a 28 bit sequence number. The 16 bit 372 sequence number is needed to support some types of legacy clients. 373 The 28 bit sequence number is used in situations where it is 374 necessary ensure that in high speed networks the sequence number 375 space does not wrap whilst packets are in flight. 377 The LSP used to forward the DetNet packet is not restricted regarding 378 any method used for establishing that LSP (for example, MPLS-LDP, 379 MPLS-TE, MPLS-TP [RFC5921], MPLS-SR [RFC8660], etc.). The LSP 380 (F-Label) label and/or the S-Label may be used to indicate the queue 381 processing as well as the forwarding parameters. Note that the 382 possible use of Penultimate Hop Popping (PHP) means that the S-Label 383 may be the only label received at the terminating DetNet service. 385 4.2. MPLS Data Plane Encapsulation 387 Figure 4 illustrates a DetNet data plane MPLS encapsulation. The 388 MPLS-based encapsulation of the DetNet flows is well suited for the 389 scenarios described in [I-D.ietf-detnet-data-plane-framework]. 390 Furthermore, an end to end DetNet service i.e., native DetNet 391 deployment (see Section 3.2) is also possible if DetNet end systems 392 are capable of initiating and termination MPLS encapsulated packets. 394 The MPLS-based DetNet data plane encapsulation consists of: 396 o DetNet control word (d-CW) containing sequencing information for 397 packet replication and duplicate elimination purposes, and the OAM 398 indicator. 400 o DetNet service Label (S-Label) that identifies a DetNet flow at 401 the receiving DetNet service sub-layer processing node. 403 o Zero or more Detnet MPLS Forwarding label(s) (F-Label) used to 404 direct the packet along the label switched path (LSP) to the next 405 service sub-layer processing node along the path. When 406 Penultimate Hop Popping is in use there may be no label F-Label in 407 the protocol stack on the final hop. 409 o The necessary data-link encapsulation is then applied prior to 410 transmission over the physical media. 412 DetNet MPLS-based encapsulation 414 +---------------------------------+ 415 | | 416 | DetNet App-Flow | 417 | Payload Packet | 418 | | 419 +---------------------------------+ <--\ 420 | DetNet Control Word | | 421 +---------------------------------+ +--> DetNet data plane 422 | S-Label | | MPLS encapsulation 423 +---------------------------------+ | 424 | [ F-Label(s) ] | | 425 +---------------------------------+ <--/ 426 | Data-Link | 427 +---------------------------------+ 428 | Physical | 429 +---------------------------------+ 431 Figure 4: Encapsulation of a DetNet App-Flow in an MPLS PSN 433 4.2.1. DetNet Control Word and the DetNet Sequence Number 435 A DetNet control word (d-CW) conforms to the Generic PW MPLS Control 436 Word (PWMCW) defined in [RFC4385]. The d-CW formatted as shown in 437 Figure 5 MUST be present in all DetNet packets containing app-flow 438 data. This format of the d-CW was created in order (1) to allow 439 larger S/N space to avoid S/N rollover frequency in some applications 440 and (2) to allow non-skip zero S/N what simplifies implementation. 442 0 1 2 3 443 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 444 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 445 |0 0 0 0| Sequence Number | 446 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 448 Figure 5: DetNet Control Word 450 (bits 0 to 3) 452 Per [RFC4385], MUST be set to zero (0). 454 Sequence Number (bits 4 to 31) 456 An unsigned value implementing the DetNet sequence number. The 457 sequence number space is a circular one. 459 A separate sequence number space MUST be maintained by the node that 460 adds the d-CW for each DetNet app-flow. The following sequence 461 number field lengths MUST be supported: 463 0 bits 465 16 bits 467 28 bits 469 The sequence number length MUST be provisioned on a per app-flow 470 basis via configuration, i.e., via the controller plane described in 471 [I-D.ietf-detnet-data-plane-framework]. 473 A 0 bit sequence number field length indicates that there is no 474 DetNet sequence number used for the flow. When the length is zero, 475 the sequence number field MUST be set to zero (0) on all packets sent 476 for the flow. 478 When the sequence number field length is 16 or 28 bits for a flow, 479 the sequence number MUST be incremented by one for each new app-flow 480 packet sent. When the field length is 16 bits, d-CW bits 4 to 15 481 MUST be set to zero (0). The values carried in this field can wrap 482 and it is important to note that zero (0) is a valid field value. 483 For example, were the sequence number size is 16 bits, the sequence 484 will contain: 65535, 0, 1, where zero (0) is an ordinary sequence 485 number. 487 It is important to note that this document differs from [RFC4448] 488 where a sequence number of zero (0) is used to indicate that the 489 sequence number check algorithm is not used. 491 The sequence number is optionally used during receive processing as 492 described below in Section 4.2.2.1 and Section 4.2.2.2. 494 4.2.2. S-Labels 496 App-flow identification at a DetNet service sub-layer is realized by 497 an S-Label. MPLS-aware DetNet end systems and edge nodes, which are 498 by definition MPLS ingress and egress nodes, MUST add and remove an 499 app-flow specific d-CW and S-Label. Relay nodes MAY swap S-Label 500 values when processing an app-flow. 502 The S-Label value MUST be provisioned per app-flow via configuration, 503 e.g., via the controller plane described in 504 [I-D.ietf-detnet-data-plane-framework]. Note that S-Labels provide 505 app-flow identification at the downstream DetNet service sub-layer 506 receiver, not the sender. As such, S-Labels MUST be allocated by the 507 entity that controls the service sub-layer receiving node's label 508 space, and MAY be allocated from the platform label space [RFC3031]. 509 Because S-Labels are local to each node rather than being a global 510 identifier within a domain, they must be advertised to their upstream 511 DetNet service-aware peer nodes (e.g., a DetNet MPLS End System or a 512 DetNet Relay or Edge Node and interpreted in the context of their 513 received F-Label. 515 The S-Label will normally be at the bottom of the label stack once 516 the last F-Label is removed, immediately preceding the d-CW. To 517 support service sub-layer level OAM, an OAM Associated Channel Header 518 (ACH) [RFC4385] together with a Generic Associated Channel Label 519 (GAL) [RFC5586] MAY be used in place of a d-CW. 521 Similarly, an Entropy Label Indicator/Entropy Label (ELI/EL) 522 [RFC6790] MAY be carried below the S-Label in the label stack in 523 networks where DetNet flows would otherwise received ECMP treatment. 524 When ELs are used, the same EL value SHOULD be used for all of the 525 packets sent using a specific S-Label to force the flow to follow the 526 same path. However, as outlines in 527 [I-D.ietf-detnet-data-plane-framework] the use of ECMP for DetNet 528 flows is NOT RECOMMENDED. ECMP MAY be used for non-DetNet flows 529 within a DetNet domain. 531 When receiving a DetNet MPLS flow, an implementation MUST identify 532 the app-flow associated with the incoming packet based on the 533 S-Label. When a node is using platform labels for S-Labels, no 534 additional information is needed as the S-label uniquely identifies 535 the app-flow. In the case where platform labels are not used, zero 536 or more F-Labels and optionally, the incoming interface, proceeding 537 the S-Label MUST be used together with the S-Label to uniquely 538 identify the app-flows associated with a received packet. The 539 incoming interface MAY also be used to together with any present 540 F-Label(s) and the S-Label to uniquely identify an incoming app- 541 flows, for example, to in the case where PHP is used. Note that 542 choice to use platform label space for S-Label or S-Label plus one or 543 more F-Labels to identify app flows is a local implementation choice, 544 with one caveat. When one or more F-labels, or incoming interface, 545 is needed together with an S-Label to uniquely identify, the 546 controller plane MUST ensure that incoming DetNet MPLS packets arrive 547 with the needed information (F-label(s) and/or incoming interface); 548 the details of such are outside the scope of this document. 550 The use of platform labels for S-Labels matches other pseudowire 551 encapsulations for consistency but there is no hard requirement in 552 this regard. 554 4.2.2.1. Packet Elimination Function Processing 556 Implementations MAY support the Packet Elimination Function (PEF) for 557 received DetNet MPLS flows. When supported, use of the PEF for a 558 particular app-flow MUST be provisioned via configuration, e.g., via 559 the controller plane described in 560 [I-D.ietf-detnet-data-plane-framework]. 562 After an app-flow is identified for a received DetNet MPLS packet, as 563 described above, an implementation MUST check if PEF is configured 564 for that app-flow. When configured, the implementation MUST track 565 the sequence number contained in received d-CWs and MUST ensure that 566 duplicate (replicated) instances of a particular sequence number are 567 discarded. The specific mechanisms used for an implementation to 568 identify which received packets are duplicates and which are new is 569 an implementation choice. Note that per Section 4.2.1 the sequence 570 number field length may be 16 or 28 bits, and the field value can 571 wrap. 573 Note that an implementation MAY wish to constrain the maximum number 574 sequence numbers that are tracked, on platform-wide or per flow 575 basis. Some implementations MAY support the provisioning of the 576 maximum number sequence numbers that are tracked number on either a 577 platform-wide or per flow basis. 579 4.2.2.2. Packet Ordering Function Processing 581 A function that is related to in-order delivery is the Packet 582 Ordering Function (POF). Implementations MAY support POF. When 583 supported, use of the POF for a particular app-flow MUST be 584 provisioned via configuration, e.g., via the controller plane 585 described by [I-D.ietf-detnet-data-plane-framework]. Implementations 586 MAY required that PEF and POF be used in combination. There is no 587 requirement related to the order of execution of the Packet 588 Elimination and Ordering Functions in an implementation. 590 After an app-flow is identified for a received DetNet MPLS packet, as 591 described above, an implementation MUST check if POF is configured 592 for that app-flow. When configured, the implementation MUST track 593 the sequence number contained in received d-CWs and MUST ensure that 594 packets are processed in the order indicated in the received d-CW 595 sequence number field, which may not be in the order the packets are 596 received. As defined in Section 4.2.1 the sequence number field 597 length may be 16 or 28 bits, is incremented by one (1) for each new 598 app-flow packet sent, and the field value can wrap. The specific 599 mechanisms used for an implementation to identify the order of 600 received packets is an implementation choice. 602 Note that an implementation MAY wish to constrain the maximum number 603 of out of order packets that can be processed, on platform-wide or 604 per flow basis. Some implementations MAY support the provisioning of 605 this number on either a platform-wide or per flow basis. The number 606 of out of order packets that can be processed also impacts the 607 latency of a flow. 609 4.2.3. F-Labels 611 F-Labels are supported the DetNet forwarding sub-layer. F-Labels are 612 used to provide LSP-based connectivity between DetNet service sub- 613 layer processing nodes. 615 4.2.3.1. Service Sub-Layer and Packet Replication Function Processing 617 DetNet MPLS end systems, edge nodes and relay nodes may operate at 618 the DetNet service sub-layer with understand of app-flows and their 619 requirements. As mentioned earlier, when operating at this layer 620 such nodes can push, pop or swap (pop then push) S-Labels. In all 621 cases, the F-Labels used for the app-flow are always replaced and the 622 following procedures apply. 624 When sending a DetNet flow, zero or more F-Labels MAY be pushed on 625 top of an S-Label by the node pushing an S-Label. The F-Labels to be 626 pushed when sending a particular app-flow MUST be provisioned per 627 app-flow via configuration, e.g., via the controller plane discussed 628 in [I-D.ietf-detnet-data-plane-framework]. F-Labels can also provide 629 context for an S-Label. To allow for the omission of F-Labels, an 630 implementation SHOULD also allow an outgoing interface to be used. 632 The Packet Replication Function (PRF) function MAY be supported by an 633 implementation for outgoing DetNet flows. When replication is 634 supported, the same app-flow data will be sent over multiple outgoing 635 forwarding sub-layer LSPs. To support PRF an implementation MUST 636 support the setting of different sets of F-Labels. To allow for the 637 omission of F-Labels, an implementation SHOULD also allow multiple 638 outgoing interfaces to be provisioned. PRF MUST NOT be used with 639 app-flows configured with a d-CW sequence number field length of 0 640 bits. 642 When a single set of F-Labels is provisioned for a particular 643 outgoing app-flow, that set of F-labels MUST be pushed after the 644 S-Label is pushed. The outgoing packet is then forwarded as 645 described below in Section 4.2.3.2. When a single outgoing interface 646 is provisioned, the outgoing packet is then forwarded as described 647 below in Section 4.2.3.2. 649 When multiple sets of F-Labels or interfaces are provisioned for a 650 particular outgoing app-flow, a copy of the outgoing packet, 651 including the pushed S-Label, MUST be made per F-label set and 652 outgoing interface. Each set of provisioned F-Labels are then pushed 653 onto a copy of the packet. Each copy is then forwarded as described 654 below in Section 4.2.3.2. 656 As described in the previous section, when receiving a DetNet MPLS 657 flow, an implementation identifies the app-flow associated with the 658 incoming packet based on the S-Label. When a node is using platform 659 labels for S-Labels, any F-Labels can be popped and the S-label 660 uniquely identifies the app-flow. In the case where platform labels 661 are not used, F-Label(s) and, optionally, the incoming interface MUST 662 also be provisioned for incoming app-flows. The provisioned 663 information MUST then be used to identify incoming app-flows based on 664 the combination of S-Label and F-Label(s) or incoming interface. 666 4.2.3.2. Common F-Label Processing 668 All DetNet aware MPLS nodes process F-Labels as needed to meet the 669 service requirements of the DetNet flow or flows carried in the LSPs 670 represented by the F-Labels. This includes normal push, pop and swap 671 operations. Such processing is essentially the same type of 672 processing provided for TE LSPs, although the specific service 673 parameters, or traffic specification, can differ. When the DetNet 674 service parameters of the app-flow or flows carried in an LSP 675 represented by an F-Label can be met by an exiting TE mechanism, the 676 forwarding sub-layer processing node MAY be a DetNet unaware, i.e., 677 standard, MPLS LSR. Such TE LSPs may provide LSP forwarding service 678 as defined in, but not limited to, [RFC3209], [RFC3270], [RFC3272], 679 [RFC3473], [RFC4875], [RFC5440], and [RFC8306]. 681 More specifically, as mentioned above, the DetNet forwarding sub- 682 layer provides explicit routes and allocated resources, and F-Labels 683 are used to map to each. Explicit routes are supported based on the 684 topmost (outermost) F-Label that is pushed or swapped and the LSP 685 that corresponds to this label. This topmost (outgoing) label MUST 686 be associated with a provisioned outgoing interface and, for non- 687 point-to-point outgoing interfaces, a next hop LSR. Note that this 688 information MUST be provisioned via configuration or the controller 689 plane. In the previously mentioned special case where there are no 690 added F-labels and the outgoing interface is not a point-to-point 691 interface, the outgoing interface MUST also be associated with a next 692 hop LSR. 694 Resources may be allocated in a hierarchical fashion per LSP that is 695 represented by each F-Label. Each LSP MAY be provisioned with a 696 service parameters that dictates the specific traffic treatment to be 697 received by the traffic carried over that LSP. Implementations of 698 this document MUST ensure that traffic carried over each LSP 699 represented by one or more F-Labels receives the traffic treatment 700 provisioned for that LSP. Typical mechanisms used to provide 701 different treatment to different flows includes the allocation of 702 system resources (such as queues and buffers) and provisioning or 703 related parameters (such as shaping, and policing). Support can also 704 be provided via an underlying network technology such IEEE802.1 TSN 705 [I-D.ietf-detnet-mpls-over-tsn]. The specific mechanisms used by a 706 DetNet node to ensure DetNet service delivery requirements are met 707 for supported DetNet flows is outside the scope of this document. 709 Packets that are marked in a way that do not correspond to allocated 710 resources, e.g., lack a provisioned F-Label, can disrupt the QoS 711 offered to properly reserved DetNet flows by using resources 712 allocated to the reserved flows. Therefore, the network nodes of a 713 DetNet network: 715 o MUST defend the DetNet QoS by discarding or remarking (to an 716 allocated DetNet flow or non-competing non-DetNet flow) packets 717 received that are not associated with a completed resource 718 allocation. 720 o MUST NOT use a DetNet allocated resource, e.g. a queue or shaper 721 reserved for DetNet flows, for any packet that does match the 722 corresponding DetNet flow. 724 o MUST ensure a QoS flow does not exceed its allocated resources or 725 provisioned service level, 727 o MUST ensure a CoS flow or service class does not impact the 728 service delivered to other flows. This requirement is similar to 729 requirement for MPLS LSRs to that CoS LSPs do not impact the 730 resources allocated to TE LSPs, e.g., via [RFC3473]. 732 Subsequent sections provide additional considerations related to CoS 733 (Section 4.6.1), QoS (Section 4.6.2) and aggregation (Section 4.4). 735 4.3. OAM Indication 737 OAM follows the procedures set out in [RFC5085] with the restriction 738 that only Virtual Circuit Connectivity Verification (VCCV) type 1 is 739 supported. 741 As shown in Figure 3 of [RFC5085] when the first nibble of the d-CW 742 is 0x0 the payload following the d-CW is normal user data. However, 743 when the first nibble of the d-CW is 0X1, the payload that follows 744 the d-DW is an OAM payload with the OAM type indicated by the value 745 in the d-CW Channel Type field. 747 The reader is referred to [RFC5085] for a more detailed description 748 of the Associated Channel mechanism, and to the DetNet work on OAM 749 for more information DetNet OAM. 751 Additional considerations on DetNet-specific OAM are subjects for 752 further study. 754 4.4. Flow Aggregation 756 The ability to aggregate individual flows, and their associated 757 resource control, into a larger aggregate is an important technique 758 for improving scaling of control in the data, management and control 759 planes. The DetNet data plane allows for the aggregation of DetNet 760 flows, to improved scaling. There are two methods of supporting flow 761 aggregation covered in this section. 763 The resource control and management aspects of aggregation (including 764 the configuration of queuing, shaping, and policing) are the 765 responsibility of the DetNet controller plane and is out of scope of 766 this documents. It is also the responsibility of the controller 767 plane to ensure that consistent aggregation methods are used. 769 4.4.1. Aggregation Via LSP Hierarchy 771 DetNet flows forwarded via MPLS can leverage MPLS-TE's existing 772 support for hierarchical LSPs (H-LSPs), see [RFC4206]. H-LSPs are 773 typically used to aggregate control and resources, they may also be 774 used to provide OAM or protection for the aggregated LSPs. Arbitrary 775 levels of aggregation naturally falls out of the definition for 776 hierarchy and the MPLS label stack [RFC3032]. DetNet nodes which 777 support aggregation (LSP hierarchy) map one or more LSPs (labels) 778 into and from an H-LSP. Both carried LSPs and H-LSPs may or may not 779 use the TC field, i.e., L-LSPs or E-LSPs. Such nodes will need to 780 ensure that individual LSPs and H-LSPs receive the traffic treatment 781 required to ensure the required DetNet service is preserved. 783 Additional details of the traffic control capabilities needed at a 784 DetNet-aware node may be covered in the new service definitions 785 mentioned above or in separate future documents. Controller plane 786 mechanisms will also need to ensure that the service required on the 787 aggregate flow are provided, which may include the discarding or 788 remarking mentioned in the previous sections. 790 4.4.2. Aggregating DetNet Flows as a new DetNet flow 792 An aggregate can be built by layering DetNet flows, including both 793 their S-Label and, when present, F-Labels as shown below: 795 +---------------------------------+ 796 | | 797 | DetNet Flow | 798 | Payload Packet | 799 | | 800 +---------------------------------+ <--\ 801 | DetNet Control Word | | 802 +=================================+ | 803 | S-Label | | 804 +---------------------------------+ | 805 | [ F-Label(s) ] | +----DetNet data plane 806 +---------------------------------+ | 807 | DetNet Control Word | | 808 +=================================+ | 809 | A-Label | | 810 +---------------------------------+ | 811 | F-Label(s) | <--/ 812 +---------------------------------+ 813 | Data-Link | 814 +---------------------------------+ 815 | Physical | 816 +---------------------------------+ 818 Figure 6: DetNet Aggregation as a new DetNet Flow 820 Both the aggregation label, which is referred to as an A-Label, and 821 the individual flow's S-Label have their MPLS S bit set indicating 822 bottom of stack, and the d-CW allows the PREOF to work. An A-Label 823 is a special case of an S-Label, whose properties are known only at 824 the aggregation and deaggregation end-points. 826 It is a property of the A-Label that what follows is a d-CW followed 827 by an MPLS label stack. A relay node processing the A-Label would 828 not know the underlying payload type, and the A-Label would be 829 process as a normal S-Label. This would only be known to a node that 830 was a peer of the node imposing the S-Label. However there is no 831 real need for it to know the payload type during aggregation 832 processing. 834 As in the previous section, nodes supporting this type of aggregation 835 will need to ensure that individual and aggregated flows receive the 836 traffic treatment required to ensure the required DetNet service is 837 preserved. Also, it is the controller plane's responsibility to to 838 ensure that the service required on the aggregate flow are properly 839 provisioned. 841 4.5. Service Sub-Layer Considerations 843 The edge and relay node internal procedures related to PREOF are 844 implementation specific. The order of a packet elimination or 845 replication is out of scope in this specification. 847 It is important that the DetNet layer is configured such that a 848 DetNet node never receives its own replicated packets. If it were to 849 receive such packets the replication function would make the loop 850 more destructive of bandwidth than a conventional unicast loop. 851 Ultimately the TTL in the S-Label will cause the packet to die during 852 a transient loop, but given the sensitivity of applications to packet 853 latency the impact on the DetNet application would be severe. To 854 avoid the problem of a transient forwarding loop, changes to an LSP 855 supporting DetNet MUST be loop-free. 857 4.5.1. Edge Node Processing 859 An edge node is responsible for matching ingress packets to the 860 service they require and encapsulating them accordingly. An edge 861 node may participate in the packet replication and duplicate packet 862 elimination. 864 The DetNet-aware forwarder selects the egress DetNet member flow 865 segment based on the flow identification. The mapping of ingress 866 DetNet member flow segment to egress DetNet member flow segment may 867 be statically or dynamically configured. Additionally the DetNet- 868 aware forwarder does duplicate frame elimination based on the flow 869 identification and the sequence number combination. The packet 870 replication is also done within the DetNet-aware forwarder. 872 4.5.2. Relay Node Processing 874 A DetNet Relay node operates in the DetNet forwarding sub-layer and 875 service sub-layer. For DetNet using MPLS forwarding related 876 processing is performed on the F-Label. This processing is done 877 within an extended forwarder function. Whether an ingress DetNet 878 member flow receives DetNet specific processing depends on how the 879 forwarding is programmed. Some relay nodes may be DetNet service 880 aware for certain DetNet services, while for other DetNet services 881 these nodes may perform as unmodified LSRs that only understand how 882 to switch MPLS-TE LSPs, i.e., as a transit nodes, see Section 4.4. 883 Again, this is entirely up to how the forwarding has been programmed. 885 During the elimination and replication process the sequence number of 886 the ingress DetNet member flow MUST be preserved and copied to the 887 corresponding egress DetNet member flow. Specifically, a relay node 888 sends the same sequence number in an outgoing packet of a DetNet 889 member flow that is received in the corresponding incoming packet of 890 a DetNet compound flow. This is true whether or not PREOF is 891 performed at the relay node. 893 The internal design of a relay node is out of scope of this document. 894 However the reader's attention is drawn to the need to make any PREOF 895 state available to the packet processor(s) dealing with packets to 896 which the PREOF functions must be applied, and to maintain that state 897 is such away that it is available to the packet processor operation 898 on the next packet in the DetNet flow (which may be a duplicate, a 899 late packet, or the next packet in sequence. 901 4.6. Forwarding Sub-Layer Considerations 903 4.6.1. Class of Service 905 Class and quality of service, i.e., CoS and QoS, are terms that are 906 often used interchangeably and confused with each other. In the 907 context of DetNet, CoS is used to refer to mechanisms that provide 908 traffic forwarding treatment based on aggregate group basis and QoS 909 is used to refer to mechanisms that provide traffic forwarding 910 treatment based on a specific DetNet flow basis. Examples of 911 existing network level CoS mechanisms include DiffServ which is 912 enabled by IP header differentiated services code point (DSCP) field 913 [RFC2474] and MPLS label traffic class field [RFC5462], and at Layer- 914 2, by IEEE 802.1p priority code point (PCP). 916 CoS for DetNet flows carried in PWs and MPLS is provided using the 917 existing MPLS Differentiated Services (DiffServ) architecture 918 [RFC3270]. Both E-LSP and L-LSP MPLS DiffServ modes MAY be used to 919 support DetNet flows. The Traffic Class field (formerly the EXP 920 field) of an MPLS label follows the definition of [RFC5462] and 921 [RFC3270]. The Uniform, Pipe, and Short Pipe DiffServ tunneling and 922 TTL processing models are described in [RFC3270] and [RFC3443] and 923 MAY be used for MPLS LSPs supporting DetNet flows. MPLS ECN MAY also 924 be used as defined in ECN [RFC5129] and updated by [RFC5462]. 926 4.6.2. Quality of Service 928 In addition to explicit routes, and packet replication and 929 elimination, described in Section 4 above, DetNet provides zero 930 congestion loss and bounded latency and jitter. As described in 931 [RFC8655], there are different mechanisms that maybe used separately 932 or in combination to deliver a zero congestion loss service. This 933 includes Quality of Service (QoS) mechanisms at the MPLS layer, that 934 may be combined with the mechanisms defined by the underlying network 935 layer such as 802.1TSN. 937 Quality of Service (QoS) mechanisms for flow specific traffic 938 treatment typically includes a guarantee/agreement for the service, 939 and allocation of resources to support the service. Example QoS 940 mechanisms include discrete resource allocation, admission control, 941 flow identification and isolation, and sometimes path control, 942 traffic protection, shaping, policing and remarking. Example 943 protocols that support QoS control include Resource ReSerVation 944 Protocol (RSVP) [RFC2205] (RSVP) and RSVP-TE [RFC3209] and [RFC3473]. 945 The existing MPLS mechanisms defined to support CoS [RFC3270] can 946 also be used to reserve resources for specific traffic classes. 948 A baseline set of QoS capabilities for DetNet flows carried in PWs 949 and MPLS can provided by MPLS with Traffic Engineering (MPLS-TE) 950 [RFC3209] and [RFC3473]. TE LSPs can also support explicit routes 951 (path pinning). Current service definitions for packet TE LSPs can 952 be found in "Specification of the Controlled Load Quality of 953 Service", [RFC2211], "Specification of Guaranteed Quality of 954 Service", [RFC2212], and "Ethernet Traffic Parameters", [RFC6003]. 955 Additional service definitions are expected in future documents to 956 support the full range of DetNet services. In all cases, the 957 existing label-based marking mechanisms defined for TE-LSPs and even 958 E-LSPs are use to support the identification of flows requiring 959 DetNet QoS. 961 5. Management and Control Information Summary 963 The specific information needed for the processing of each DetNet 964 service depends on the DetNet node type and the functions being 965 provided on the node. This section summarizes based on the DetNet 966 sub-layer and if the DetNet traffic is being sent or received. All 967 DetNet node types are DetNet forwarding sub-layer aware, while all 968 but transit nodes are service sub-layer aware. This is shown in 969 Figure 2. 971 Much like other MPLS labels, there are a number of alternatives 972 available for DetNet S-Label and F-Label advertisement to an upstream 973 peer node. These include distributed signaling protocols such as 974 RSVP-TE, centralized label distribution via a controller that manages 975 both the sender and the receiver using NETCONF/YANG, BGP, PCEP, etc., 976 and hybrid combinations of the two. The details of the controller 977 plane solution required for the label distribution and the management 978 of the label number space are out of scope of this document. There 979 are particular DetNet considerations and requirements that are 980 discussed in [I-D.ietf-detnet-data-plane-framework]. 982 5.1. Service Sub-Layer Information Summary 984 The following summarizes the information that is needed on service 985 sub-layer aware nodes that send DetNet MPLS traffic, on a per service 986 basis: 988 o App-Flow identification information, e.g., an incoming service on 989 a relay node or IP information as defined in 990 [I-D.ietf-detnet-ip-over-mpls]. 992 o The sequence number length to be used for the service. Valid 993 values included 0, 16 and 28 bits. 0 bits cannot be used when PRF 994 is configured for the service. 996 o The S-Label for the service. 998 o If PRF is to be provided for the service. 1000 o The forwarding sub-layer information associated with the output of 1001 the service sub-layer. Note that when the PRF function is 1002 provisioned, this information is per DetNet member flow. 1003 Logically the forwarding sub-layer information is a pointer to 1004 further details of transmission of Detnet flows at the forwarding 1005 sub-layer. 1007 The following summarizes the information that is needed on service 1008 sub-layer aware nodes that receives DetNet MPLS traffic, on a per 1009 service basis: 1011 o The forwarding sub-layer information associated with the input of 1012 the service sub-layer. Note that when the PEF function is 1013 provisioned, this information is per DetNet member flow. 1014 Logically the forwarding sub-layer information is a pointer to 1015 further details of the reception of Detnet flows at the forwarding 1016 sub-layer or A-Label. 1018 o The S-Label for the received service. 1020 o If PEF or POF is to be provided for the service. 1022 o The sequence number length to be used for the service. Valid 1023 values included 0, 16 and 28 bits. 0 bits cannot be used when PEF 1024 or POF are configured for the service. 1026 5.1.1. Service Aggregation Information Summary 1028 Nodes performing aggregation using A-Labels, per 1029 Section Section 4.4.2, require the additional information summarized 1030 in this section. 1032 The following summarizes the information that is needed on a node 1033 that sends aggregated flows using A-Labels: 1035 o The S-Labels or F-Labels that are to be carried over each 1036 aggregated service. 1038 o The A-Label associated with each aggregated service. 1040 o The other S-Label information summarized above. 1042 On the receiving node, the A-Label provides the forwarding context of 1043 an incoming interface or an F-Label and is used in subsequent service 1044 or forwarding sub-layer receive processing, as appropriated. The 1045 related addition configuration that may be provided discussed 1046 elsewhere in this section. 1048 5.2. Forwarding Sub-Layer Information Summary 1050 The following summarizes the information that is needed on forwarding 1051 sub-layer aware nodes that send DetNet MPLS traffic, on a per 1052 forwarding sub-layer flow basis: 1054 o The outgoing F-Label stack to be pushed. The stack may include 1055 H-LSP labels. 1057 o The traffic parameters associated with a specific label in the 1058 stack. Note that there may be multiple sets of traffic paramters 1059 associated with specific labels in the stack, e.g., when H-LSPs 1060 are used. 1062 o Outgoing interface and, for unicast traffic, the next hop 1063 information. 1065 o Sub-network specific parameters on a technology specific basis. 1066 For example, see [I-D.ietf-detnet-mpls-over-tsn]. 1068 The following summarizes the information that is needed on forwarding 1069 sub-layer aware nodes that receive DetNet MPLS traffic, on a per 1070 forwarding sub-layer flow basis: 1072 o The incoming interface. For some implementations and deployment 1073 scenarios, this information may not be needed. 1075 o The incoming F-Label stack to be popped. The stack may include 1076 H-LSP labels. 1078 o How the incoming forwarding sub-layer flow is to be handled, i.e., 1079 forwarded as a transit node, or provided to the service sub-layer. 1081 It is the responsibility of the DetNet controller plane to properly 1082 provision both flow identification information and the flow specific 1083 resources needed to provided the traffic treatment needed to meet 1084 each flow's service requirements. This applies for aggregated and 1085 individual flows. 1087 6. Security Considerations 1089 General security considerations are described in [RFC8655]. 1090 Additionally, security considerations and a threat analysis are 1091 described in [I-D.ietf-detnet-security]. This section considers 1092 security considerations which are specific to the DetNet MPLS data 1093 plane. The considerations raised related to MPLS networks in general 1094 in [RFC5920] are equally applicable to the the DetNet MPLS data 1095 plane. 1097 Security aspects which are unique to DetNet are those whose aim is to 1098 provide the specific quality of service aspects of DetNet, which are 1099 primarily to deliver data flows with extremely low packet loss rates 1100 and bounded end-to-end delivery latency. 1102 The primary considerations for the data plane is to maintain 1103 integrity of data and delivery of the associated DetNet service 1104 traversing the DetNet network. Application flows can be protected 1105 through whatever means is provided by the underlying technology. For 1106 example, encryption may be used, such as that provided by IPSec 1107 [RFC4301] for IP flows and/or by an underlying sub-net using MACSec 1108 [IEEE802.1AE-2018] for IP over Ethernet (Layer-2) flows. 1110 From a data plane perspective this document does not add or modify 1111 any header information. 1113 At the management and control level DetNet flows are identified on a 1114 per-flow basis, which may provide controller plane attackers with 1115 additional information about the data flows (when compared to 1116 controller planes that do not include per-flow identification). This 1117 is an inherent property of DetNet which has security implications 1118 that should be considered when determining if DetNet is a suitable 1119 technology for any given use case. 1121 To provide uninterrupted availability of the DetNet service, 1122 provisions can be made against DOS attacks and delay attacks. To 1123 protect against DOS attacks, excess traffic due to malicious or 1124 malfunctioning devices is prevented or mitigated through the use of 1125 existing mechanisms, for example by policing and shaping incoming 1126 traffic. To prevent DetNet packets from being delayed by an entity 1127 external to a DetNet domain, DetNet technology definition can allow 1128 for the mitigation of Man-In-The-Middle attacks, for example through 1129 use of authentication and authorization of devices within the DetNet 1130 domain. 1132 7. IANA Considerations 1134 This document makes no IANA requests. 1136 8. Acknowledgements 1138 The authors wish to thank Pat Thaler, Norman Finn, Loa Anderson, 1139 David Black, Rodney Cummings, Ethan Grossman, Tal Mizrahi, David 1140 Mozes, Craig Gunther, George Swallow, Yuanlong Jiang and Carlos J. 1141 Bernardos for their various contributions to this work. 1143 9. Contributors 1145 RFC7322 limits the number of authors listed on the front page of a 1146 draft to a maximum of 5. The editor wishes to thank and acknowledge 1147 the follow author for contributing text to this draft. 1149 Don Fedyk 1150 LabN Consulting, L.L.C. 1151 Email: dfedyk@labn.net 1153 10. References 1155 10.1. Normative References 1157 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1158 Requirement Levels", BCP 14, RFC 2119, 1159 DOI 10.17487/RFC2119, March 1997, 1160 . 1162 [RFC2211] Wroclawski, J., "Specification of the Controlled-Load 1163 Network Element Service", RFC 2211, DOI 10.17487/RFC2211, 1164 September 1997, . 1166 [RFC2212] Shenker, S., Partridge, C., and R. Guerin, "Specification 1167 of Guaranteed Quality of Service", RFC 2212, 1168 DOI 10.17487/RFC2212, September 1997, 1169 . 1171 [RFC3031] Rosen, E., Viswanathan, A., and R. Callon, "Multiprotocol 1172 Label Switching Architecture", RFC 3031, 1173 DOI 10.17487/RFC3031, January 2001, 1174 . 1176 [RFC3032] Rosen, E., Tappan, D., Fedorkow, G., Rekhter, Y., 1177 Farinacci, D., Li, T., and A. Conta, "MPLS Label Stack 1178 Encoding", RFC 3032, DOI 10.17487/RFC3032, January 2001, 1179 . 1181 [RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V., 1182 and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP 1183 Tunnels", RFC 3209, DOI 10.17487/RFC3209, December 2001, 1184 . 1186 [RFC3270] Le Faucheur, F., Wu, L., Davie, B., Davari, S., Vaananen, 1187 P., Krishnan, R., Cheval, P., and J. Heinanen, "Multi- 1188 Protocol Label Switching (MPLS) Support of Differentiated 1189 Services", RFC 3270, DOI 10.17487/RFC3270, May 2002, 1190 . 1192 [RFC3443] Agarwal, P. and B. Akyol, "Time To Live (TTL) Processing 1193 in Multi-Protocol Label Switching (MPLS) Networks", 1194 RFC 3443, DOI 10.17487/RFC3443, January 2003, 1195 . 1197 [RFC3473] Berger, L., Ed., "Generalized Multi-Protocol Label 1198 Switching (GMPLS) Signaling Resource ReserVation Protocol- 1199 Traffic Engineering (RSVP-TE) Extensions", RFC 3473, 1200 DOI 10.17487/RFC3473, January 2003, 1201 . 1203 [RFC4206] Kompella, K. and Y. Rekhter, "Label Switched Paths (LSP) 1204 Hierarchy with Generalized Multi-Protocol Label Switching 1205 (GMPLS) Traffic Engineering (TE)", RFC 4206, 1206 DOI 10.17487/RFC4206, October 2005, 1207 . 1209 [RFC4385] Bryant, S., Swallow, G., Martini, L., and D. McPherson, 1210 "Pseudowire Emulation Edge-to-Edge (PWE3) Control Word for 1211 Use over an MPLS PSN", RFC 4385, DOI 10.17487/RFC4385, 1212 February 2006, . 1214 [RFC5085] Nadeau, T., Ed. and C. Pignataro, Ed., "Pseudowire Virtual 1215 Circuit Connectivity Verification (VCCV): A Control 1216 Channel for Pseudowires", RFC 5085, DOI 10.17487/RFC5085, 1217 December 2007, . 1219 [RFC5129] Davie, B., Briscoe, B., and J. Tay, "Explicit Congestion 1220 Marking in MPLS", RFC 5129, DOI 10.17487/RFC5129, January 1221 2008, . 1223 [RFC5462] Andersson, L. and R. Asati, "Multiprotocol Label Switching 1224 (MPLS) Label Stack Entry: "EXP" Field Renamed to "Traffic 1225 Class" Field", RFC 5462, DOI 10.17487/RFC5462, February 1226 2009, . 1228 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 1229 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 1230 May 2017, . 1232 [RFC8655] Finn, N., Thubert, P., Varga, B., and J. Farkas, 1233 "Deterministic Networking Architecture", RFC 8655, 1234 DOI 10.17487/RFC8655, October 2019, 1235 . 1237 10.2. Informative References 1239 [I-D.ietf-detnet-data-plane-framework] 1240 Varga, B., Farkas, J., Berger, L., Malis, A., and S. 1241 Bryant, "DetNet Data Plane Framework", draft-ietf-detnet- 1242 data-plane-framework-06 (work in progress), May 2020. 1244 [I-D.ietf-detnet-ip] 1245 Varga, B., Farkas, J., Berger, L., Fedyk, D., and S. 1246 Bryant, "DetNet Data Plane: IP", draft-ietf-detnet-ip-06 1247 (work in progress), April 2020. 1249 [I-D.ietf-detnet-ip-over-mpls] 1250 Varga, B., Berger, L., Fedyk, D., Bryant, S., and J. 1251 Korhonen, "DetNet Data Plane: IP over MPLS", draft-ietf- 1252 detnet-ip-over-mpls-06 (work in progress), May 2020. 1254 [I-D.ietf-detnet-mpls-over-tsn] 1255 Varga, B., Farkas, J., Malis, A., and S. Bryant, "DetNet 1256 Data Plane: MPLS over IEEE 802.1 Time Sensitive Networking 1257 (TSN)", draft-ietf-detnet-mpls-over-tsn-02 (work in 1258 progress), March 2020. 1260 [I-D.ietf-detnet-security] 1261 Mizrahi, T. and E. Grossman, "Deterministic Networking 1262 (DetNet) Security Considerations", draft-ietf-detnet- 1263 security-10 (work in progress), May 2020. 1265 [IEEE802.1AE-2018] 1266 IEEE Standards Association, "IEEE Std 802.1AE-2018 MAC 1267 Security (MACsec)", 2018, 1268 . 1270 [RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S. 1271 Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1 1272 Functional Specification", RFC 2205, DOI 10.17487/RFC2205, 1273 September 1997, . 1275 [RFC2474] Nichols, K., Blake, S., Baker, F., and D. Black, 1276 "Definition of the Differentiated Services Field (DS 1277 Field) in the IPv4 and IPv6 Headers", RFC 2474, 1278 DOI 10.17487/RFC2474, December 1998, 1279 . 1281 [RFC3272] Awduche, D., Chiu, A., Elwalid, A., Widjaja, I., and X. 1282 Xiao, "Overview and Principles of Internet Traffic 1283 Engineering", RFC 3272, DOI 10.17487/RFC3272, May 2002, 1284 . 1286 [RFC3985] Bryant, S., Ed. and P. Pate, Ed., "Pseudo Wire Emulation 1287 Edge-to-Edge (PWE3) Architecture", RFC 3985, 1288 DOI 10.17487/RFC3985, March 2005, 1289 . 1291 [RFC4301] Kent, S. and K. Seo, "Security Architecture for the 1292 Internet Protocol", RFC 4301, DOI 10.17487/RFC4301, 1293 December 2005, . 1295 [RFC4448] Martini, L., Ed., Rosen, E., El-Aawar, N., and G. Heron, 1296 "Encapsulation Methods for Transport of Ethernet over MPLS 1297 Networks", RFC 4448, DOI 10.17487/RFC4448, April 2006, 1298 . 1300 [RFC4875] Aggarwal, R., Ed., Papadimitriou, D., Ed., and S. 1301 Yasukawa, Ed., "Extensions to Resource Reservation 1302 Protocol - Traffic Engineering (RSVP-TE) for Point-to- 1303 Multipoint TE Label Switched Paths (LSPs)", RFC 4875, 1304 DOI 10.17487/RFC4875, May 2007, 1305 . 1307 [RFC5440] Vasseur, JP., Ed. and JL. Le Roux, Ed., "Path Computation 1308 Element (PCE) Communication Protocol (PCEP)", RFC 5440, 1309 DOI 10.17487/RFC5440, March 2009, 1310 . 1312 [RFC5586] Bocci, M., Ed., Vigoureux, M., Ed., and S. Bryant, Ed., 1313 "MPLS Generic Associated Channel", RFC 5586, 1314 DOI 10.17487/RFC5586, June 2009, 1315 . 1317 [RFC5920] Fang, L., Ed., "Security Framework for MPLS and GMPLS 1318 Networks", RFC 5920, DOI 10.17487/RFC5920, July 2010, 1319 . 1321 [RFC5921] Bocci, M., Ed., Bryant, S., Ed., Frost, D., Ed., Levrau, 1322 L., and L. Berger, "A Framework for MPLS in Transport 1323 Networks", RFC 5921, DOI 10.17487/RFC5921, July 2010, 1324 . 1326 [RFC6003] Papadimitriou, D., "Ethernet Traffic Parameters", 1327 RFC 6003, DOI 10.17487/RFC6003, October 2010, 1328 . 1330 [RFC6073] Martini, L., Metz, C., Nadeau, T., Bocci, M., and M. 1331 Aissaoui, "Segmented Pseudowire", RFC 6073, 1332 DOI 10.17487/RFC6073, January 2011, 1333 . 1335 [RFC6790] Kompella, K., Drake, J., Amante, S., Henderickx, W., and 1336 L. Yong, "The Use of Entropy Labels in MPLS Forwarding", 1337 RFC 6790, DOI 10.17487/RFC6790, November 2012, 1338 . 1340 [RFC8306] Zhao, Q., Dhody, D., Ed., Palleti, R., and D. King, 1341 "Extensions to the Path Computation Element Communication 1342 Protocol (PCEP) for Point-to-Multipoint Traffic 1343 Engineering Label Switched Paths", RFC 8306, 1344 DOI 10.17487/RFC8306, November 2017, 1345 . 1347 [RFC8660] Bashandy, A., Ed., Filsfils, C., Ed., Previdi, S., 1348 Decraene, B., Litkowski, S., and R. Shakir, "Segment 1349 Routing with the MPLS Data Plane", RFC 8660, 1350 DOI 10.17487/RFC8660, December 2019, 1351 . 1353 Authors' Addresses 1354 Balazs Varga (editor) 1355 Ericsson 1356 Magyar Tudosok krt. 11. 1357 Budapest 1117 1358 Hungary 1360 Email: balazs.a.varga@ericsson.com 1362 Janos Farkas 1363 Ericsson 1364 Magyar Tudosok krt. 11. 1365 Budapest 1117 1366 Hungary 1368 Email: janos.farkas@ericsson.com 1370 Lou Berger 1371 LabN Consulting, L.L.C. 1373 Email: lberger@labn.net 1375 Andrew G. Malis 1376 Malis Consulting 1378 Email: agmalis@gmail.com 1380 Stewart Bryant 1381 Futurewei Technologies 1383 Email: stewart.bryant@gmail.com 1385 Jouni Korhonen 1387 Email: jouni.nospam@gmail.com