idnits 2.17.1 draft-ietf-detnet-dp-sol-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (January 29, 2018) is 2272 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'TBD' is mentioned on line 1398, but not defined == Unused Reference: 'RFC6073' is defined on line 1522, but no explicit reference was found in the text == Outdated reference: A later version (-26) exists of draft-ietf-6man-segment-routing-header-08 == Outdated reference: A later version (-13) exists of draft-ietf-detnet-architecture-04 Summary: 0 errors (**), 0 flaws (~~), 5 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 DetNet J. Korhonen, Ed. 3 Internet-Draft Nordic 4 Intended status: Standards Track L. Andersson 5 Expires: August 2, 2018 Y. Jiang 6 N. Finn 7 Huawei 8 B. Varga 9 J. Farkas 10 Ericsson 11 CJ. Bernardos 12 UC3M 13 T. Mizrahi 14 Marvell 15 L. Berger 16 LabN 17 January 29, 2018 19 DetNet Data Plane Encapsulation 20 draft-ietf-detnet-dp-sol-01 22 Abstract 24 This document specifies Deterministic Networking data plane 25 encapsulation solutions. The described data plane solutions can be 26 applied over either IP or MPLS Packet Switched Networks. 28 Status of This Memo 30 This Internet-Draft is submitted in full conformance with the 31 provisions of BCP 78 and BCP 79. 33 Internet-Drafts are working documents of the Internet Engineering 34 Task Force (IETF). Note that other groups may also distribute 35 working documents as Internet-Drafts. The list of current Internet- 36 Drafts is at https://datatracker.ietf.org/drafts/current/. 38 Internet-Drafts are draft documents valid for a maximum of six months 39 and may be updated, replaced, or obsoleted by other documents at any 40 time. It is inappropriate to use Internet-Drafts as reference 41 material or to cite them other than as "work in progress." 43 This Internet-Draft will expire on August 2, 2018. 45 Copyright Notice 47 Copyright (c) 2018 IETF Trust and the persons identified as the 48 document authors. All rights reserved. 50 This document is subject to BCP 78 and the IETF Trust's Legal 51 Provisions Relating to IETF Documents 52 (https://trustee.ietf.org/license-info) in effect on the date of 53 publication of this document. Please review these documents 54 carefully, as they describe your rights and restrictions with respect 55 to this document. Code Components extracted from this document must 56 include Simplified BSD License text as described in Section 4.e of 57 the Trust Legal Provisions and are provided without warranty as 58 described in the Simplified BSD License. 60 Table of Contents 62 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 63 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 64 2.1. Terms used in this document . . . . . . . . . . . . . . . 4 65 2.2. Abbreviations . . . . . . . . . . . . . . . . . . . . . . 5 66 3. Requirements language . . . . . . . . . . . . . . . . . . . . 6 67 4. DetNet data plane overview . . . . . . . . . . . . . . . . . 6 68 4.1. DetNet data plane encapsulation requirements . . . . . . 8 69 4.2. Packet replication and elimination considerations . . . . 10 70 4.3. Packet reordering considerations . . . . . . . . . . . . 10 71 5. MPLS-based DetNet data plane solution . . . . . . . . . . . . 10 72 5.1. DetNet specific packet fields . . . . . . . . . . . . . . 10 73 5.2. Data plane encapsulation . . . . . . . . . . . . . . . . 11 74 5.3. DetNet control word . . . . . . . . . . . . . . . . . . . 12 75 5.4. Flow identification . . . . . . . . . . . . . . . . . . . 13 76 5.5. Service layer considerations . . . . . . . . . . . . . . 13 77 5.5.1. Edge node processing . . . . . . . . . . . . . . . . 13 78 5.5.2. Relay node processing . . . . . . . . . . . . . . . . 15 79 5.5.3. End system processing . . . . . . . . . . . . . . . . 17 80 5.6. Transport node considerations . . . . . . . . . . . . . . 17 81 5.6.1. Congestion protection . . . . . . . . . . . . . . . . 17 82 5.6.2. Explicit routes . . . . . . . . . . . . . . . . . . . 17 83 6. IPv6-based DetNet data plane solution . . . . . . . . . . . . 17 84 6.1. Data plane encapsulation . . . . . . . . . . . . . . . . 17 85 6.2. DetNet destination option . . . . . . . . . . . . . . . . 19 86 6.3. Flow identification . . . . . . . . . . . . . . . . . . . 20 87 6.4. Service layer considerations . . . . . . . . . . . . . . 20 88 6.4.1. Edge node processing . . . . . . . . . . . . . . . . 21 89 6.4.2. Relay node processing . . . . . . . . . . . . . . . . 23 90 6.4.3. End system processing . . . . . . . . . . . . . . . . 23 91 6.5. Transport node processing . . . . . . . . . . . . . . . . 23 92 6.5.1. Congestion protection . . . . . . . . . . . . . . . . 23 93 6.5.2. Explicit routes . . . . . . . . . . . . . . . . . . . 24 94 7. Other DetNet data plane considerations . . . . . . . . . . . 24 95 7.1. Class of Service . . . . . . . . . . . . . . . . . . . . 24 96 7.2. Quality of Service . . . . . . . . . . . . . . . . . . . 24 97 7.3. Cross-DetNet flow resource aggregation . . . . . . . . . 26 98 7.4. Bidirectional traffic . . . . . . . . . . . . . . . . . . 27 99 7.5. Layer 2 addressing and QoS Considerations . . . . . . . . 27 100 7.6. Interworking between MPLS- and IPv6-based encapsulations 28 101 7.7. IPv4 considerations . . . . . . . . . . . . . . . . . . . 28 102 8. Time synchronization . . . . . . . . . . . . . . . . . . . . 28 103 9. Management and control considerations . . . . . . . . . . . . 30 104 9.1. MPLS-based data plane . . . . . . . . . . . . . . . . . . 30 105 9.1.1. S-Label assignment and distribution . . . . . . . . . 30 106 9.1.2. Explicit routes . . . . . . . . . . . . . . . . . . . 30 107 9.2. IPv6-based data plane . . . . . . . . . . . . . . . . . . 30 108 9.2.1. Flow Label assignment and distribution . . . . . . . 30 109 9.2.2. Explicit routes . . . . . . . . . . . . . . . . . . . 31 110 9.3. Packet replication and elimination . . . . . . . . . . . 31 111 9.4. Congestion protection and latency control . . . . . . . . 31 112 9.5. Flow aggregation control . . . . . . . . . . . . . . . . 31 113 10. Security considerations . . . . . . . . . . . . . . . . . . . 31 114 11. IANA considerations . . . . . . . . . . . . . . . . . . . . . 31 115 12. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 31 116 13. References . . . . . . . . . . . . . . . . . . . . . . . . . 32 117 13.1. Normative references . . . . . . . . . . . . . . . . . . 32 118 13.2. Informative references . . . . . . . . . . . . . . . . . 34 119 Appendix A. Example of DetNet data plane operation . . . . . . . 35 120 Appendix B. Example of pinned paths using IPv6 . . . . . . . . . 36 121 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 36 123 1. Introduction 125 Deterministic Networking (DetNet) is a service that can be offered by 126 a network to DetNet flows. DetNet provides these flows extremely low 127 packet loss rates and assured maximum end-to-end delivery latency. 128 General background and concepts of DetNet can be found in 129 [I-D.ietf-detnet-architecture]. 131 This document specifies the DetNet data plane and the on-wire 132 encapsulation of DetNet flows. The specified encapsulation provides 133 the building blocks to enable the DetNet service layer functions and 134 allow flow identification as described in the DetNet Architecture. 135 Two data plane definitions are given. 137 1. MPLS-based: The enacapsulation resembles PseudoWires (PW) with an 138 MPLS Packet Switched Network (PSN) [RFC3985][RFC4385]. 140 2. Native-IP-based: The encapsulating protocol is IPv6 and the 141 solution relies on IP header fields, existing and DetNet specific 142 IPv6 eaxtension header options [RFC8200]. 144 [Editor's note: MPLS- and IPv6-based solutions are likely to be 145 split into different documents.] 147 It is worth noting that while MPLS-based solution can transport IP 148 packets a native-IP solution is meant for deployments where the 149 DetNet service layer functions are provided at the IP-layer rather 150 than the underlying transport network. The primary reason for this 151 is the benefit gained by enabling the use of a normal application 152 stack, where transport protocols such as TCP or UDP are directly 153 encapsulated in IP. 155 The DetNet transport layer functionality that provides congestion 156 protection for DetNet flows is assumed to be in place in a DetNet 157 node. 159 Furthermore, this document also describes how DetNet flows are 160 identified, how a DetNet Relay/Edge/Transit nodes work, and how the 161 Packet Replication and Elimination function (PREF) is implemented 162 with the two data plane solutions. 164 This document does not define the associated control plane functions, 165 or Operations, Administration, and Maintenance (OAM). It also does 166 not specify traffic handling capabilities required to deliver 167 congestion protection and latency control for DetNet flows at the 168 DetNet transport layer. 170 2. Terminology 172 2.1. Terms used in this document 174 This document uses the terminology established in the DetNet 175 architecture [I-D.ietf-detnet-architecture] and the DetNet Data Plane 176 Solution Alternatives [I-D.ietf-detnet-dp-alt]. 178 T-Label A label used to identify the LSP used to transport a 179 DetNet flow across an MPLS PSN, e.g., a hop-by-hop 180 label used between label switching routers (LSR). 182 S-Label A DetNet "service" label that is used between DetNet 183 nodes that implment also the DetNet service layer 184 functions. An S-Label is also used to identify a 185 DetNet flow at DetNet service layer. 187 Flow Label IPv6 header field that is used to identify a DetNet 188 flow (together with the source IP address field). 190 Local-ID A DetNet Edge and Relay node internal construct that 191 uniquely identifies a DetNet flow within a node and 192 never appear on-wire. It may be used to select proper 193 forwarding and/or DetNet specific service function. 195 PREF A Packet Replication and Elimination Function (PREF) 196 does the replication and elimination processing of 197 DetNet flow packets in edge or relay nodes. The 198 replication function is essentially the existing 1+1 199 protection mechanism. The elimination function reuses 200 and extends the existing duplicate detection mechanism 201 to operate over multiple (separate) DetNet member flows 202 of a DetNet compound flow. 204 DetNet Control Word A control word used for sequencing and 205 identifying duplaicate packets at the DetNet service 206 layer. 208 2.2. Abbreviations 210 The following abbreviations used in this document: 212 AC Attachment Circuit. 214 CE Customer Edge equipment. 216 CoS Class of Service. 218 CW Control Word. 220 d-CW DetNet Control Word. 222 DetNet Deterministic Networking. 224 DF DetNet Flow. 226 L2VPN Layer 2 Virtual Private Network. 228 LSR Label Switching Router. 230 MPLS Multiprotocol Label Switching. 232 MPLS-TP Multiprotocol Label Switching - Transport Profile. 234 MS-PW Multi-Segment PseudoWire (MS-PW). 236 NSP Native Service Processing. 238 OAM Operations, Administration, and Maintenance. 240 PE Provider Edge. 242 PREF Packet Replication and Elimination Function. 244 PSN Packet Switched Network. 246 PW PseudoWire. 248 QoS Quality of Service. 250 TSN Time-Sensitive Network. 252 3. Requirements language 254 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 255 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 256 document are to be interpreted as described in [RFC2119]. 258 4. DetNet data plane overview 260 This document describes how to use IP and/or MPLS to support a data 261 plane method of flow identification and packet formwarding over 262 layer-3. Two different cases are covered: (i) the inter-connect 263 scenario, in which IEEE802.1 TSN is routed over a layer-3 network 264 (i.e., to enlarge the layer-2 domain), and (ii) native connectivity 265 between DetNet-aware end systems. 267 Figure 1 illustrates how DetNet can provide services for IEEE 268 802.1TSN end systems over a DetNet enabled network. The edge nodes 269 insert and remove required DetNet data plane encapsulation. The 'X' 270 in the edge and relay nodes represents a potential DetNet flow packet 271 replication and elimination point. This conceptually parallels L2VPN 272 services, and could leverage existing related solutions as discussed 273 below. 275 TSN |<---------- End to End DetNet Service ------>| TSN 276 Service | Transit Transit | Service 277 TSN (AC) | |<-Tunnel->| |<-Tnl->| | (AC) TSN 278 End | V V 1 V V 2 V V | End 279 System | +--------+ +--------+ +--------+ | System 280 +---+ | | E1 |==========| R1 |=======| E2 | | +---+ 281 | |--|----|._X_....|..DetNet..|.._ _...|..DF3..|...._X_.|---|---| | 282 |CE1| | | \ | Flow 1 | X | | / | | |CE2| 283 | | | \_.|...DF2....|._/ \_..|..DF4..|._/ | | | 284 +---+ | |==========| |=======| | +---+ 285 ^ +--------+ +--------+ +--------+ ^ 286 | Edge Node Relay Node Edge Node | 287 | | 288 |<----- Emulated Time Sensitive Networking (TSN) Service ---->| 290 Figure 1: IEEE 802.1TSN over DetNet 292 Figure 2 illustrates how end to end MPLS-based DetNet service can be 293 provided. In this case, the end systems are able to send and receive 294 DetNet flows. For example, an end system sends data encapsulated in 295 MPLS. Like earlier the 'X' in the end systems, edge and relay nodes 296 represents potential DetNet flow packet replication and elimination 297 points. Here the relay nodes may change the underlying transport, 298 for example tunneling IP over MPLS, or simply interconnect network 299 segments. 301 DetNet DetNet 302 Service Transit Transit Service 303 DetNet | |<-Tnl->| |<-Tnl->| | DetNet 304 End | V 1 V V 2 V | End 305 System | +--------+ +--------+ +--------+ | System 306 +---+ | | R1 |=======| R2 |=======| R3 | | +---+ 307 | X...DFa...|._X_....|..DF1..|.__ ___.|..DF3..|...._X_.|.DFa..|.X | 308 |CE1|========| \ | | X | | / |======|CE2| 309 | | | | \_.|..DF2..|._/ \__.|..DF4..|._/ | | | | 310 +---+ | |=======| |=======| | +---+ 311 ^ +--------+ +--------+ +--------+ ^ 312 | Relay Node Relay Node Relay Node | 313 | | 314 |<--------------- End to End DetNet Service -------------->| 316 Figure 2: MPLS-Based Native DetNet 318 Figure 3 illustrates how end to end IP-based DetNet service can be 319 provided. In this case, the end systems are able to send and receive 320 DetNet flows. [Editor's note: TBD] 321 NOTE: This figures is TBD 323 DetNet DetNet 324 Service Transit Transit Service 325 DetNet | |<-Tnl->| |<-Tnl->| | DetNet 326 End | V 1 V V 2 V | End 327 System | +--------+ +--------+ +--------+ | System 328 +---+ | | R1 |=======| R2 |=======| R3 | | +---+ 329 | X...DFa...| | | | | | .|.X | 330 | H1|========| | | | | |======| H2| 331 | | | | | | | | | | | | 332 +---+ | |=======| |=======| | +---+ 333 ^ +--------+ +--------+ +--------+ ^ 334 | Relay Node Relay Node Relay Node | 335 | | 336 |<--------------- End to End DetNet Service -------------->| 338 Figure 3: IP-Based Native DetNet 340 4.1. DetNet data plane encapsulation requirements 342 Two major groups of scenarios can be distinguished which require flow 343 identification during transport: 345 1. DetNet function related scenarios: 347 * Congestion protection and latency control: usage of allocated 348 resources (queuing, policing, shaping). 350 * Explicit routes: select/apply the flow specific path. 352 * Service protection: recognize DetNet compound and member flows 353 for replication an elimination. 355 Comment #12 I am not sure whether the correct architectural 356 construct is flow or flow group. Flow suggests that sharing/ 357 aggregation is not allowed but whether this is allowed or not 358 is an application specific issue. 360 Discussion: Agree that a flow group would be a better 361 characterization. 363 Comment #13 I think that there needs to be some clarification as 364 to whether FG is is understood by the DN system exclusively or 365 whether there is an expectation that it is understood by the 366 underlay. 368 Discussion: Agree that more detail is needed here. DetNet aware 369 nodes need to understand flow groups. Underlay needs to be 370 aware of flow groups at the resource allocation level. 372 2. OAM function related scenarios: 374 * troubleshooting (e.g., identify misbehaving flows, etc.) 376 * recognize flow(s) for analytics (e.g., increase counters, 377 etc.) 379 * correlate events with flows (e.g., volume above threshold, 380 etc.) 382 * etc. 384 Each DetNet node (edge, relay and transit) use an internal/ 385 implementation specific local-ID of the DetNet-(compound)-flow in 386 order to accomplish its role during transport. Recognizing the 387 DetNet flow is more relaxed for edge and relay nodes, as they are 388 fully aware of both the DetNet service and transport layers. The 389 primary DetNet role of intermediate transport nodes is limited to 390 ensuring congestion protection and latency control for the above 391 listed DetNet functions. 393 The DetNet data plane allows for the aggregation of DetNet flows, 394 e.g., via MPLS hierarchical LSPs, to improved scaling. When DetNet 395 flows are aggregated, transit nodes may have limited ability to 396 provide service on per-flow DetNet identifiers. Therefore, 397 identifying each individual DetNet flow on a transit node may not be 398 achieved in some network scenarios, but DetNet service can still be 399 assured in these scenarios through resource allocation and control. 401 Comment #14 You could introduce the concept of a flow group 402 identified into the packet. You may also include a flow id at a 403 lower layer. 405 Discussion: Agree on the identification properties. Adding a 406 specific id into actual on-wire formats is not necessarily needed. 408 On each DetNet node dealing with DetNet flows, an internal local-ID 409 is assumed to determine what local operation a packet goes through. 410 Therefore, local-IDs has to be unique on each edge and relay nodes. 411 Local-ID is unambiguously bound to the DetNet flow. 413 4.2. Packet replication and elimination considerations 415 DetNet service layer introduces packet replication and elimination 416 functionality (PREF) for use in DetNet edge and relay node and end 417 system packet processing. PREF MAY be enabled in a DetNet node and 418 the required processing is only applied to packets with a positive 419 flow identification at the DetNet service layer. PREF utilizes a 420 sequence number carried within a DetNet flow packets. 422 At a DetNet node level the output of the PREF elimination function is 423 always a single packet. The output of the PREF replication function 424 at a DetNet node level is always one or more packets (i.e., 1:M 425 replication). The replicated packets MUST share the same d-CW i.e., 426 the sequence number is the same for each member flow of the compound 427 flow. The location and mechanism on the packet processing pipeline 428 used for replication is implementation specific. 430 The complex part of the DetNet PREF processing is tracking the 431 history of received packets for multiple DetNet member flows. These 432 ingress DetNet member flows (to a node) MUST have the same local-ID 433 if they belong to the same DetNet (compound) flow and share the same 434 sequence number counter and the history information. The location of 435 the packet elimination on the packet processing pipeline is 436 implementation specific. 438 4.3. Packet reordering considerations 440 DetNet service layer introduces also packet reordering functionality 441 for use in DetNet edge and relay node and end system packet 442 processing. The reordering functionality MAY be enabled in a DetNet 443 node. The reordeing functionality relies on a presence of sequence 444 numbers in a DetNet (compound) flows. The reordering processing is 445 only applied to packets with a positive flow identification at the 446 DetNet service layer. 448 5. MPLS-based DetNet data plane solution 450 5.1. DetNet specific packet fields 452 The DetNet data plane encapsulation MUST include two DetNet specific 453 information elements in each packet of a DetNet flow: (1) a flow 454 identification and (2) a sequence number. 456 The DetNet data plane encapsulation may consists further elements 457 used for overlay tunneling, to distinguish between DetNet member 458 flows of the same DetNet compound flow or to support OAM functions. 460 5.2. Data plane encapsulation 462 Figure 4 illustrates a DetNet data plane MPLS encapsulation. The 463 MPLS-based encapsulation of the DetNet flows is a good fit for the 464 Layer-2 interconnect deployment cases (see Figure 1). Furthermore, 465 end to end DetNet service i.e., native DetNet deployment (see 466 Figure 2) is also possible if DetNet end systems are capable of 467 initiating and termination MPLS encapsulated packets. Transport of 468 IP encapsulated DetNet flows, see Section 6, over MPLS-based DetNet 469 data plane is also possible. Interworking between PW- and IPv6-based 470 encapsulations is discussed further in Section 7.6. 472 The MPLS-based DetNet data plane encapsulation consists of: 474 o DetNet control word (d-CW) containing sequencing information for 475 packet replication and duplicate elimination purposes. There MUST 476 a separate sequence number space for each DetNet flow. 478 o DetNet Label that identifies a DetNet flow within a DetNet Edge or 479 a Relay node. The DetNet label MUST be at the bottom of the label 480 stack. 482 o An optional DetNet service lable (S-Label) that represents DetNet 483 Service LSP used between DetNet Egde and/or Relay nodes. One 484 possible use of an S-Label is to identify DetNet member flows used 485 to provide protection to a DetNet compound flow, perhaps even when 486 both LSPs appear on the same link for some reason. 488 One or more MPLS transport LSP label(s) (T-label) which may be a hop- 489 by-hop label used between LSR and MUST appear higher in the label 490 stack than S-labels. A top of stack T-label may be PHPed before 491 arriving at a DetNet node. In general T-labels should be considered 492 to be part of the underlying transport network rather the actual 493 DetNet data plane encapsulation. 495 DetNet MPLS-based encapsulation 497 +---------------------------------+ 498 | | 499 | DetNet Flow | 500 | Payload Packet | 501 | | 502 +---------------------------------+ <--\ 503 | DetNet Control Word | | 504 +---------------------------------+ +--> DetNet data plane 505 | S-Label | | MPLS encapsulation 506 +---------------------------------+ <--/ 507 | T-Label(s) | 508 +---------------------------------+ 509 | Data-Link | 510 +---------------------------------+ 511 | Physical | 512 +---------------------------------+ 514 Figure 4: Encapsulation of a DetNet flow in an MPLS(-TP) PSN 516 5.3. DetNet control word 518 A DetNet control word (d-CW) conforms to the Generic PW MPLS Control 519 Word (PWMCW) defined in [RFC4385] and is illustrated in Figure 5. 520 The upper nibble of the d-CW MUST be set to zero (0). The effective 521 sequence number bit length is between 0 and 28 bits, and configured 522 either by a control plane or manually for each DetNet flow. The 523 sequence number is aligned to the right (least significant bits) and 524 unused bits MUST be set to zero (0). Each DetNet flow MUST have its 525 own sequence number counter. The sequence number is incremented by 526 one for each new packet. 528 The d-CW MUST always be present in a packet. In a case the sequence 529 number is not used (e.g., for DetNet-t-flows) the control plane or 530 the manual configuration has to define zero (0) bit length seqeunce 531 number and the value of the sequence number MUST be set to zero (0). 533 0 1 2 3 534 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 535 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 536 |0 0 0 0| Sequence Number | 537 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 539 Figure 5: DetNet Control Word 541 5.4. Flow identification 543 DetNet flow identification at a DetNet service layer is realized by 544 an S-label. It maps a Detnet flow to a specific d-CW in a DetNet 545 node. The S-label used for flow identification MUST be bottom label 546 of the label stack for a DetNet-s- or DetNet-st-flow and MUST precede 547 the d-CW. 549 An S-label for a single DetNet flow does not need to be unique DetNet 550 domain wide. As long as two or more different DetNet flows do not 551 errorneously map to a same d-CW in a DetNet node the labels may vary. 553 5.5. Service layer considerations 555 [Editor's note: quite a bit of unfinished and old text in the 556 following sections.] 558 The edge and relay node internal procedures of the PREF are 559 implementation specific. The order of a packet elimination or 560 replication is out of scope in this specification. However, care 561 should be taken that the replication function does not actually 562 loopback packets as "replicas". Looped back packets include 563 artificial delay when the node that originally initiated the packet 564 receives it again. Also, looped back packets may make the network 565 condition to look healthier than it actually is (in some cases link 566 failures are not reflected properly because looped back packets make 567 the situation appear better than it actually is). 569 Comment #29: SB> There needs to be some text about preventing a node 570 ever receiving its own replicated packets. Indeed that would 571 suggest that the flow id should be changed and replication should 572 only take place on configured flow IDs. I have a feeling that 573 this would all be a lot safer if replication only happened at 574 ingress and we managed the diversity of the paths. 576 Discussion: Agree on hardening the loopback text considerations. 578 5.5.1. Edge node processing 580 TBD. 582 [Editor's note: Since we are not defining the inner workings and 583 implementation of the DetNet Egde node - rather only what goes in and 584 what comes out, and of course the on-wire details, then the figures 585 shown in the coming section would not need to detail the inner 586 architecture of a DetNet Node.] 587 +---------------------------------------+ 588 | DetNet Edge Node | 589 +---------------------------------------+ 590 | | | | 591 | | | | 592 Client AC | +---o <-------> o o<----------> 593 | Elim. | | | | 594 <---------->o <-------| | +-------------+ 595 | | | | | 596 | +---o <-------> o | 597 | | | o<----------> 598 | | +-> o | 599 +-------------+ | +-------------+ 600 | | | | | 601 Client AC | | Repl. | | | 602 <---------->o o <-----X-> o o<----------> 603 | | Elim. | | 604 +-------------+ +-------------+ 605 | | | | 606 Client AC | | | | 607 <---------->o o <-------> o o<----------> 608 | | | | 609 +---------------------------------------+ 611 Figure 6: DetNet Edge Node processing 613 An edge node participates to the packet replication and duplication 614 elimination. Required processing is done within an extended 615 forwarder function. In the case the native service processing (NSP) 616 is IEEE 802.1CB [IEEE8021CB] capable, the packet replication and 617 duplicate elimination MAY entirely be done in the NSP and bypassing 618 the DetNet flow encapsulation and logic entirely, and thus is able to 619 operate over unmodified implementation and deployment. The NSP 620 approach works only between edge nodes and cannot make use of relay 621 nodes (see Section 5.5.2). 623 Comment #31 SB> This would be a fine way to operate the PW system - 624 edge to edge. 626 Discussion: When it comes to use of NSPs, agree. Also for "island 627 interconnect" this is a fine. However, when there is a need to do 628 PREF in a middle, plain edge to edge is not enough. 630 The DetNet-aware extended forwarder selects the egress DetNet member 631 flow based on the DetNet forwarding rules. In both "normal AC" and 632 "Packet AC" cases there may be no DetNet encapsulation header 633 available yet as it is the case with relay nodes (see Section 5.5.2). 635 It is the responsibility of the extended forwarder within the edge 636 node to push the DetNet specific encapsulation (if not already 637 present) to the packet before forwarding it to the appropriate egress 638 DetNet member flow instance(s). 640 Comment #32 SB> I am not convinced of the wisdom of having a mid- 641 point node convert a flow into a DN flow, which is what you are 642 implying here. This seems like an ingress function. 644 Discussion: OK. The text here has issues and seems to mix relay and 645 edge. 647 The extended forwarder MAY copy the sequencing information from the 648 native DetNet packet into the DetNet sequence number field and vice 649 versa. If there is no existing sequencing information available in 650 the native packet or the forwarder chose not to copy it from the 651 native packet, then the extended forwarder MUST maintain a sequence 652 number counter for each DetNet flow (indexed by the DetNet flow 653 identification). 655 5.5.2. Relay node processing 657 TBD. 659 A DetNet Relay node participates to the packet replication and 660 duplication elimination. This processing is done within an extended 661 forwarder function. Whether an ingress DetNet member flow receives 662 DetNet specific processing depends on how the forwarding is 663 programmed. For some DetNet member flows the relay node can act as a 664 normal relay node and for some apply the DetNet specific processing 665 (i.e., PREF). 667 Comment #34 SB> Again relay node is not a normal term, so am not 668 sure what it does in the absence of a PREF function. 670 Discussion: Relay node was a DetNet aware S-PE originally, which is 671 not explicitly stated here anymore, thus slightly confusing text 672 here. The text here needs to clarify the roles of PREF and 673 switching functions. A DetNet relay is described in the 674 architecture document. However, there is definitely room for 675 termonilogy and text improvements. 677 It is also possible to treat the relay node as a transit node, see 678 Section 7.3. Again, this is entirely up to how the forwarding has 679 been programmed. 681 The DetNet-aware forwarder selects the egress DetNet member flow 682 segment based on the flow identification. The mapping of ingress 683 DetNet member flow segment to egress DetNet member flow segment may 684 be statically or dynamically configured. Additionally the DetNet- 685 aware forwarder does duplicate frame elimination based on the flow 686 identification and the sequence number combination. The packet 687 replication is also done within the DetNet-aware forwarder. During 688 elimination and the replication process the sequence number of the 689 DetNet member flow MUST be preserved and copied to the egress DetNet 690 member flow. 692 +---------------------------------------+ 693 | DetNet Relay Node | 694 Ingress +---------------------------------------+ 695 | | | | Egress 696 | o---------> o--+ Elim. | 697 ----------->o | | +--------->o-----------> 698 | | +-----> o--+ | 699 Ingress +-------------+ | +-------------+ 700 | | | | | Egress 701 | | | | | 702 ----------->o o --+ +-> o o-----------> 703 | | | | | 704 Ingress +-------------+ | +-------------+ 705 | | | | | Egress 706 | | Repl. | | | 707 ----------->o o ------+-> o o-----------> 708 | | | | 709 Ingress +-------------+ +-------------+ 710 | | | | Egress 711 | | | | 712 ----------->o o --------> o o-----------> 713 | | | | 714 +---------------------------------------+ 716 Figure 7: DetNet Relay Node processing 718 Comment #35 SB> Somewhere in the dp document there needs to be a 719 note of the requirement for interfaces to do fast exchange of 720 counter state, and a note to those planning the network and 721 designing the control plane that they need to provide support for 722 this. 724 Discussion: We kinf of agree but also think the above exchange or 725 synchronization of counter states is not in our scope to solve. 727 5.5.3. End system processing 729 TBD. 731 5.6. Transport node considerations 733 5.6.1. Congestion protection 735 TBD. 737 5.6.2. Explicit routes 739 TBD. 741 6. IPv6-based DetNet data plane solution 743 6.1. Data plane encapsulation 745 Figure 8 illustrates a DetNet native IPv6 encapsulation. The native 746 IPv6 encapsulation is meant for end to end Detnet service use cases, 747 where the end stations are DetNet-aware (see Figure 3). Technically 748 it is possible to use the IPv6 encapsulation to tunnel any traffic 749 over a DetNet enabled network, which would make native IPv6 750 encapsulation also a valid data plane choice for an interconnect use 751 case (see Figure 1). 753 The native IPv6-based DetNet data plane encapsulation consists of: 755 o IPv6 header as the transport protocol. 757 o IPv6 header Flow Label that is used to help to identify a DetNet 758 flow (i.e., roughly an equivalent to an S-Label for the MPLS 759 encapsulation). A Flow Label together with the IPv6 source 760 address uniquely identifies a DetNet flow. 762 Comment #21 SB> Have we validated that it is unconditionally safe to 763 make this assumption about the use of the FL? 765 Discussion: RFC6437 does not restrict such use and DetNet DP 766 solution can always define their own use of flow label. It should 767 be noted that a DetNet aware node will always contain new code and 768 is not a load balancer. 770 o Zero, one or two DetNet Destination Options containing sequencing 771 information for packet replication and duplicate elimination 772 function (PREF), and/or packet reordering purposes. The DetNet 773 Destination Option is equivalent to the DetNet Control Word. If 774 PREF or packet reordering is not needed for the DetNet flow then 775 no DetNet Destination Option is inserted into the IPv6 header. 777 A DetNet-aware end station (a host) or an intermediate Detnet node 778 initiating an (or adding a tunnelling) IPv6 packet is responsible for 779 setting the Flow Label, adding the optional DetNet Destination 780 Option(s) for DetNet-s- or DetNet-st-flows, and possibly adding a 781 routing header such as the segment routing option (e.g., for pre- 782 defined paths [I-D.ietf-6man-segment-routing-header]). If a routing 783 header is inserted into the IPv6 packet for DetNet-s- or DetNet-st- 784 flows then a second instance of the DetNet Destination Option MUST be 785 added before the routing header (see Section 4.1 of [RFC8200]). 787 A DetNet-aware end station (a host) or an intermediate node receiving 788 an IPv6 packet destined to it and containing a DetNet Destination 789 Option does the appropriate processing of the packet. This may 790 involve packet duplication and elimination (PREF processing), 791 terminating a tunnel or delivering the packet to the upper layers/ 792 Applications. 794 +---------------------------------+ 795 | | 796 | DetNet Flow | 797 | Payload | 798 | | 799 /---------------------------------\ 800 H Optional DetNet DstOpt Hdr H 801 \---------------------------------/ 802 | IPv6 header | 803 | (with set Flow label) | 804 +---------------------------------+ 806 Figure 8: Encapsulation of a native IPv6 DetNet-s- or DetNet-st-flow 807 without a routing header 809 Figure 9 illustrates an IPv6 packet for the case where a routing 810 header has been added into the packet by a DetNet-aware end system 811 (again assuming DetNet-s- or DetNet-st-flows). Note that the use of 812 routing header such as the one with the segment routing option is not 813 mandatory for explicit routes. Similar functionality can be arranged 814 using other means as well (e.g., using policy routing or layer-2 815 means). 817 +---------------------------------+ 818 | | 819 | DetNet Flow | 820 | Payload | 821 | | 822 /---------------------------------\ 823 H DetNet DstOpt Hdr H 824 \---------------------------------/ 825 | Routing header | 826 /---------------------------------\ 827 H DetNet DstOpt Hdr H 828 \---------------------------------/ 829 | IPv6 header | 830 | (with set Flow label) | 831 +---------------------------------+ 833 Figure 9: Encapsulation of a native IPv6 DetNet-s- or DetNet-st-flows 834 with routing header 836 IPv6 extension headers can only be inserted by a node that initiated 837 the IPv6 packet. IPv6 extension headers, except for the Hop-by-Hop 838 Option headers, can only be processed by an IPv6 node that is 839 identified by the Destination Address field of the IPv6 header (see 840 Section 0 of [RFC8200]. Therefore, if a DetNet-aware end system only 841 inserted the DetNet Destination Option into the IPv6 but e.g., a 842 DetNet Edge node is configured to enforce an explicit route for the 843 IPv6 packet using a source routing header, then it has no other 844 possibility than add an outer tunneling IPv6 header with required 845 extension headers in it. The processing of IPv6 packets in a DetNet 846 Edge node is discussed further in Section 6.4.1. 848 6.2. DetNet destination option 850 A DetNet flow must carry sequencing information for packet 851 replication and elimination function (PREF) purposes. This document 852 specifies a new IPv6 Destination Option: the DetNet Destination 853 Option for that purpose. The format of the option is illustrated in 854 Figure 10. 856 0 1 2 3 857 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 858 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 859 | TBD1 | 4 | 0 | 28 bit sequence 860 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 861 number | 862 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 864 Figure 10: DetNet Destination Option 866 The Option Type for the DetNet Destination Option is set to TBD1. 867 [To be removed from the final version of the document: The Option 868 Type MUST have the two most significant bits set to 10b] 870 If an IPv6 packet gets dropped due the DetNet Service layer 871 processing based on the DetNet Destination Option an ICMPv6 packet of 872 any type MUST NOT be sent back to the source of the packet. 874 6.3. Flow identification 876 The DetNet flow identification is based on the IPv6 Flow Label and 877 the source address combination. The two fields uniquelly identify 878 the end to end native IPv6 encapsulated DetNet flow. Obviously, the 879 identification fails if any intermediate node modifies either the 880 source address or the Flow Label. 882 Comment #27 SB> See earlier. If there are enough IPv6 addresses to 883 address video fragments, why not DN flows? Then this problem goes 884 away. 886 Discussion: See the earlier comment #25 discussion. If nodes get 887 their addressies via DHCPv6 basically ruins this mechanism. Also 888 the assumption for this to work is that the node has a full /64 to 889 use, which is not always the case. Otherwise the idea is just 890 fine. 892 6.4. Service layer considerations 894 [Editor's note: this section is TBD. It will detail the PREF 895 functionality.] 897 o PREF - requires both flow identification and sequence numbering. 899 o Packet reordeing - requires both flow identification and sequence 900 numbering. 902 A DetNet service layer processing can be done at each DetNet node 903 that matches the IPv6 header's Destination Address. Then, if the 904 DetNet flow identification provides a positive match for the DetNet 905 flow that the node has a service layer state installed e.g., for PREF 906 or packet reordering purposes, further service layer processing takes 907 place. In a case of PREF or packet reordering that means processing 908 the DetNet Destination Option for the identified DetNet flow. 910 6.4.1. Edge node processing 912 [Editor's note: This is the start of the IPv6 handling text - there 913 are errors and bad language. The founding assumption is the use of 914 source routing when intermediate nodes (relays/edges) need to modify 915 packets. This is due the text in RFC8200 and the fact that without 916 hph options only routing+dsthdr is usable with intermediates under 917 strict RFC8200.. ] 919 [Editor's note: Regrading the source routing and the "example" SRv6 920 approach. Current text is based on the assumption that intermediates 921 cannot add/delete extension headers such as the SRv6. That said 922 adding adding a header implies adding a tunneling outer IPv6 header 923 and deleting a header implies a tunnel decapsulation. This is not 924 probably desired due to the involved overhead and to be discussed 925 whether it is possible/acceptable to just "process" the Application 926 flow packets.] 928 For a DetNet Edge node there are several scenarios that involve 929 modifications to the DetNet flow IPv6 packets. The assumption is 930 that a DetNet-aware end system has always set the IPv6 header flow 931 label properly for the flow identification purposes. A DetNet- or 932 DetNet-t-flow does not include the DetNet Destination Option. 933 Following cases have been identified: 935 1. A DetNet App-flow or a DetNet-t-flow packet arrives at an ingress 936 DetNet Edge node and DetNet service layer functions are done only 937 at DetNet Edge nodes. Possible explicit routes between edge 938 nodes are arranged by other than IPv6 specific means. 940 2. A DetNet App-flow or a DetNet-t-flow packet arrives at an ingress 941 DetNet Edge node and multiple DetNet Relay nodes may process 942 DetNet flow packets before reaching an egress DetNet Edge node. 943 Explicit routes between edge nodes has to be arranged by IPv6 944 specific means. 946 3. A DetNet-s- or a DetNet-st-flow packet arrives at an ingress 947 DetNet Edge node and DetNet service layer functions are done only 948 at DetNet Edge nodes. Possible explicit routes between edge 949 nodes are arranged by other than IPv6 specific means. 951 4. A DetNet-s- or a DetNet-st-flow packet arrives at an ingress 952 DetNet Edge node and multiple DetNet Relay nodes may process 953 DetNet flow packets before reaching an egress DetNet Edge node. 954 Explicit routes between edge nodes has to be arranged by IPv6 955 specific means. 957 A generic DetNet IPv6 encapsulation for a DetNet flow packet between 958 DetNet Edge nodes is shown in Figure 11. Essentially every time an 959 igress DetNet Edge node has to insert something into the DetNet flow 960 packet it has to add an outer tunneling IPv6 header, which then 961 contain possible additional extension headers. 963 +---------------------------------+ 964 | | 965 | DetNet Flow | 966 | Payload | 967 | | 968 +---------------------------------+ 969 | Optional DetNet DstOpt Hdr (1) | 970 +---------------------------------+ 971 | Inner IPv6 header | 972 | (with set Flow label) (1) | 973 +---------------------------------+ <--+ 974 | Optional Routing header | | 975 +---------------------------------+ | 976 | Optional DetNet DstOpt Hdr (2) | +-- Added/removed by 977 +---------------------------------+ | DetNet Edge node 978 | Outer IPv6 header | | 979 | (with set Flow label) (2) | | 980 +---------------------------------+ <--+ 982 Figure 11: Encapsulation of a DetNet-flow IPv6 packet at the DetNet 983 Edge node 985 6.4.1.1. Ingress DetNet Edge node processing 987 Case 1) MAY require an addition of the DetNet Destination Option if 988 packet reordering is requested at the egress DetNet Edge node. 989 Otherwise, no modifications except rewriting the IPv6 header flow 990 label to the packet is done. If modifications are required then: 992 o The outer IPv6 header is added with the Source Address set to the 993 ingress DetNet Edge node address and the Destination Address set 994 to the egress DetNet Edge node address. 996 o The flow label of the outer IPv6 header SHOULD be set to a value 997 maintained by the edge node. 999 o The DetNet Destination Option with the edge node managed per 1000 DetNet flow sequence number value is inserted into the outer IPv6 1001 header. 1003 Case 2) requires an addition of the DetNet Destination Option unless 1004 neither packet reordeing or PREF is enable at any DetNet Edge/Relay 1005 node. A source routing header has to be added for the explicit route 1006 purposes. An example of the source routing header is the Segment 1007 Routing header. The following modifications to DetNet flow IPv6 1008 packets are required: 1010 o An outer IPv6 header is added with the Source Address set to the 1011 ingress DetNet Edge node address and the Destination Address set 1012 to the egress DetNet Edge node address. 1014 o The flow label of the outer IPv6 header SHOULD be set to a value 1015 maintained by the edge node. 1017 o The DetNet Destination Option with the edge node managed per 1018 DetNet flow sequence number value MAY be inserted into the outer 1019 IPv6 header. 1021 o A source routing header with addresses of those DetNet Relay nodes 1022 that must be traversed is inserted into the outer IPv6 header. 1024 Case 3) ...[Editor's note: is it OK if the sequece number added here 1025 by the edge node has only local significance between the edge nodes 1026 and not end to end between end systems? ] 1028 Case 4) ... 1030 6.4.1.2. Engress DetNet Edge node processing 1032 6.4.2. Relay node processing 1034 TBD. 1036 6.4.3. End system processing 1038 TBD. 1040 6.5. Transport node processing 1042 6.5.1. Congestion protection 1043 6.5.2. Explicit routes 1045 7. Other DetNet data plane considerations 1047 7.1. Class of Service 1049 Class and quality of service, i.e., CoS and QoS, are terms that are 1050 often used interchangeably and confused. In the context of DetNet, 1051 CoS is used to refer to mechanisms that provide traffic forwarding 1052 treatment based on aggregate group basis and QoS is used to refer to 1053 mechanisms that provide traffic forwarding treatment based on a 1054 specific DetNet flow basis. Examples of existing network level CoS 1055 mechanisms include DiffServ which is enabled by IP header 1056 differentiated services code point (DSCP) field [RFC2474] and MPLS 1057 label traffic class field [RFC5462], and at Layer-2, by IEEE 802.1p 1058 priority code point (PCP). 1060 CoS for DetNet flows carried in PWs and MPLS is provided using the 1061 existing MPLS Differentiated Services (DiffServ) architecture 1062 [RFC3270]. Both E-LSP and L-LSP MPLS DiffServ modes MAY be used to 1063 support DetNet flows. The Traffic Class field (formerly the EXP 1064 field) of an MPLS label follows the definition of [RFC5462] and 1065 [RFC3270]. The Uniform, Pipe, and Short Pipe DiffServ tunneling and 1066 TTL processing models are described in [RFC3270] and [RFC3443] and 1067 MAY be used for MPLS LSPs supporting DetNet flows. MPLS ECN MAY also 1068 be used as defined in ECN [RFC5129] and updated by [RFC5462]. 1070 CoS for DetNet flows carried in IPv6 is provided using the standard 1071 differentiated services code point (DSCP) field [RFC2474] and related 1072 mechanisms. The 2-bit explicit congestion notification (ECN) 1073 [RFC3168] field MAY also be used. 1075 One additional consideration for DetNet nodes which support CoS 1076 services is that they MUST ensure that the CoS service classes do not 1077 impact the congestion protection and latency control mechanisms used 1078 to provide DetNet QoS. This requirement is similar to requirement 1079 for MPLS LSRs to that CoS LSPs do not impact the resources allocated 1080 to TE LSPs via [RFC3473]. 1082 7.2. Quality of Service 1084 Quality of Service (QoS) mechanisms for flow specific traffic 1085 treatment typically includes a guarantee/agreement for the service, 1086 and allocation of resources to support the service. Example QoS 1087 mechanisms include discrete resource allocation, admission control, 1088 flow identification and isolation, and sometimes path control, 1089 traffic protection, shaping, policing and remarking. Example 1090 protocols that support QoS control include Resource ReSerVation 1091 Protocol (RSVP) [RFC2205] (RSVP) and RSVP-TE [RFC3209] and [RFC3473]. 1092 The existing MPLS mechanisms defined to support CoS [RFC3270] can 1093 also be used to reserve resources for specific traffic classes. 1095 In addition to explicit routes, and packet replication and 1096 elimination, described in Section 5 above, DetNet provides zero 1097 congestion loss and bounded latency and jitter. As described in 1098 [I-D.ietf-detnet-architecture], there are different mechanisms that 1099 maybe used separately or in combination to deliver a zero congestion 1100 loss service. These mechanisms are provided by the either the MPLS 1101 or IP layers, and may be combined with the mechanisms defined by the 1102 underlying network layer such as 802.1TSN. 1104 A baseline set of QoS capabilities for DetNet flows carried in PWs 1105 and MPLS can provided by MPLS with Traffic Engineering (MPLS-TE) 1106 [RFC3209] and [RFC3473]. TE LSPs can also support explicit routes 1107 (path pinning). Current service definitions for packet TE LSPs can 1108 be found in "Specification of the Controlled Load Quality of 1109 Service", [RFC2211], "Specification of Guaranteed Quality of 1110 Service", [RFC2212], and "Ethernet Traffic Parameters", [RFC6003]. 1111 Additional service definitions are expected in future documents to 1112 support the full range of DetNet services. In all cases, the 1113 existing label-based marking mechanisms defined for TE-LSPs and even 1114 E-LSPs are use to support the identification of flows requiring 1115 DetNet QoS. 1117 QoS for DetNet flows carried in IPv6 MUST be provided locally by the 1118 DetNet-aware hosts and routers supporting DetNet flows. Such support 1119 will leverage the underlying network layer such as 802.1TSN. The 1120 traffic control mechanisms used to deliver QoS for IP encapsulated 1121 DetNet flows are expected to be defined in a future document. From 1122 an encapsulation perspective, and as defined in Section 6, the 1123 combination of the Flow Label together with the IP source address 1124 uniquely identifies a DetNet flow. 1126 Packets that are marked with a DetNet Class of Service value, but 1127 that have not been the subject of a completed reservation, can 1128 disrupt the QoS offered to properly reserved DetNet flows by using 1129 resources allocated to the reserved flows. Therefore, the network 1130 nodes of a DetNet network MUST: 1132 o Defend the DetNet QoS by discarding or remarking (to a non-DetNet 1133 CoS) packets received that are not the subject of a completed 1134 reservation. 1136 o Not use a DetNet reserved resource, e.g. a queue or shaper 1137 reserved for DetNet flows, for any packet that does not carry a 1138 DetNet Class of Service marker. 1140 7.3. Cross-DetNet flow resource aggregation 1142 The ability to aggregate individual flows, and their associated 1143 resource control, into a larger aggregate is an important technique 1144 for improving scaling of control in the data, management and control 1145 planes. This document identifies the traffic identification related 1146 aspects of aggregation of DetNet flows. The resource control and 1147 management aspects of aggregation (including the queuing/shaping/ 1148 policing implications) will be covered in other documents. The data 1149 plane implications of aggregation are independent for PW/MPLS and IP 1150 encapsulated DetNet flows. 1152 DetNet flows transported via MPLS can leverage MPLS-TE's existing 1153 support for hierarchical LSPs (H-LSPs), see [RFC4206]. H-LSPs are 1154 typically used to aggregate control and resources, they may also be 1155 used to provide OAM or protection for the aggregated LSPs. Arbitrary 1156 levels of aggregation naturally falls out of the definition for 1157 hierarchy and the MPLS label stack [RFC3032]. DetNet nodes which 1158 support aggregation (LSP hierarchy) map one or more LSPs (labels) 1159 into and from an H-LSP. Both carried LSPs and H-LSPs may or may not 1160 use the TC field, i.e., L-LSPs or E-LSPs. Such nodes will need to 1161 ensure that traffic from aggregated LSPs are placed (shaped/policed/ 1162 enqueued) onto the H-LSPs in a fashion that ensures the required 1163 DetNet service is preserved. 1165 DetNet flows transported via IP have more limited aggregation 1166 options, due to the available traffic flow identification fields of 1167 the IP solution. One available approach is to manage the resources 1168 associated with a DSCP identified traffic class and to map (remark) 1169 individually controlled DetNet flows onto that traffic class. This 1170 approach also requires that nodes support aggregation ensure that 1171 traffic from aggregated LSPs are placed (shaped/policed/enqueued) in 1172 a fashion that ensures the required DetNet service is preserved. 1174 Comment #38 SB> I am sure we can do better than this with SR, or the 1175 use of routing techniques that map certain addresses to certain 1176 paths. 1178 Discussion: -- 1180 In both the MPLS and IP cases, additional details of the traffic 1181 control capabilities needed at a DetNet-aware node may be covered in 1182 the new service descriptions mentioned above or in separate future 1183 documents. Management and control plane mechanisms will also need to 1184 ensure that the service required on the aggregate flow (H-LSP or 1185 DSCP) are provided, which may include the discarding or remarking 1186 mentioned in the previous sections. 1188 7.4. Bidirectional traffic 1190 Some DetNet applications generate bidirectional traffic. Using MPLS 1191 definitions [RFC5654] there are associated bidirectional flows, and 1192 co-routed bidirectional flows. MPLS defines a point-to-point 1193 associated bidirectional LSP as consisting of two unidirectional 1194 point-to-point LSPs, one from A to B and the other from B to A, which 1195 are regarded as providing a single logical bidirectional transport 1196 path. This would be analogous of standard IP routing, or PWs running 1197 over two reciprocal unidirection LSPs. MPLS defines a point-to-point 1198 co-routed bidirectional LSP as an associated bidirectional LSP which 1199 satisfies the additional constraint that its two unidirectional 1200 component LSPs follow the same path (in terms of both nodes and 1201 links) in both directions. An important property of co-routed 1202 bidirectional LSPs is that their unidirectional component LSPs share 1203 fate. In both types of bidirectional LSPs, resource allocations may 1204 differ in each direction. The concepts of associated bidirectional 1205 flows and co-routed bidirectional flows can be applied to DetNet 1206 flows as well whether IPv6 or MPLS is used. 1208 While the IPv6 and MPLS data planes must support bidirectional DetNet 1209 flows, there are no special bidirectional features with respect to 1210 the data plane other than need for the two directions take the same 1211 paths. Fate sharing and associated vs co-routed bidirectional flows 1212 can be managed at the control level. Note, that there is no stated 1213 requirement for bidirectional DetNet flows to be supported using the 1214 same IPv6 Flow Labels or MPLS Labels in each direction. Control 1215 mechanisms will need to support such bidirectional flows for both 1216 IPv6 and MPLS, but such mechanisms are out of scope of this document. 1217 An example control plane solution for MPLS can be found in [RFC7551]. 1219 7.5. Layer 2 addressing and QoS Considerations 1221 The Time-Sensitive Networking (TSN) Task Group of the IEEE 802.1 1222 Working Group have defined (and are defining) a number of amendments 1223 to IEEE 802.1Q [IEEE8021Q] that provide zero congestion loss and 1224 bounded latency in bridged networks. IEEE 802.1CB [IEEE8021CB] 1225 defines packet replication and elimination functions that should 1226 prove both compatible with and useful to, DetNet networks. 1228 As is the case for DetNet, a Layer 2 network node such as a bridge 1229 may need to identify the specific DetNet flow to which a packet 1230 belongs in order to provide the TSN/DetNet QoS for that packet. It 1231 also will likely need a CoS marking, such as the priority field of an 1232 IEEE Std 802.1Q VLAN tag, to give the packet proper service. 1234 Although the flow identification methods described in IEEE 802.1CB 1235 [IEEE8021CB] are flexible, and in fact, include IP 5-tuple 1236 identification methods, the baseline TSN standards assume that every 1237 Ethernet frame belonging to a TSN stream (i.e. DetNet flow) carries 1238 a multicast destination MAC address that is unique to that flow 1239 within the bridged network over which it is carried. Furthermore, 1240 IEEE 802.1CB [IEEE8021CB] describes three methods by which a packet 1241 sequence number can be encoded in an Ethernet frame. 1243 Ensuring that the proper Ethernet VLAN tag priority and destination 1244 MAC address are used on a DetNet/TSN packet may require further 1245 clarification of the customary L2/L3 transformations carried out by 1246 routers and edge label switches. Edge nodes may also have to move 1247 sequence number fields among Layer 2, PW, and IPv6 encapsulations. 1249 7.6. Interworking between MPLS- and IPv6-based encapsulations 1251 [Editor's note: add considerations for interworking between MPLS- 1252 based and native IPv6-based DetNet encapsuations.] 1254 7.7. IPv4 considerations 1256 [Editor's note: The fact is that there are and will be deployments 1257 using IPv4. Neglecting it entirely is not feasible.] 1259 8. Time synchronization 1261 Comment #39 SB> This section should point the reader to RFC8169 1262 (residence time in MPLS n/w. We need to consider if we need to 1263 introduce the same concept in IP. 1265 Discussion: Agree. For IP we could reference to PTPv2 or v3 over 1266 UDP/IP, since it measures residence time among other things. 1268 [Editor's note: describe a bit of issues and deployment 1269 considerations related to time-synchronization within DetNet. Refer 1270 to DT discussion and the slides that summarize different approaches 1271 and rough synchronization performance numbers. Finally, scope time- 1272 synchronization solution outside data plane.] 1274 When DetNet is used, there is an underlying assumption that the 1275 applicaton(s) require clock synchronization such as the Precision 1276 Time Protocol (PTP) [IEEE1588]. The relay nodes may or may not 1277 utilize clock synchronization in order to provide zero congestion 1278 loss and controlled latency delivery. In either case, there are a 1279 few possible approaches of how synchronization protocol packets are 1280 forwarded and handled by the network: 1282 o PTP packets can be sent either as DetNet flows or as high-priority 1283 best effort packets. Using DetNet for PTP packets requires 1284 careful consideration to prevent unwanted interactions between 1285 clock-synchronized network nodes and the packets that synchronize 1286 the clocks. 1288 o PTP packets are sent as a normal DetNet flow through network nodes 1289 that are not time-synchronized: in this approach PTP traffic is 1290 forwarded as a DetNet flow, and as such it is forwarded in a way 1291 that allows a low delay variation. However, since intermediate 1292 nodes do not take part in the synchronization protocol, this 1293 approach provides a relatively low degree of accuracy. 1295 o PTP with on-path support: in this approach PTP packets are sent as 1296 ordinary or as DetNet flows, and intermediate nodes take part in 1297 the protocol as Transparent Clocks or Boundary Clocks [IEEE1588]. 1298 The on-path PTP support by intermediate nodes provides a higher 1299 degree of accuracy than the previous approach. The actual 1300 accuracy depends on whether all intermediate nodes are PTP- 1301 capable, or only a subset of them. 1303 o Time-as-a-service: in this approach accurate time is provided as- 1304 a-service to the DetNet source and destination, as well as the 1305 intermediate nodes. Since traffic between the source and 1306 destination is sent over a provider network, if the provider 1307 supports time-as-a-service, then accurate time can be provided to 1308 both the source and the destination of DetNet traffic. This 1309 approach can potentially provide the highest degree of accuracy. 1311 It is expected that the latter approach will be the most common one, 1312 as it provides the highest degree of accuracy, and creates a layer 1313 separation between the DetNet data and the synchronization service. 1315 It should be noted that in all four approaches it is not recommended 1316 to use replication and elimination for synchronization packets; the 1317 replication/elimination approach may in some cases reduce the 1318 synchronization accuracy, since the observed path delay will be 1319 bivalent. 1321 Comment #40 SB> I am not sure why we sould not use PREP. We should 1322 explain to the reader. 1324 Discussion: Agree that a this can be opened a bit more in detail. 1325 The issue is explained briefly in the last sentence but it could 1326 be more clear. 1328 9. Management and control considerations 1330 [Editor's note: This section needs to be different for MPLS and IPv6 1331 solutions. Most solutions are technology dependant,] 1333 While management plane and control planes are traditionally 1334 considered separately, from the Data Plane perspective there is no 1335 practical difference based on the origin of flow provisioning 1336 information. This document therefore does not distinguish between 1337 information provided by a control plane protocol, e.g., RSVP-TE 1338 [RFC3209] and [RFC3473], or by a network management mechanisms, e.g., 1339 RestConf [RFC8040] and YANG [RFC7950]. 1341 [Editor's note: This section is a work in progress. discuss here 1342 what kind of enhancements are needed for DetNet and specifically for 1343 PREF and DetNet zero congest loss and latency control. Need to cover 1344 both traffic control (queuing) and connection control (control 1345 plane).] 1347 9.1. MPLS-based data plane 1349 9.1.1. S-Label assignment and distribution 1351 [Editor's note: Outdated and MPLS specific.. and needs more work.] 1353 The DetNet S-Label distribution follows the same mechanisms specified 1354 for XYZ . The details of the control plane protocol solution required 1355 for the label distribution and the management of the label number 1356 space are out of scope of this document. 1358 9.1.2. Explicit routes 1360 [Editor's note: Outdated.. and needs more work.] 1362 [TBD: based on MPLS TE and possibly IPv6 SR] 1364 9.2. IPv6-based data plane 1366 9.2.1. Flow Label assignment and distribution 1368 [Editor's note: Outdated and IPv6 Specific.. and needs more work.] 1370 The IPv6 Flow Label distribution and the label number space are out 1371 of scope of this document. However, it should be noted that the 1372 combination of the IPv6 source address and the IPv6 Flow Label is 1373 assumed to be unique within the DetNet-enabled network. Therefore, 1374 as long as each node is able to assign unique Flow Labels for the 1375 source address(es) it is using the DetNet-enabled network wide flow 1376 identification uniqueness is guaranteed. 1378 9.2.2. Explicit routes 1380 [Editor's note: Outdated.. and needs more work.] 1382 [TBD: What we have there for IPv6 and explicit routes] 1384 9.3. Packet replication and elimination 1386 [Editor's note: Outdated and at the functional level technology 1387 independent.. but needs more work.] 1389 The control plane protocol solution required for managing the PREF 1390 processing is outside the scope of this document. 1392 9.4. Congestion protection and latency control 1394 [TBD] 1396 9.5. Flow aggregation control 1398 [TBD] 1400 10. Security considerations 1402 The security considerations of DetNet in general are discussed in 1403 [I-D.ietf-detnet-architecture] and [I-D.sdt-detnet-security]. Other 1404 security considerations will be added in a future version of this 1405 draft. 1407 11. IANA considerations 1409 TBD. 1411 12. Acknowledgements 1413 The author(s) ACK and NACK. 1415 The following people were part of the DetNet Data Plane Solution 1416 Design Team: 1418 Jouni Korhonen 1420 Janos Farkas 1422 Norman Finn 1423 Balazs Varga 1425 Loa Andersson 1427 Tal Mizrahi 1429 David Mozes 1431 Yuanlong Jiang 1433 Carlos J. Bernardos 1435 The DetNet chairs serving during the DetNet Data Plane Solution 1436 Design Team: 1438 Lou Berger 1440 Pat Thaler 1442 13. References 1444 13.1. Normative references 1446 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1447 Requirement Levels", BCP 14, RFC 2119, 1448 DOI 10.17487/RFC2119, March 1997, 1449 . 1451 [RFC2211] Wroclawski, J., "Specification of the Controlled-Load 1452 Network Element Service", RFC 2211, DOI 10.17487/RFC2211, 1453 September 1997, . 1455 [RFC2212] Shenker, S., Partridge, C., and R. Guerin, "Specification 1456 of Guaranteed Quality of Service", RFC 2212, 1457 DOI 10.17487/RFC2212, September 1997, 1458 . 1460 [RFC2474] Nichols, K., Blake, S., Baker, F., and D. Black, 1461 "Definition of the Differentiated Services Field (DS 1462 Field) in the IPv4 and IPv6 Headers", RFC 2474, 1463 DOI 10.17487/RFC2474, December 1998, 1464 . 1466 [RFC3032] Rosen, E., Tappan, D., Fedorkow, G., Rekhter, Y., 1467 Farinacci, D., Li, T., and A. Conta, "MPLS Label Stack 1468 Encoding", RFC 3032, DOI 10.17487/RFC3032, January 2001, 1469 . 1471 [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition 1472 of Explicit Congestion Notification (ECN) to IP", 1473 RFC 3168, DOI 10.17487/RFC3168, September 2001, 1474 . 1476 [RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V., 1477 and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP 1478 Tunnels", RFC 3209, DOI 10.17487/RFC3209, December 2001, 1479 . 1481 [RFC3270] Le Faucheur, F., Wu, L., Davie, B., Davari, S., Vaananen, 1482 P., Krishnan, R., Cheval, P., and J. Heinanen, "Multi- 1483 Protocol Label Switching (MPLS) Support of Differentiated 1484 Services", RFC 3270, DOI 10.17487/RFC3270, May 2002, 1485 . 1487 [RFC3443] Agarwal, P. and B. Akyol, "Time To Live (TTL) Processing 1488 in Multi-Protocol Label Switching (MPLS) Networks", 1489 RFC 3443, DOI 10.17487/RFC3443, January 2003, 1490 . 1492 [RFC3473] Berger, L., Ed., "Generalized Multi-Protocol Label 1493 Switching (GMPLS) Signaling Resource ReserVation Protocol- 1494 Traffic Engineering (RSVP-TE) Extensions", RFC 3473, 1495 DOI 10.17487/RFC3473, January 2003, 1496 . 1498 [RFC4206] Kompella, K. and Y. Rekhter, "Label Switched Paths (LSP) 1499 Hierarchy with Generalized Multi-Protocol Label Switching 1500 (GMPLS) Traffic Engineering (TE)", RFC 4206, 1501 DOI 10.17487/RFC4206, October 2005, 1502 . 1504 [RFC4385] Bryant, S., Swallow, G., Martini, L., and D. McPherson, 1505 "Pseudowire Emulation Edge-to-Edge (PWE3) Control Word for 1506 Use over an MPLS PSN", RFC 4385, DOI 10.17487/RFC4385, 1507 February 2006, . 1509 [RFC5129] Davie, B., Briscoe, B., and J. Tay, "Explicit Congestion 1510 Marking in MPLS", RFC 5129, DOI 10.17487/RFC5129, January 1511 2008, . 1513 [RFC5462] Andersson, L. and R. Asati, "Multiprotocol Label Switching 1514 (MPLS) Label Stack Entry: "EXP" Field Renamed to "Traffic 1515 Class" Field", RFC 5462, DOI 10.17487/RFC5462, February 1516 2009, . 1518 [RFC6003] Papadimitriou, D., "Ethernet Traffic Parameters", 1519 RFC 6003, DOI 10.17487/RFC6003, October 2010, 1520 . 1522 [RFC6073] Martini, L., Metz, C., Nadeau, T., Bocci, M., and M. 1523 Aissaoui, "Segmented Pseudowire", RFC 6073, 1524 DOI 10.17487/RFC6073, January 2011, 1525 . 1527 [RFC8200] Deering, S. and R. Hinden, "Internet Protocol, Version 6 1528 (IPv6) Specification", STD 86, RFC 8200, 1529 DOI 10.17487/RFC8200, July 2017, 1530 . 1532 13.2. Informative references 1534 [I-D.ietf-6man-segment-routing-header] 1535 Previdi, S., Filsfils, C., Raza, K., Dukes, D., Leddy, J., 1536 Field, B., daniel.voyer@bell.ca, d., 1537 daniel.bernier@bell.ca, d., Matsushima, S., Leung, I., 1538 Linkova, J., Aries, E., Kosugi, T., Vyncke, E., Lebrun, 1539 D., Steinberg, D., and R. Raszuk, "IPv6 Segment Routing 1540 Header (SRH)", draft-ietf-6man-segment-routing-header-08 1541 (work in progress), January 2018. 1543 [I-D.ietf-detnet-architecture] 1544 Finn, N., Thubert, P., Varga, B., and J. Farkas, 1545 "Deterministic Networking Architecture", draft-ietf- 1546 detnet-architecture-04 (work in progress), October 2017. 1548 [I-D.ietf-detnet-dp-alt] 1549 Korhonen, J., Farkas, J., Mirsky, G., Thubert, P., 1550 Zhuangyan, Z., and L. Berger, "DetNet Data Plane Protocol 1551 and Solution Alternatives", draft-ietf-detnet-dp-alt-00 1552 (work in progress), October 2016. 1554 [I-D.sdt-detnet-security] 1555 Mizrahi, T., Grossman, E., Hacker, A., Das, S., 1556 "Deterministic Networking (DetNet) Security 1557 Considerations, draft-sdt-detnet-security, work in 1558 progress", 2017. 1560 [IEEE1588] 1561 IEEE, "IEEE 1588 Standard for a Precision Clock 1562 Synchronization Protocol for Networked Measurement and 1563 Control Systems Version 2", 2008. 1565 [IEEE8021CB] 1566 Finn, N., "Draft Standard for Local and metropolitan area 1567 networks - Seamless Redundancy", IEEE P802.1CB 1568 /D2.1 P802.1CB, December 2015, 1569 . 1572 [IEEE8021Q] 1573 IEEE 802.1, "Standard for Local and metropolitan area 1574 networks--Bridges and Bridged Networks (IEEE Std 802.1Q- 1575 2014)", 2014, . 1577 [RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S. 1578 Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1 1579 Functional Specification", RFC 2205, DOI 10.17487/RFC2205, 1580 September 1997, . 1582 [RFC3985] Bryant, S., Ed. and P. Pate, Ed., "Pseudo Wire Emulation 1583 Edge-to-Edge (PWE3) Architecture", RFC 3985, 1584 DOI 10.17487/RFC3985, March 2005, 1585 . 1587 [RFC5654] Niven-Jenkins, B., Ed., Brungard, D., Ed., Betts, M., Ed., 1588 Sprecher, N., and S. Ueno, "Requirements of an MPLS 1589 Transport Profile", RFC 5654, DOI 10.17487/RFC5654, 1590 September 2009, . 1592 [RFC7551] Zhang, F., Ed., Jing, R., and R. Gandhi, Ed., "RSVP-TE 1593 Extensions for Associated Bidirectional Label Switched 1594 Paths (LSPs)", RFC 7551, DOI 10.17487/RFC7551, May 2015, 1595 . 1597 [RFC7950] Bjorklund, M., Ed., "The YANG 1.1 Data Modeling Language", 1598 RFC 7950, DOI 10.17487/RFC7950, August 2016, 1599 . 1601 [RFC8040] Bierman, A., Bjorklund, M., and K. Watsen, "RESTCONF 1602 Protocol", RFC 8040, DOI 10.17487/RFC8040, January 2017, 1603 . 1605 Appendix A. Example of DetNet data plane operation 1607 [Editor's note: Add a simplified example of DetNet data plane and how 1608 labels etc work in the case of MPLS-based PSN and utilizing PREF. 1609 The figure is subject to change depending on the further DT decisions 1610 on the label handling..] 1612 Appendix B. Example of pinned paths using IPv6 1614 TBD. 1616 Authors' Addresses 1618 Jouni Korhonen (editor) 1619 Nordic Semiconductor 1621 Email: jouni.nospam@gmail.com 1623 Loa Andersson 1624 Huawei 1626 Email: loa@pi.nu 1628 Yuanlong Jiang 1629 Huawei 1631 Email: jiangyuanlong@huawei.com 1633 Norman Finn 1634 Huawei 1635 3101 Rio Way 1636 Spring Valley, CA 91977 1637 USA 1639 Email: norman.finn@mail01.huawei.com 1641 Balazs Varga 1642 Ericsson 1643 Konyves Kalman krt. 11/B 1644 Budapest 1097 1645 Hungary 1647 Email: balazs.a.varga@ericsson.com 1648 Janos Farkas 1649 Ericsson 1650 Konyves Kalman krt. 11/B 1651 Budapest 1097 1652 Hungary 1654 Email: janos.farkas@ericsson.com 1656 Carlos J. Bernardos 1657 Universidad Carlos III de Madrid 1658 Av. Universidad, 30 1659 Leganes, Madrid 28911 1660 Spain 1662 Phone: +34 91624 6236 1663 Email: cjbc@it.uc3m.es 1664 URI: http://www.it.uc3m.es/cjbc/ 1666 Tal Mizrahi 1667 Marvell 1668 6 Hamada st. 1669 Yokneam 1670 Israel 1672 Email: talmi@marvell.com 1674 Lou Berger 1675 LabN Consulting, L.L.C. 1677 Email: lberger@labn.net