idnits 2.17.1 draft-ietf-detnet-dp-sol-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (March 5, 2018) is 2243 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'TBD' is mentioned on line 1736, but not defined == Unused Reference: 'RFC6073' is defined on line 1860, but no explicit reference was found in the text == Outdated reference: A later version (-26) exists of draft-ietf-6man-segment-routing-header-08 == Outdated reference: A later version (-13) exists of draft-ietf-detnet-architecture-04 Summary: 0 errors (**), 0 flaws (~~), 5 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 DetNet J. Korhonen, Ed. 3 Internet-Draft Nordic 4 Intended status: Standards Track L. Andersson 5 Expires: September 6, 2018 Y. Jiang 6 N. Finn 7 Huawei 8 B. Varga 9 J. Farkas 10 Ericsson 11 CJ. Bernardos 12 UC3M 13 T. Mizrahi 14 Marvell 15 L. Berger 16 LabN 17 March 5, 2018 19 DetNet Data Plane Encapsulation 20 draft-ietf-detnet-dp-sol-03 22 Abstract 24 This document specifies Deterministic Networking data plane 25 encapsulation solutions. The described data plane solutions can be 26 applied over either IP or MPLS Packet Switched Networks. 28 Status of This Memo 30 This Internet-Draft is submitted in full conformance with the 31 provisions of BCP 78 and BCP 79. 33 Internet-Drafts are working documents of the Internet Engineering 34 Task Force (IETF). Note that other groups may also distribute 35 working documents as Internet-Drafts. The list of current Internet- 36 Drafts is at https://datatracker.ietf.org/drafts/current/. 38 Internet-Drafts are draft documents valid for a maximum of six months 39 and may be updated, replaced, or obsoleted by other documents at any 40 time. It is inappropriate to use Internet-Drafts as reference 41 material or to cite them other than as "work in progress." 43 This Internet-Draft will expire on September 6, 2018. 45 Copyright Notice 47 Copyright (c) 2018 IETF Trust and the persons identified as the 48 document authors. All rights reserved. 50 This document is subject to BCP 78 and the IETF Trust's Legal 51 Provisions Relating to IETF Documents 52 (https://trustee.ietf.org/license-info) in effect on the date of 53 publication of this document. Please review these documents 54 carefully, as they describe your rights and restrictions with respect 55 to this document. Code Components extracted from this document must 56 include Simplified BSD License text as described in Section 4.e of 57 the Trust Legal Provisions and are provided without warranty as 58 described in the Simplified BSD License. 60 Table of Contents 62 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 63 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 64 2.1. Terms used in this document . . . . . . . . . . . . . . . 4 65 2.2. Abbreviations . . . . . . . . . . . . . . . . . . . . . . 5 66 3. Requirements language . . . . . . . . . . . . . . . . . . . . 6 67 4. DetNet data plane overview . . . . . . . . . . . . . . . . . 6 68 4.1. DetNet data plane encapsulation requirements . . . . . . 8 69 4.2. Packet replication and elimination considerations . . . . 10 70 4.3. Packet reordering considerations . . . . . . . . . . . . 10 71 5. DetNet encapsulation . . . . . . . . . . . . . . . . . . . . 10 72 5.1. End-system specific considerations . . . . . . . . . . . 10 73 5.2. DetNet domain specific considerations . . . . . . . . . . 12 74 5.2.1. DetNet Bridging Service . . . . . . . . . . . . . . . 13 75 5.2.2. DetNet Routing Service . . . . . . . . . . . . . . . 14 76 5.3. DetNet Inter-Working Function (DN-IWF) . . . . . . . . . 17 77 5.3.1. Networks with multiple technology segments . . . . . 17 78 5.3.2. DN-IWF related considerations . . . . . . . . . . . . 18 79 6. MPLS-based DetNet data plane solution . . . . . . . . . . . . 19 80 6.1. DetNet specific packet fields . . . . . . . . . . . . . . 19 81 6.2. Data plane encapsulation . . . . . . . . . . . . . . . . 19 82 6.3. DetNet control word . . . . . . . . . . . . . . . . . . . 20 83 6.4. Flow identification . . . . . . . . . . . . . . . . . . . 21 84 6.5. Service layer considerations . . . . . . . . . . . . . . 21 85 6.5.1. Edge node processing . . . . . . . . . . . . . . . . 22 86 6.5.2. Relay node processing . . . . . . . . . . . . . . . . 23 87 6.5.3. End system processing . . . . . . . . . . . . . . . . 25 88 6.6. Transport node considerations . . . . . . . . . . . . . . 25 89 6.6.1. Congestion protection . . . . . . . . . . . . . . . . 25 90 6.6.2. Explicit routes . . . . . . . . . . . . . . . . . . . 25 91 7. IPv6-based DetNet data plane solution . . . . . . . . . . . . 25 92 7.1. Data plane encapsulation . . . . . . . . . . . . . . . . 25 93 7.2. DetNet destination option . . . . . . . . . . . . . . . . 27 94 7.3. Flow identification . . . . . . . . . . . . . . . . . . . 28 95 7.4. Service layer considerations . . . . . . . . . . . . . . 28 96 7.4.1. Edge node processing . . . . . . . . . . . . . . . . 29 97 7.4.2. Relay node processing . . . . . . . . . . . . . . . . 31 98 7.4.3. End system processing . . . . . . . . . . . . . . . . 31 99 7.5. Transport node processing . . . . . . . . . . . . . . . . 31 100 7.5.1. Congestion protection . . . . . . . . . . . . . . . . 31 101 7.5.2. Explicit routes . . . . . . . . . . . . . . . . . . . 32 102 8. Other DetNet data plane considerations . . . . . . . . . . . 32 103 8.1. Class of Service . . . . . . . . . . . . . . . . . . . . 32 104 8.2. Quality of Service . . . . . . . . . . . . . . . . . . . 32 105 8.3. Cross-DetNet flow resource aggregation . . . . . . . . . 34 106 8.4. Bidirectional traffic . . . . . . . . . . . . . . . . . . 35 107 8.5. Layer 2 addressing and QoS Considerations . . . . . . . . 35 108 8.6. Interworking between MPLS- and IPv6-based encapsulations 36 109 8.7. IPv4 considerations . . . . . . . . . . . . . . . . . . . 36 110 9. Time synchronization . . . . . . . . . . . . . . . . . . . . 36 111 10. Management and control considerations . . . . . . . . . . . . 38 112 10.1. MPLS-based data plane . . . . . . . . . . . . . . . . . 38 113 10.1.1. S-Label assignment and distribution . . . . . . . . 38 114 10.1.2. Explicit routes . . . . . . . . . . . . . . . . . . 38 115 10.2. IPv6-based data plane . . . . . . . . . . . . . . . . . 38 116 10.2.1. Flow Label assignment and distribution . . . . . . . 38 117 10.2.2. Explicit routes . . . . . . . . . . . . . . . . . . 39 118 10.3. Packet replication and elimination . . . . . . . . . . . 39 119 10.4. Congestion protection and latency control . . . . . . . 39 120 10.5. Flow aggregation control . . . . . . . . . . . . . . . . 39 121 11. Security considerations . . . . . . . . . . . . . . . . . . . 39 122 12. IANA considerations . . . . . . . . . . . . . . . . . . . . . 39 123 13. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 39 124 14. References . . . . . . . . . . . . . . . . . . . . . . . . . 40 125 14.1. Normative references . . . . . . . . . . . . . . . . . . 40 126 14.2. Informative references . . . . . . . . . . . . . . . . . 42 127 Appendix A. Example of DetNet data plane operation . . . . . . . 43 128 Appendix B. Example of pinned paths using IPv6 . . . . . . . . . 44 129 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 44 131 1. Introduction 133 Deterministic Networking (DetNet) is a service that can be offered by 134 a network to DetNet flows. DetNet provides these flows extremely low 135 packet loss rates and assured maximum end-to-end delivery latency. 136 General background and concepts of DetNet can be found in 137 [I-D.ietf-detnet-architecture]. 139 This document specifies the DetNet data plane and the on-wire 140 encapsulation of DetNet flows. The specified encapsulation provides 141 the building blocks to enable the DetNet service layer functions and 142 allow flow identification as described in the DetNet Architecture. 143 Two data plane definitions are given. 145 1. MPLS-based: The enacapsulation resembles PseudoWires (PW) with an 146 MPLS Packet Switched Network (PSN) [RFC3985][RFC4385]. 148 2. Native-IP-based: The encapsulating protocol is IPv6 and the 149 solution relies on IP header fields, existing and DetNet specific 150 IPv6 eaxtension header options [RFC8200]. 152 [Editor's note: MPLS- and IPv6-based solutions are likely to be 153 split into different documents.] 155 It is worth noting that while MPLS-based solution can transport IP 156 packets a native-IP solution is meant for deployments where the 157 DetNet service layer functions are provided at the IP-layer rather 158 than the underlying transport network. The primary reason for this 159 is the benefit gained by enabling the use of a normal application 160 stack, where transport protocols such as TCP or UDP are directly 161 encapsulated in IP. 163 The DetNet transport layer functionality that provides congestion 164 protection for DetNet flows is assumed to be in place in a DetNet 165 node. 167 Furthermore, this document also describes how DetNet flows are 168 identified, how a DetNet Relay/Edge/Transit nodes work, and how the 169 Packet Replication and Elimination function (PREF) is implemented 170 with the two data plane solutions. 172 This document does not define the associated control plane functions, 173 or Operations, Administration, and Maintenance (OAM). It also does 174 not specify traffic handling capabilities required to deliver 175 congestion protection and latency control for DetNet flows at the 176 DetNet transport layer. 178 2. Terminology 180 2.1. Terms used in this document 182 This document uses the terminology established in the DetNet 183 architecture [I-D.ietf-detnet-architecture] and the DetNet Data Plane 184 Solution Alternatives [I-D.ietf-detnet-dp-alt]. 186 T-Label A label used to identify the LSP used to transport a 187 DetNet flow across an MPLS PSN, e.g., a hop-by-hop 188 label used between label switching routers (LSR). 190 S-Label A DetNet "service" label that is used between DetNet 191 nodes that implment also the DetNet service layer 192 functions. An S-Label is also used to identify a 193 DetNet flow at DetNet service layer. 195 Flow Label IPv6 header field that is used to identify a DetNet 196 flow (together with the source IP address field). 198 Local-ID A DetNet Edge and Relay node internal construct that 199 uniquely identifies a DetNet flow within a node and 200 never appear on-wire. It may be used to select proper 201 forwarding and/or DetNet specific service function. 203 PREF A Packet Replication and Elimination Function (PREF) 204 does the replication and elimination processing of 205 DetNet flow packets in edge or relay nodes. The 206 replication function is essentially the existing 1+1 207 protection mechanism. The elimination function reuses 208 and extends the existing duplicate detection mechanism 209 to operate over multiple (separate) DetNet member flows 210 of a DetNet compound flow. 212 DetNet Control Word A control word used for sequencing and 213 identifying duplaicate packets at the DetNet service 214 layer. 216 2.2. Abbreviations 218 The following abbreviations used in this document: 220 AC Attachment Circuit. 222 CE Customer Edge equipment. 224 CoS Class of Service. 226 CW Control Word. 228 d-CW DetNet Control Word. 230 DetNet Deterministic Networking. 232 DF DetNet Flow. 234 L2VPN Layer 2 Virtual Private Network. 236 LSR Label Switching Router. 238 MPLS Multiprotocol Label Switching. 240 MPLS-TP Multiprotocol Label Switching - Transport Profile. 242 MS-PW Multi-Segment PseudoWire (MS-PW). 244 NSP Native Service Processing. 246 OAM Operations, Administration, and Maintenance. 248 PE Provider Edge. 250 PREF Packet Replication and Elimination Function. 252 PSN Packet Switched Network. 254 PW PseudoWire. 256 QoS Quality of Service. 258 TSN Time-Sensitive Network. 260 3. Requirements language 262 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 263 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 264 document are to be interpreted as described in [RFC2119]. 266 4. DetNet data plane overview 268 This document describes how to use IP and/or MPLS to support a data 269 plane method of flow identification and packet formwarding over 270 layer-3. Two different cases are covered: (i) the inter-connect 271 scenario, in which IEEE802.1 TSN is routed over a layer-3 network 272 (i.e., to enlarge the layer-2 domain), and (ii) native connectivity 273 between DetNet-aware end systems. 275 Figure 1 illustrates how DetNet can provide services for IEEE 276 802.1TSN end systems over a DetNet enabled network. The edge nodes 277 insert and remove required DetNet data plane encapsulation. The 'X' 278 in the edge and relay nodes represents a potential DetNet flow packet 279 replication and elimination point. This conceptually parallels L2VPN 280 services, and could leverage existing related solutions as discussed 281 below. 283 TSN |<---------- End to End DetNet Service ------>| TSN 284 Service | Transit Transit | Service 285 TSN (AC) | |<-Tunnel->| |<-Tnl->| | (AC) TSN 286 End | V V 1 V V 2 V V | End 287 System | +--------+ +--------+ +--------+ | System 288 +---+ | | E1 |==========| R1 |=======| E2 | | +---+ 289 | |--|----|._X_....|..DetNet..|.._ _...|..DF3..|...._X_.|---|---| | 290 |CE1| | | \ | Flow 1 | X | | / | | |CE2| 291 | | | \_.|...DF2....|._/ \_..|..DF4..|._/ | | | 292 +---+ | |==========| |=======| | +---+ 293 ^ +--------+ +--------+ +--------+ ^ 294 | Edge Node Relay Node Edge Node | 295 | | 296 |<----- Emulated Time Sensitive Networking (TSN) Service ---->| 298 Figure 1: IEEE 802.1TSN over DetNet 300 Figure 2 illustrates how end to end MPLS-based DetNet service can be 301 provided. In this case, the end systems are able to send and receive 302 DetNet flows. For example, an end system sends data encapsulated in 303 MPLS. Like earlier the 'X' in the end systems, edge and relay nodes 304 represents potential DetNet flow packet replication and elimination 305 points. Here the relay nodes may change the underlying transport, 306 for example tunneling IP over MPLS, or simply interconnect network 307 segments. 309 DetNet DetNet 310 Service Transit Transit Service 311 DetNet | |<-Tnl->| |<-Tnl->| | DetNet 312 End | V 1 V V 2 V | End 313 System | +--------+ +--------+ +--------+ | System 314 +---+ | | R1 |=======| R2 |=======| R3 | | +---+ 315 | X...DFa...|._X_....|..DF1..|.__ ___.|..DF3..|...._X_.|.DFa..|.X | 316 |CE1|========| \ | | X | | / |======|CE2| 317 | | | | \_.|..DF2..|._/ \__.|..DF4..|._/ | | | | 318 +---+ | |=======| |=======| | +---+ 319 ^ +--------+ +--------+ +--------+ ^ 320 | Relay Node Relay Node Relay Node | 321 | | 322 |<--------------- End to End DetNet Service -------------->| 324 Figure 2: MPLS-Based Native DetNet 326 Figure 3 illustrates how end to end IP-based DetNet service can be 327 provided. In this case, the end systems are able to send and receive 328 DetNet flows. [Editor's note: TBD] 329 NOTE: This figures is TBD 331 DetNet DetNet 332 Service Transit Transit Service 333 DetNet | |<-Tnl->| |<-Tnl->| | DetNet 334 End | V 1 V V 2 V | End 335 System | +--------+ +--------+ +--------+ | System 336 +---+ | | R1 |=======| R2 |=======| R3 | | +---+ 337 | X...DFa...| | | | | | .|.X | 338 | H1|========| | | | | |======| H2| 339 | | | | | | | | | | | | 340 +---+ | |=======| |=======| | +---+ 341 ^ +--------+ +--------+ +--------+ ^ 342 | Relay Node Relay Node Relay Node | 343 | | 344 |<--------------- End to End DetNet Service -------------->| 346 Figure 3: IP-Based Native DetNet 348 4.1. DetNet data plane encapsulation requirements 350 Two major groups of scenarios can be distinguished which require flow 351 identification during transport: 353 1. DetNet function related scenarios: 355 * Congestion protection and latency control: usage of allocated 356 resources (queuing, policing, shaping). 358 * Explicit routes: select/apply the flow specific path. 360 * Service protection: recognize DetNet compound and member flows 361 for replication an elimination. 363 Comment #12 I am not sure whether the correct architectural 364 construct is flow or flow group. Flow suggests that sharing/ 365 aggregation is not allowed but whether this is allowed or not 366 is an application specific issue. 368 Discussion: Agree that a flow group would be a better 369 characterization. 371 Comment #13 I think that there needs to be some clarification as 372 to whether FG is is understood by the DN system exclusively or 373 whether there is an expectation that it is understood by the 374 underlay. 376 Discussion: Agree that more detail is needed here. DetNet aware 377 nodes need to understand flow groups. Underlay needs to be 378 aware of flow groups at the resource allocation level. 380 2. OAM function related scenarios: 382 * troubleshooting (e.g., identify misbehaving flows, etc.) 384 * recognize flow(s) for analytics (e.g., increase counters, 385 etc.) 387 * correlate events with flows (e.g., volume above threshold, 388 etc.) 390 * etc. 392 Each DetNet node (edge, relay and transit) use an internal/ 393 implementation specific local-ID of the DetNet-(compound)-flow in 394 order to accomplish its role during transport. Recognizing the 395 DetNet flow is more relaxed for edge and relay nodes, as they are 396 fully aware of both the DetNet service and transport layers. The 397 primary DetNet role of intermediate transport nodes is limited to 398 ensuring congestion protection and latency control for the above 399 listed DetNet functions. 401 The DetNet data plane allows for the aggregation of DetNet flows, 402 e.g., via MPLS hierarchical LSPs, to improved scaling. When DetNet 403 flows are aggregated, transit nodes may have limited ability to 404 provide service on per-flow DetNet identifiers. Therefore, 405 identifying each individual DetNet flow on a transit node may not be 406 achieved in some network scenarios, but DetNet service can still be 407 assured in these scenarios through resource allocation and control. 409 Comment #14 You could introduce the concept of a flow group 410 identified into the packet. You may also include a flow id at a 411 lower layer. 413 Discussion: Agree on the identification properties. Adding a 414 specific id into actual on-wire formats is not necessarily needed. 416 On each DetNet node dealing with DetNet flows, an internal local-ID 417 is assumed to determine what local operation a packet goes through. 418 Therefore, local-IDs has to be unique on each edge and relay nodes. 419 Local-ID is unambiguously bound to the DetNet flow. 421 4.2. Packet replication and elimination considerations 423 DetNet service layer introduces packet replication and elimination 424 functionality (PREF) for use in DetNet edge and relay node and end 425 system packet processing. PREF MAY be enabled in a DetNet node and 426 the required processing is only applied to packets with a positive 427 flow identification at the DetNet service layer. PREF utilizes a 428 sequence number carried within a DetNet flow packets. 430 At a DetNet node level the output of the PREF elimination function is 431 always a single packet. The output of the PREF replication function 432 at a DetNet node level is always one or more packets (i.e., 1:M 433 replication). The replicated packets MUST share the same d-CW i.e., 434 the sequence number is the same for each member flow of the compound 435 flow. The location and mechanism on the packet processing pipeline 436 used for replication is implementation specific. 438 The complex part of the DetNet PREF processing is tracking the 439 history of received packets for multiple DetNet member flows. These 440 ingress DetNet member flows (to a node) MUST have the same local-ID 441 if they belong to the same DetNet (compound) flow and share the same 442 sequence number counter and the history information. The location of 443 the packet elimination on the packet processing pipeline is 444 implementation specific. 446 4.3. Packet reordering considerations 448 DetNet service layer introduces also packet reordering functionality 449 for use in DetNet edge and relay node and end system packet 450 processing. The reordering functionality MAY be enabled in a DetNet 451 node. The reordeing functionality relies on a presence of sequence 452 numbers in a DetNet (compound) flows. The reordering processing is 453 only applied to packets with a positive flow identification at the 454 DetNet service layer. 456 5. DetNet encapsulation 458 5.1. End-system specific considerations 460 Data-flows requiring DetNet service are generated and terminated on 461 end-systems. Encapsulation depends on application and its 462 preferences. In a DetNet (or even a TSN) domain the DN (TSN) 463 functions use at most two flow parameters, namely Flow-ID and 464 Seq.Number. However, an application may exchange further flow 465 related parameters (e.g., time-stamp), which are not considered by DN 466 functions. 468 Two types of end-systems are distinguished: 470 o L3 (IP) end-system: application over L3 472 o L2 (Ethernet) end-system: application directly over L2 474 In case of Ethernet end-systems the application data is encapsulated 475 directly in L2. From the DN domain perspective no upper layer 476 protocols are visible. The Data-flow uses only Ethernet tag(s) and 477 further flow specific parameters (if needed) are hidden inside the 478 PDU. 480 The IP end-system scenario is different. Data-flows are encapsulated 481 directly in L3 (i.e., IP) and the application may use further upper 482 layer protocols (e.g., RTP). Many valid combinations exist, and it 483 may be application specific how the IP header fields are used. Also, 484 usage of further upper layer protocols depends on application 485 requirements (e.g., time-stamp). Some examples for encoding of Flow- 486 ID or Seq.Number attributes: IP address, IPv6-Flow-label, L4 ports, 487 RTP-header, etc. 489 As a general rule, DetNet domains MUST be capable to forward any 490 Data-flows and the DetNet domain MUST NOT intend to mandate end- 491 system encapsulation format. 493 Furthermore, no application-level-proxy function is envisioned inside 494 the DetNet domain, so end-systems peer with end-systems using the 495 same application encapsulation format (see figure below): 497 o L3 end-systems peer with L3 end-systems and 499 o L2 end-systems peer with L2 end-systems 500 +-----+ 501 | X | +-----+ 502 +-----+ | X | 503 | Eth | ________ +-----+ 504 +-----+ _____ / \ | Eth | 505 \ / \__/ \___ +-----+ 506 \ / \ / 507 0======== tunnel-1 ========0_ 508 | \ 509 \ | 510 0========= tunnel-2 =========0 511 / \ __/ \ 512 +-----+ \__ DetNet domain / \ 513 | X | \ __ / +-----+ 514 +-----+ \_______/ \_____/ | X | 515 | IP | +-----+ 516 +-----+ | IP | 517 +-----+ 519 Figure 4: End-systems and the DetNet domain 521 5.2. DetNet domain specific considerations 523 From connection type perspective three scenarios are distinguished: 525 1. Directly attached: end-system is directly connected to an edge 526 node 528 2. Indirectly attached: end-system is behind a (L2-TSN / L3-DetNet) 529 sub-net 531 3. DN integrated: end-system is part of the DetNet domain 533 L3 end-systems may use any of these connection types, however L2 end- 534 systems may use only the first two (directly or indirectly attached). 535 DetNet domain MUST allow communication between any end-systems of the 536 same type (L2-L2, L3-L3), independent of their connection type and 537 DetNet capability. However directly attached and indirectly attached 538 end-systems have no knowledge about the DetNet domain and its 539 encapsulation format at all. See the figure below for L3 end-system 540 scenarios. 542 ____+----+ 543 +----+ _____ / | ES3| 544 | ES1|____ / \__/ +----+___ 545 +----+ \ / \ 546 + | 547 ____ \ _/ 548 +----+ __/ \ +__ DetNet domain / 549 | ES2|____/ L2/L3 |___/ \ __ __/ 550 +----+ \_______/ \_______/ \___/ 552 Figure 5: Connection types of L3 end-systems 554 5.2.1. DetNet Bridging Service 556 The simplest DetNet service is to provide bridging (i.e., tunneling 557 for L2), where the connected hosts are in the same broadcast (BC) 558 domain. Forwarding over the DetNet domain is based on L2 (MAC) 559 addresses (i.e. dst-MAC), so L2 headers MUST be kept. For both IP 560 and MPLS PSN a DetNet specific tunnel encapsulation MUST be 561 introduced. 563 +------+ 564 | X | 565 +------+ +------+ 566 | X | | IP | 567 +------+ +------+ 568 End+system | L2 | | L2 | 569 +-----+======+-+======+--+======+-+======++ 570 DetNet tunnel | IP | | MPLS | 571 +------+ +------+ 572 | L2 | | L2 | 573 +------+ +------+ 575 Examples 577 +------+ +------+ 578 | X | | X | 579 +------+ +------+ +------+ +------+ 580 | X | | X | | IP | | IP | 581 +------+ +------+ +------+ +------+ 582 | L2 | | L2 | | L2 | | L2 | 583 ++======+-+======+--+======+-+======++ 584 | IP | | MPLS | | IP | | MPLS | 585 +------+ +------+ +------+ +------+ 586 | L2 | | L2 | | L2 | | L2 | 587 +------+ +------+ +------+ +------+ 589 Figure 6: Encapsulation format for DetNet Bridging 591 As shown on the figure both L2 and L3 end-systems can be served by 592 such a DetNet Bridging service. 594 5.2.2. DetNet Routing Service 596 DetNet Routing service provides routing, therefore available only for 597 L3 hosts that are in different BC domains. Forwarding over the 598 DetNet domain is based on L3 (IP) addresses (i.e. dst-IP). 600 5.2.2.1. MPLS PSN 602 In case of an MPLS PSN at the ingress/egress (i.e., PE nodes of 603 DetNet domain) the IP packets are encapsulated in MPLS. The data- 604 flow IP header MUST be preserved as-is. 606 +------+ +------+ 607 | X | | X | 608 +------+ +------+ 609 End+system | IP | | IP | 610 -----+------+-------+======+--- --+======+-- 611 DetNet | MPLS | | MPLS | 612 +------+ +------+ 613 | L2 | | L2 | 614 +------+ +------+ 616 Figure 7: Encapsulation format for DetNet Routing in MPLS PSN for L3 617 end-systems 619 5.2.2.2. IP PSN 621 In case of an IP PSN the same tunneling concept can be used as for an 622 MPLS PSN, but the tunnel is constructed by a new IP header (and 623 possible upper layer fields). The data-flow IP header MUST be 624 preserved as-is. 626 +------+ +------+ 627 | X | | X | 628 +======+ +------+ 629 End-system | IP | | IP | 630 -----+------+-------+======+--- --+======+-- 631 DetNet | IP | | IP | 632 +------+ +------+ 633 | L2 | | L2 | 634 +------+ +------+ 636 Figure 8: Encapsulation format for DetNet Routing in IP PSN for L3 637 end-systems 639 DetNet IP header contains the IP addresses of the ingress/egress PE 640 nodes of DetNet domain. The End-system IP header contains the IP 641 addresses of the end-systems. 643 Note: In case of IP PSN one may consider avoiding the additional IP 644 encapsulation, however there are many issues with such an approach. 645 First, the DetNet nodes MUST be able to extract from the IP header 646 (and maybe upper layers) the attributes required by DetNet functions 647 (i.e. Flow-ID, Seq.Number). The challenge is that encoding of those 648 attributes may be application specific, so DetNet nodes MUST be 649 prepared to handle all application specific formats. Second, adding 650 further fields (e.g., explicit path information) to an existing IP 651 header may be impossible (e.g., due to security/encryption). 653 Furthermore, DetNet domain IP-header format may collide with IP- 654 header format used by the source of a flow. Implementing such an 655 approach requires that source encapsulation is in-line with DetNet 656 domain encapsulation format, however we do not intend to mandate end- 657 systems' encapsulation format (see former text: As a general rule, 658 DetNet domains MUST be capable to forward any Data-flows and the 659 DetNet domain MUST NOT intend to mandate end-system encapsulation 660 format.). 662 5.2.2.3. Simplified IP Service 664 In this case there is no "tunneling" below the DetNet Service, but 665 the DetNet Service flows are mapped to each link / sub net using its 666 technology specific methods. The DetNet IP header containes the IP 667 address destination DetNet end system. The data-flow IP header MUST 668 be preserved as-is. 670 This solution provides end to end DetNet service consisting of 671 congestion protection and latency control and the rouse allocation 672 (queuing, policing, shaping) done using the underlying link / sub net 673 specific mechanisms. Compared to previously described DetNet routing 674 services, the service protections (packet replication and packet 675 emilination functions) and not provided end to end, but per 676 underlying layer-2 link / sub net. 678 +------+ +------+ 679 | X | | X | 680 +======+ +------+ 681 End-system | IP | | IP | 682 -----+------+-------+======+--- --+======+-- 683 DetNet |L2/SbN| |L2/SbN| 684 +------+ +------+ 686 Figure 9: Encapsulation of DetNet Routing in simplified IP service L3 687 end-systems 689 Note: the DetNet Service Flow MUST be mapped to the link / sub net 690 specific resources using an underlying system specific means. This 691 implies each DetNet aware node on path MUST look into the transported 692 DetNet Service Flow packet and utilize e.g., a five tuple to find out 693 the required mapping in a node. As noted earlier, the Service 694 Protection is done within each link / sub net independently using the 695 domain specific mechanisms (due the lack of a unified end to end 696 sequencing information that would be available for intermediate 697 nodes). If end to end service protection is desired that can be 698 implemented, for example, by the DetNet end systems using Layer-4 699 transport protocols or application protocols. However, these are out 700 of scope of this document. 702 [Editor's note: the service protection to be clarified further.] 704 5.3. DetNet Inter-Working Function (DN-IWF) 706 5.3.1. Networks with multiple technology segments 708 There are network scenarios, where the DetNet domain contains 709 multiple technology segments (IP, MPLS) and all those segments are 710 under the same administrative control. Furthermore, DetNet nodes may 711 be interconnected via TSN segments. 713 An important aspect of DetNet network design is placement of DetNet 714 functions across the domain. Designs based on segment-by-segment 715 optimization can provide only suboptimal solutions. In order to 716 achieve global optimum Inter-Working Functions (DN-IWF) can be placed 717 at segment border nodes, which stich together DetNet flows across 718 connected segments. 720 DN-IWF may ensure that flow attributes are correlated across segment 721 borders. For example, there are two DetNet functions which require 722 Seq.Numbers: (1) Elimination: removes duplications from flows and (2) 723 IOD: ensures in-order-delivery of packet in a flow. Stitching flows 724 together and correlating attributes means for example that 725 replication of packets can happen in one segment and elimination of 726 duplicates in a different one. 728 ______ 729 _____ / \__ 730 ____ / \__/ \___ ______ 731 +----+ __/ +======+ +==+ \ +----+ 732 |src |__/ Seg1 ) | | \ Seg3 \____| dst| 733 +----+ \_______+ \ Segment-2 | \+_____/ +----+ 734 \======+__ _+===/ 735 \ __ __/ 736 \_______/ \___/ 738 +------------+ 739 +------------------E----+ | | 740 +----+ | | +----R---+ | +----+ 741 |src |-------R +---+ | E----------+ dst| 742 +----+ | | E--------+ +----+ 743 +--------------R | 744 +-----------------+ 746 Figure 10: Optimal replication and elimination placement across 747 technology segments example 749 5.3.2. DN-IWF related considerations 751 The ultimate goal of DN-IWF is to (1) match and (2) translate segment 752 specific flow attributes. The DN-IWF ensures that segment specific 753 attributes comprise per domain unique attributes for the whole DetNet 754 domain. This characteristic can ensure that DetNet functions can be 755 based on per domain attributes and not per segment attributes. 757 The two DetNet specific attributes have the following 758 characteristics: 760 o Flow-ID: it is same in all packets of a flow 762 o Seq.Number: it is different packet-by-packet 764 For the Flow-ID the DN-IWF can implement a static mapping. The 765 situation is more complicated for Seq.Number as it is different 766 packet-by-packet, so it may need more sophisticated translation 767 unless its format is exactly the same in the two technology segments. 768 In this later case the DN-IWF can simple copy the Seq.Number field 769 between the tunneling encapsulation of the two technology segments. 771 In case of three technology segments (IP, MPLS and TSN) three DN-IWF 772 functions can be specified. In the rest of this section the focus is 773 on the (1) IP - MPLS network scenario. Note: the use-cases are out- 774 of-scope for (2) TSN - IP, (3) TSN - MPLS. Note2: incompatible 775 format of Seq.Number with TSN. 777 Simplest implementation of DN-IWF is provided if the flow attributes 778 have the same format. Such a common denominator of the tunnel 779 encapsulation format is the pseudowire encapsulation over both IP and 780 MPLS. 782 Placeholder 784 Figure 11: FIGURE Placeholder PW over X 786 6. MPLS-based DetNet data plane solution 788 6.1. DetNet specific packet fields 790 The DetNet data plane encapsulation MUST include two DetNet specific 791 information elements in each packet of a DetNet flow: (1) a flow 792 identification and (2) a sequence number. 794 The DetNet data plane encapsulation may consists further elements 795 used for overlay tunneling, to distinguish between DetNet member 796 flows of the same DetNet compound flow or to support OAM functions. 798 6.2. Data plane encapsulation 800 Figure 12 illustrates a DetNet data plane MPLS encapsulation. The 801 MPLS-based encapsulation of the DetNet flows is a good fit for the 802 Layer-2 interconnect deployment cases (see Figure 1). Furthermore, 803 end to end DetNet service i.e., native DetNet deployment (see 804 Figure 2) is also possible if DetNet end systems are capable of 805 initiating and termination MPLS encapsulated packets. Transport of 806 IP encapsulated DetNet flows, see Section 7, over MPLS-based DetNet 807 data plane is also possible. Interworking between PW- and IPv6-based 808 encapsulations is discussed further in Section 8.6. 810 The MPLS-based DetNet data plane encapsulation consists of: 812 o DetNet control word (d-CW) containing sequencing information for 813 packet replication and duplicate elimination purposes. There MUST 814 a separate sequence number space for each DetNet flow. 816 o DetNet Label that identifies a DetNet flow within a DetNet Edge or 817 a Relay node. The DetNet label MUST be at the bottom of the label 818 stack. 820 o An optional DetNet service lable (S-Label) that represents DetNet 821 Service LSP used between DetNet Egde and/or Relay nodes. One 822 possible use of an S-Label is to identify DetNet member flows used 823 to provide protection to a DetNet compound flow, perhaps even when 824 both LSPs appear on the same link for some reason. 826 One or more MPLS transport LSP label(s) (T-label) which may be a hop- 827 by-hop label used between LSR and MUST appear higher in the label 828 stack than S-labels. A top of stack T-label may be PHPed before 829 arriving at a DetNet node. In general T-labels should be considered 830 to be part of the underlying transport network rather the actual 831 DetNet data plane encapsulation. 833 DetNet MPLS-based encapsulation 835 +---------------------------------+ 836 | | 837 | DetNet Flow | 838 | Payload Packet | 839 | | 840 +---------------------------------+ <--\ 841 | DetNet Control Word | | 842 +---------------------------------+ +--> DetNet data plane 843 | S-Label | | MPLS encapsulation 844 +---------------------------------+ <--/ 845 | T-Label(s) | 846 +---------------------------------+ 847 | Data-Link | 848 +---------------------------------+ 849 | Physical | 850 +---------------------------------+ 852 Figure 12: Encapsulation of a DetNet flow in an MPLS(-TP) PSN 854 6.3. DetNet control word 856 A DetNet control word (d-CW) conforms to the Generic PW MPLS Control 857 Word (PWMCW) defined in [RFC4385] and is illustrated in Figure 13. 858 The upper nibble of the d-CW MUST be set to zero (0). The effective 859 sequence number bit length is between 0 and 28 bits, and configured 860 either by a control plane or manually for each DetNet flow. The 861 sequence number is aligned to the right (least significant bits) and 862 unused bits MUST be set to zero (0). Each DetNet flow MUST have its 863 own sequence number counter. The sequence number is incremented by 864 one for each new packet. 866 The d-CW MUST always be present in a packet. In a case the sequence 867 number is not used (e.g., for DetNet-t-flows) the control plane or 868 the manual configuration has to define zero (0) bit length seqeunce 869 number and the value of the sequence number MUST be set to zero (0). 871 0 1 2 3 872 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 873 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 874 |0 0 0 0| Sequence Number | 875 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 877 Figure 13: DetNet Control Word 879 6.4. Flow identification 881 DetNet flow identification at a DetNet service layer is realized by 882 an S-label. It maps a Detnet flow to a specific d-CW in a DetNet 883 node. The S-label used for flow identification MUST be bottom label 884 of the label stack for a DetNet-s- or DetNet-st-flow and MUST precede 885 the d-CW. 887 An S-label for a single DetNet flow does not need to be unique DetNet 888 domain wide. As long as two or more different DetNet flows do not 889 errorneously map to a same d-CW in a DetNet node the labels may vary. 891 6.5. Service layer considerations 893 [Editor's note: quite a bit of unfinished and old text in the 894 following sections.] 896 The edge and relay node internal procedures of the PREF are 897 implementation specific. The order of a packet elimination or 898 replication is out of scope in this specification. However, care 899 should be taken that the replication function does not actually 900 loopback packets as "replicas". Looped back packets include 901 artificial delay when the node that originally initiated the packet 902 receives it again. Also, looped back packets may make the network 903 condition to look healthier than it actually is (in some cases link 904 failures are not reflected properly because looped back packets make 905 the situation appear better than it actually is). 907 Comment #29: SB> There needs to be some text about preventing a node 908 ever receiving its own replicated packets. Indeed that would 909 suggest that the flow id should be changed and replication should 910 only take place on configured flow IDs. I have a feeling that 911 this would all be a lot safer if replication only happened at 912 ingress and we managed the diversity of the paths. 914 Discussion: Agree on hardening the loopback text considerations. 916 6.5.1. Edge node processing 918 TBD. 920 [Editor's note: Since we are not defining the inner workings and 921 implementation of the DetNet Egde node - rather only what goes in and 922 what comes out, and of course the on-wire details, then the figures 923 shown in the coming section would not need to detail the inner 924 architecture of a DetNet Node.] 926 +---------------------------------------+ 927 | DetNet Edge Node | 928 +---------------------------------------+ 929 | | | | 930 | | | | 931 Client AC | +---o <-------> o o<----------> 932 | Elim. | | | | 933 <---------->o <-------| | +-------------+ 934 | | | | | 935 | +---o <-------> o | 936 | | | o<----------> 937 | | +-> o | 938 +-------------+ | +-------------+ 939 | | | | | 940 Client AC | | Repl. | | | 941 <---------->o o <-----X-> o o<----------> 942 | | Elim. | | 943 +-------------+ +-------------+ 944 | | | | 945 Client AC | | | | 946 <---------->o o <-------> o o<----------> 947 | | | | 948 +---------------------------------------+ 950 Figure 14: DetNet Edge Node processing 952 An edge node participates to the packet replication and duplication 953 elimination. Required processing is done within an extended 954 forwarder function. In the case the native service processing (NSP) 955 is IEEE 802.1CB [IEEE8021CB] capable, the packet replication and 956 duplicate elimination MAY entirely be done in the NSP and bypassing 957 the DetNet flow encapsulation and logic entirely, and thus is able to 958 operate over unmodified implementation and deployment. The NSP 959 approach works only between edge nodes and cannot make use of relay 960 nodes (see Section 6.5.2). 962 Comment #31 SB> This would be a fine way to operate the PW system - 963 edge to edge. 965 Discussion: When it comes to use of NSPs, agree. Also for "island 966 interconnect" this is a fine. However, when there is a need to do 967 PREF in a middle, plain edge to edge is not enough. 969 The DetNet-aware extended forwarder selects the egress DetNet member 970 flow based on the DetNet forwarding rules. In both "normal AC" and 971 "Packet AC" cases there may be no DetNet encapsulation header 972 available yet as it is the case with relay nodes (see Section 6.5.2). 973 It is the responsibility of the extended forwarder within the edge 974 node to push the DetNet specific encapsulation (if not already 975 present) to the packet before forwarding it to the appropriate egress 976 DetNet member flow instance(s). 978 Comment #32 SB> I am not convinced of the wisdom of having a mid- 979 point node convert a flow into a DN flow, which is what you are 980 implying here. This seems like an ingress function. 982 Discussion: OK. The text here has issues and seems to mix relay and 983 edge. 985 The extended forwarder MAY copy the sequencing information from the 986 native DetNet packet into the DetNet sequence number field and vice 987 versa. If there is no existing sequencing information available in 988 the native packet or the forwarder chose not to copy it from the 989 native packet, then the extended forwarder MUST maintain a sequence 990 number counter for each DetNet flow (indexed by the DetNet flow 991 identification). 993 6.5.2. Relay node processing 995 TBD. 997 A DetNet Relay node participates to the packet replication and 998 duplication elimination. This processing is done within an extended 999 forwarder function. Whether an ingress DetNet member flow receives 1000 DetNet specific processing depends on how the forwarding is 1001 programmed. For some DetNet member flows the relay node can act as a 1002 normal relay node and for some apply the DetNet specific processing 1003 (i.e., PREF). 1005 Comment #34 SB> Again relay node is not a normal term, so am not 1006 sure what it does in the absence of a PREF function. 1008 Discussion: Relay node was a DetNet aware S-PE originally, which is 1009 not explicitly stated here anymore, thus slightly confusing text 1010 here. The text here needs to clarify the roles of PREF and 1011 switching functions. A DetNet relay is described in the 1012 architecture document. However, there is definitely room for 1013 termonilogy and text improvements. 1015 It is also possible to treat the relay node as a transit node, see 1016 Section 8.3. Again, this is entirely up to how the forwarding has 1017 been programmed. 1019 The DetNet-aware forwarder selects the egress DetNet member flow 1020 segment based on the flow identification. The mapping of ingress 1021 DetNet member flow segment to egress DetNet member flow segment may 1022 be statically or dynamically configured. Additionally the DetNet- 1023 aware forwarder does duplicate frame elimination based on the flow 1024 identification and the sequence number combination. The packet 1025 replication is also done within the DetNet-aware forwarder. During 1026 elimination and the replication process the sequence number of the 1027 DetNet member flow MUST be preserved and copied to the egress DetNet 1028 member flow. 1030 +---------------------------------------+ 1031 | DetNet Relay Node | 1032 Ingress +---------------------------------------+ 1033 | | | | Egress 1034 | o---------> o--+ Elim. | 1035 ----------->o | | +--------->o-----------> 1036 | | +-----> o--+ | 1037 Ingress +-------------+ | +-------------+ 1038 | | | | | Egress 1039 | | | | | 1040 ----------->o o --+ +-> o o-----------> 1041 | | | | | 1042 Ingress +-------------+ | +-------------+ 1043 | | | | | Egress 1044 | | Repl. | | | 1045 ----------->o o ------+-> o o-----------> 1046 | | | | 1047 Ingress +-------------+ +-------------+ 1048 | | | | Egress 1049 | | | | 1050 ----------->o o --------> o o-----------> 1051 | | | | 1052 +---------------------------------------+ 1054 Figure 15: DetNet Relay Node processing 1056 Comment #35 SB> Somewhere in the dp document there needs to be a 1057 note of the requirement for interfaces to do fast exchange of 1058 counter state, and a note to those planning the network and 1059 designing the control plane that they need to provide support for 1060 this. 1062 Discussion: We kinf of agree but also think the above exchange or 1063 synchronization of counter states is not in our scope to solve. 1065 6.5.3. End system processing 1067 TBD. 1069 6.6. Transport node considerations 1071 6.6.1. Congestion protection 1073 TBD. 1075 6.6.2. Explicit routes 1077 TBD. 1079 7. IPv6-based DetNet data plane solution 1081 7.1. Data plane encapsulation 1083 Figure 16 illustrates a DetNet native IPv6 encapsulation. The native 1084 IPv6 encapsulation is meant for end to end Detnet service use cases, 1085 where the end stations are DetNet-aware (see Figure 3). Technically 1086 it is possible to use the IPv6 encapsulation to tunnel any traffic 1087 over a DetNet enabled network, which would make native IPv6 1088 encapsulation also a valid data plane choice for an interconnect use 1089 case (see Figure 1). 1091 The native IPv6-based DetNet data plane encapsulation consists of: 1093 o IPv6 header as the transport protocol. 1095 o IPv6 header Flow Label that is used to help to identify a DetNet 1096 flow (i.e., roughly an equivalent to an S-Label for the MPLS 1097 encapsulation). A Flow Label together with the IPv6 source 1098 address uniquely identifies a DetNet flow. 1100 Comment #21 SB> Have we validated that it is unconditionally safe to 1101 make this assumption about the use of the FL? 1103 Discussion: RFC6437 does not restrict such use and DetNet DP 1104 solution can always define their own use of flow label. It should 1105 be noted that a DetNet aware node will always contain new code and 1106 is not a load balancer. 1108 o Zero, one or two DetNet Destination Options containing sequencing 1109 information for packet replication and duplicate elimination 1110 function (PREF), and/or packet reordering purposes. The DetNet 1111 Destination Option is equivalent to the DetNet Control Word. If 1112 PREF or packet reordering is not needed for the DetNet flow then 1113 no DetNet Destination Option is inserted into the IPv6 header. 1115 A DetNet-aware end station (a host) or an intermediate Detnet node 1116 initiating an (or adding a tunnelling) IPv6 packet is responsible for 1117 setting the Flow Label, adding the optional DetNet Destination 1118 Option(s) for DetNet-s- or DetNet-st-flows, and possibly adding a 1119 routing header such as the segment routing option (e.g., for pre- 1120 defined paths [I-D.ietf-6man-segment-routing-header]). If a routing 1121 header is inserted into the IPv6 packet for DetNet-s- or DetNet-st- 1122 flows then a second instance of the DetNet Destination Option MUST be 1123 added before the routing header (see Section 4.1 of [RFC8200]). 1125 A DetNet-aware end station (a host) or an intermediate node receiving 1126 an IPv6 packet destined to it and containing a DetNet Destination 1127 Option does the appropriate processing of the packet. This may 1128 involve packet duplication and elimination (PREF processing), 1129 terminating a tunnel or delivering the packet to the upper layers/ 1130 Applications. 1132 +---------------------------------+ 1133 | | 1134 | DetNet Flow | 1135 | Payload | 1136 | | 1137 /---------------------------------\ 1138 H Optional DetNet DstOpt Hdr H 1139 \---------------------------------/ 1140 | IPv6 header | 1141 | (with set Flow label) | 1142 +---------------------------------+ 1144 Figure 16: Encapsulation of a native IPv6 DetNet-s- or DetNet-st-flow 1145 without a routing header 1147 Figure 17 illustrates an IPv6 packet for the case where a routing 1148 header has been added into the packet by a DetNet-aware end system 1149 (again assuming DetNet-s- or DetNet-st-flows). Note that the use of 1150 routing header such as the one with the segment routing option is not 1151 mandatory for explicit routes. Similar functionality can be arranged 1152 using other means as well (e.g., using policy routing or layer-2 1153 means). 1155 +---------------------------------+ 1156 | | 1157 | DetNet Flow | 1158 | Payload | 1159 | | 1160 /---------------------------------\ 1161 H DetNet DstOpt Hdr H 1162 \---------------------------------/ 1163 | Routing header | 1164 /---------------------------------\ 1165 H DetNet DstOpt Hdr H 1166 \---------------------------------/ 1167 | IPv6 header | 1168 | (with set Flow label) | 1169 +---------------------------------+ 1171 Figure 17: Encapsulation of a native IPv6 DetNet-s- or DetNet-st- 1172 flows with routing header 1174 IPv6 extension headers can only be inserted by a node that initiated 1175 the IPv6 packet. IPv6 extension headers, except for the Hop-by-Hop 1176 Option headers, can only be processed by an IPv6 node that is 1177 identified by the Destination Address field of the IPv6 header (see 1178 Section 0 of [RFC8200]. Therefore, if a DetNet-aware end system only 1179 inserted the DetNet Destination Option into the IPv6 but e.g., a 1180 DetNet Edge node is configured to enforce an explicit route for the 1181 IPv6 packet using a source routing header, then it has no other 1182 possibility than add an outer tunneling IPv6 header with required 1183 extension headers in it. The processing of IPv6 packets in a DetNet 1184 Edge node is discussed further in Section 7.4.1. 1186 7.2. DetNet destination option 1188 A DetNet flow must carry sequencing information for packet 1189 replication and elimination function (PREF) purposes. This document 1190 specifies a new IPv6 Destination Option: the DetNet Destination 1191 Option for that purpose. The format of the option is illustrated in 1192 Figure 18. 1194 0 1 2 3 1195 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1196 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1197 | TBD1 | 4 | 0 | 28 bit sequence 1198 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1199 number | 1200 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1202 Figure 18: DetNet Destination Option 1204 The Option Type for the DetNet Destination Option is set to TBD1. 1205 [To be removed from the final version of the document: The Option 1206 Type MUST have the two most significant bits set to 10b] 1208 If an IPv6 packet gets dropped due the DetNet Service layer 1209 processing based on the DetNet Destination Option an ICMPv6 packet of 1210 any type MUST NOT be sent back to the source of the packet. 1212 7.3. Flow identification 1214 The DetNet flow identification is based on the IPv6 Flow Label and 1215 the source address combination. The two fields uniquelly identify 1216 the end to end native IPv6 encapsulated DetNet flow. Obviously, the 1217 identification fails if any intermediate node modifies either the 1218 source address or the Flow Label. 1220 Comment #27 SB> See earlier. If there are enough IPv6 addresses to 1221 address video fragments, why not DN flows? Then this problem goes 1222 away. 1224 Discussion: See the earlier comment #25 discussion. If nodes get 1225 their addressies via DHCPv6 basically ruins this mechanism. Also 1226 the assumption for this to work is that the node has a full /64 to 1227 use, which is not always the case. Otherwise the idea is just 1228 fine. 1230 7.4. Service layer considerations 1232 [Editor's note: this section is TBD. It will detail the PREF 1233 functionality.] 1235 o PREF - requires both flow identification and sequence numbering. 1237 o Packet reordeing - requires both flow identification and sequence 1238 numbering. 1240 A DetNet service layer processing can be done at each DetNet node 1241 that matches the IPv6 header's Destination Address. Then, if the 1242 DetNet flow identification provides a positive match for the DetNet 1243 flow that the node has a service layer state installed e.g., for PREF 1244 or packet reordering purposes, further service layer processing takes 1245 place. In a case of PREF or packet reordering that means processing 1246 the DetNet Destination Option for the identified DetNet flow. 1248 7.4.1. Edge node processing 1250 [Editor's note: This is the start of the IPv6 handling text - there 1251 are errors and bad language. The founding assumption is the use of 1252 source routing when intermediate nodes (relays/edges) need to modify 1253 packets. This is due the text in RFC8200 and the fact that without 1254 hph options only routing+dsthdr is usable with intermediates under 1255 strict RFC8200.. ] 1257 [Editor's note: Regrading the source routing and the "example" SRv6 1258 approach. Current text is based on the assumption that intermediates 1259 cannot add/delete extension headers such as the SRv6. That said 1260 adding adding a header implies adding a tunneling outer IPv6 header 1261 and deleting a header implies a tunnel decapsulation. This is not 1262 probably desired due to the involved overhead and to be discussed 1263 whether it is possible/acceptable to just "process" the Application 1264 flow packets.] 1266 For a DetNet Edge node there are several scenarios that involve 1267 modifications to the DetNet flow IPv6 packets. The assumption is 1268 that a DetNet-aware end system has always set the IPv6 header flow 1269 label properly for the flow identification purposes. A DetNet- or 1270 DetNet-t-flow does not include the DetNet Destination Option. 1271 Following cases have been identified: 1273 1. A DetNet App-flow or a DetNet-t-flow packet arrives at an ingress 1274 DetNet Edge node and DetNet service layer functions are done only 1275 at DetNet Edge nodes. Possible explicit routes between edge 1276 nodes are arranged by other than IPv6 specific means. 1278 2. A DetNet App-flow or a DetNet-t-flow packet arrives at an ingress 1279 DetNet Edge node and multiple DetNet Relay nodes may process 1280 DetNet flow packets before reaching an egress DetNet Edge node. 1281 Explicit routes between edge nodes has to be arranged by IPv6 1282 specific means. 1284 3. A DetNet-s- or a DetNet-st-flow packet arrives at an ingress 1285 DetNet Edge node and DetNet service layer functions are done only 1286 at DetNet Edge nodes. Possible explicit routes between edge 1287 nodes are arranged by other than IPv6 specific means. 1289 4. A DetNet-s- or a DetNet-st-flow packet arrives at an ingress 1290 DetNet Edge node and multiple DetNet Relay nodes may process 1291 DetNet flow packets before reaching an egress DetNet Edge node. 1292 Explicit routes between edge nodes has to be arranged by IPv6 1293 specific means. 1295 A generic DetNet IPv6 encapsulation for a DetNet flow packet between 1296 DetNet Edge nodes is shown in Figure 19. Essentially every time an 1297 igress DetNet Edge node has to insert something into the DetNet flow 1298 packet it has to add an outer tunneling IPv6 header, which then 1299 contain possible additional extension headers. 1301 +---------------------------------+ 1302 | | 1303 | DetNet Flow | 1304 | Payload | 1305 | | 1306 +---------------------------------+ 1307 | Optional DetNet DstOpt Hdr (1) | 1308 +---------------------------------+ 1309 | Inner IPv6 header | 1310 | (with set Flow label) (1) | 1311 +---------------------------------+ <--+ 1312 | Optional Routing header | | 1313 +---------------------------------+ | 1314 | Optional DetNet DstOpt Hdr (2) | +-- Added/removed by 1315 +---------------------------------+ | DetNet Edge node 1316 | Outer IPv6 header | | 1317 | (with set Flow label) (2) | | 1318 +---------------------------------+ <--+ 1320 Figure 19: Encapsulation of a DetNet-flow IPv6 packet at the DetNet 1321 Edge node 1323 7.4.1.1. Ingress DetNet Edge node processing 1325 Case 1) MAY require an addition of the DetNet Destination Option if 1326 packet reordering is requested at the egress DetNet Edge node. 1327 Otherwise, no modifications except rewriting the IPv6 header flow 1328 label to the packet is done. If modifications are required then: 1330 o The outer IPv6 header is added with the Source Address set to the 1331 ingress DetNet Edge node address and the Destination Address set 1332 to the egress DetNet Edge node address. 1334 o The flow label of the outer IPv6 header SHOULD be set to a value 1335 maintained by the edge node. 1337 o The DetNet Destination Option with the edge node managed per 1338 DetNet flow sequence number value is inserted into the outer IPv6 1339 header. 1341 Case 2) requires an addition of the DetNet Destination Option unless 1342 neither packet reordeing or PREF is enable at any DetNet Edge/Relay 1343 node. A source routing header has to be added for the explicit route 1344 purposes. An example of the source routing header is the Segment 1345 Routing header. The following modifications to DetNet flow IPv6 1346 packets are required: 1348 o An outer IPv6 header is added with the Source Address set to the 1349 ingress DetNet Edge node address and the Destination Address set 1350 to the egress DetNet Edge node address. 1352 o The flow label of the outer IPv6 header SHOULD be set to a value 1353 maintained by the edge node. 1355 o The DetNet Destination Option with the edge node managed per 1356 DetNet flow sequence number value MAY be inserted into the outer 1357 IPv6 header. 1359 o A source routing header with addresses of those DetNet Relay nodes 1360 that must be traversed is inserted into the outer IPv6 header. 1362 Case 3) ...[Editor's note: is it OK if the sequece number added here 1363 by the edge node has only local significance between the edge nodes 1364 and not end to end between end systems? ] 1366 Case 4) ... 1368 7.4.1.2. Engress DetNet Edge node processing 1370 7.4.2. Relay node processing 1372 TBD. 1374 7.4.3. End system processing 1376 TBD. 1378 7.5. Transport node processing 1380 7.5.1. Congestion protection 1381 7.5.2. Explicit routes 1383 8. Other DetNet data plane considerations 1385 8.1. Class of Service 1387 Class and quality of service, i.e., CoS and QoS, are terms that are 1388 often used interchangeably and confused. In the context of DetNet, 1389 CoS is used to refer to mechanisms that provide traffic forwarding 1390 treatment based on aggregate group basis and QoS is used to refer to 1391 mechanisms that provide traffic forwarding treatment based on a 1392 specific DetNet flow basis. Examples of existing network level CoS 1393 mechanisms include DiffServ which is enabled by IP header 1394 differentiated services code point (DSCP) field [RFC2474] and MPLS 1395 label traffic class field [RFC5462], and at Layer-2, by IEEE 802.1p 1396 priority code point (PCP). 1398 CoS for DetNet flows carried in PWs and MPLS is provided using the 1399 existing MPLS Differentiated Services (DiffServ) architecture 1400 [RFC3270]. Both E-LSP and L-LSP MPLS DiffServ modes MAY be used to 1401 support DetNet flows. The Traffic Class field (formerly the EXP 1402 field) of an MPLS label follows the definition of [RFC5462] and 1403 [RFC3270]. The Uniform, Pipe, and Short Pipe DiffServ tunneling and 1404 TTL processing models are described in [RFC3270] and [RFC3443] and 1405 MAY be used for MPLS LSPs supporting DetNet flows. MPLS ECN MAY also 1406 be used as defined in ECN [RFC5129] and updated by [RFC5462]. 1408 CoS for DetNet flows carried in IPv6 is provided using the standard 1409 differentiated services code point (DSCP) field [RFC2474] and related 1410 mechanisms. The 2-bit explicit congestion notification (ECN) 1411 [RFC3168] field MAY also be used. 1413 One additional consideration for DetNet nodes which support CoS 1414 services is that they MUST ensure that the CoS service classes do not 1415 impact the congestion protection and latency control mechanisms used 1416 to provide DetNet QoS. This requirement is similar to requirement 1417 for MPLS LSRs to that CoS LSPs do not impact the resources allocated 1418 to TE LSPs via [RFC3473]. 1420 8.2. Quality of Service 1422 Quality of Service (QoS) mechanisms for flow specific traffic 1423 treatment typically includes a guarantee/agreement for the service, 1424 and allocation of resources to support the service. Example QoS 1425 mechanisms include discrete resource allocation, admission control, 1426 flow identification and isolation, and sometimes path control, 1427 traffic protection, shaping, policing and remarking. Example 1428 protocols that support QoS control include Resource ReSerVation 1429 Protocol (RSVP) [RFC2205] (RSVP) and RSVP-TE [RFC3209] and [RFC3473]. 1430 The existing MPLS mechanisms defined to support CoS [RFC3270] can 1431 also be used to reserve resources for specific traffic classes. 1433 In addition to explicit routes, and packet replication and 1434 elimination, described in Section 6 above, DetNet provides zero 1435 congestion loss and bounded latency and jitter. As described in 1436 [I-D.ietf-detnet-architecture], there are different mechanisms that 1437 maybe used separately or in combination to deliver a zero congestion 1438 loss service. These mechanisms are provided by the either the MPLS 1439 or IP layers, and may be combined with the mechanisms defined by the 1440 underlying network layer such as 802.1TSN. 1442 A baseline set of QoS capabilities for DetNet flows carried in PWs 1443 and MPLS can provided by MPLS with Traffic Engineering (MPLS-TE) 1444 [RFC3209] and [RFC3473]. TE LSPs can also support explicit routes 1445 (path pinning). Current service definitions for packet TE LSPs can 1446 be found in "Specification of the Controlled Load Quality of 1447 Service", [RFC2211], "Specification of Guaranteed Quality of 1448 Service", [RFC2212], and "Ethernet Traffic Parameters", [RFC6003]. 1449 Additional service definitions are expected in future documents to 1450 support the full range of DetNet services. In all cases, the 1451 existing label-based marking mechanisms defined for TE-LSPs and even 1452 E-LSPs are use to support the identification of flows requiring 1453 DetNet QoS. 1455 QoS for DetNet flows carried in IPv6 MUST be provided locally by the 1456 DetNet-aware hosts and routers supporting DetNet flows. Such support 1457 will leverage the underlying network layer such as 802.1TSN. The 1458 traffic control mechanisms used to deliver QoS for IP encapsulated 1459 DetNet flows are expected to be defined in a future document. From 1460 an encapsulation perspective, and as defined in Section 7, the 1461 combination of the Flow Label together with the IP source address 1462 uniquely identifies a DetNet flow. 1464 Packets that are marked with a DetNet Class of Service value, but 1465 that have not been the subject of a completed reservation, can 1466 disrupt the QoS offered to properly reserved DetNet flows by using 1467 resources allocated to the reserved flows. Therefore, the network 1468 nodes of a DetNet network MUST: 1470 o Defend the DetNet QoS by discarding or remarking (to a non-DetNet 1471 CoS) packets received that are not the subject of a completed 1472 reservation. 1474 o Not use a DetNet reserved resource, e.g. a queue or shaper 1475 reserved for DetNet flows, for any packet that does not carry a 1476 DetNet Class of Service marker. 1478 8.3. Cross-DetNet flow resource aggregation 1480 The ability to aggregate individual flows, and their associated 1481 resource control, into a larger aggregate is an important technique 1482 for improving scaling of control in the data, management and control 1483 planes. This document identifies the traffic identification related 1484 aspects of aggregation of DetNet flows. The resource control and 1485 management aspects of aggregation (including the queuing/shaping/ 1486 policing implications) will be covered in other documents. The data 1487 plane implications of aggregation are independent for PW/MPLS and IP 1488 encapsulated DetNet flows. 1490 DetNet flows transported via MPLS can leverage MPLS-TE's existing 1491 support for hierarchical LSPs (H-LSPs), see [RFC4206]. H-LSPs are 1492 typically used to aggregate control and resources, they may also be 1493 used to provide OAM or protection for the aggregated LSPs. Arbitrary 1494 levels of aggregation naturally falls out of the definition for 1495 hierarchy and the MPLS label stack [RFC3032]. DetNet nodes which 1496 support aggregation (LSP hierarchy) map one or more LSPs (labels) 1497 into and from an H-LSP. Both carried LSPs and H-LSPs may or may not 1498 use the TC field, i.e., L-LSPs or E-LSPs. Such nodes will need to 1499 ensure that traffic from aggregated LSPs are placed (shaped/policed/ 1500 enqueued) onto the H-LSPs in a fashion that ensures the required 1501 DetNet service is preserved. 1503 DetNet flows transported via IP have more limited aggregation 1504 options, due to the available traffic flow identification fields of 1505 the IP solution. One available approach is to manage the resources 1506 associated with a DSCP identified traffic class and to map (remark) 1507 individually controlled DetNet flows onto that traffic class. This 1508 approach also requires that nodes support aggregation ensure that 1509 traffic from aggregated LSPs are placed (shaped/policed/enqueued) in 1510 a fashion that ensures the required DetNet service is preserved. 1512 Comment #38 SB> I am sure we can do better than this with SR, or the 1513 use of routing techniques that map certain addresses to certain 1514 paths. 1516 Discussion: -- 1518 In both the MPLS and IP cases, additional details of the traffic 1519 control capabilities needed at a DetNet-aware node may be covered in 1520 the new service descriptions mentioned above or in separate future 1521 documents. Management and control plane mechanisms will also need to 1522 ensure that the service required on the aggregate flow (H-LSP or 1523 DSCP) are provided, which may include the discarding or remarking 1524 mentioned in the previous sections. 1526 8.4. Bidirectional traffic 1528 Some DetNet applications generate bidirectional traffic. Using MPLS 1529 definitions [RFC5654] there are associated bidirectional flows, and 1530 co-routed bidirectional flows. MPLS defines a point-to-point 1531 associated bidirectional LSP as consisting of two unidirectional 1532 point-to-point LSPs, one from A to B and the other from B to A, which 1533 are regarded as providing a single logical bidirectional transport 1534 path. This would be analogous of standard IP routing, or PWs running 1535 over two reciprocal unidirection LSPs. MPLS defines a point-to-point 1536 co-routed bidirectional LSP as an associated bidirectional LSP which 1537 satisfies the additional constraint that its two unidirectional 1538 component LSPs follow the same path (in terms of both nodes and 1539 links) in both directions. An important property of co-routed 1540 bidirectional LSPs is that their unidirectional component LSPs share 1541 fate. In both types of bidirectional LSPs, resource allocations may 1542 differ in each direction. The concepts of associated bidirectional 1543 flows and co-routed bidirectional flows can be applied to DetNet 1544 flows as well whether IPv6 or MPLS is used. 1546 While the IPv6 and MPLS data planes must support bidirectional DetNet 1547 flows, there are no special bidirectional features with respect to 1548 the data plane other than need for the two directions take the same 1549 paths. Fate sharing and associated vs co-routed bidirectional flows 1550 can be managed at the control level. Note, that there is no stated 1551 requirement for bidirectional DetNet flows to be supported using the 1552 same IPv6 Flow Labels or MPLS Labels in each direction. Control 1553 mechanisms will need to support such bidirectional flows for both 1554 IPv6 and MPLS, but such mechanisms are out of scope of this document. 1555 An example control plane solution for MPLS can be found in [RFC7551]. 1557 8.5. Layer 2 addressing and QoS Considerations 1559 The Time-Sensitive Networking (TSN) Task Group of the IEEE 802.1 1560 Working Group have defined (and are defining) a number of amendments 1561 to IEEE 802.1Q [IEEE8021Q] that provide zero congestion loss and 1562 bounded latency in bridged networks. IEEE 802.1CB [IEEE8021CB] 1563 defines packet replication and elimination functions that should 1564 prove both compatible with and useful to, DetNet networks. 1566 As is the case for DetNet, a Layer 2 network node such as a bridge 1567 may need to identify the specific DetNet flow to which a packet 1568 belongs in order to provide the TSN/DetNet QoS for that packet. It 1569 also will likely need a CoS marking, such as the priority field of an 1570 IEEE Std 802.1Q VLAN tag, to give the packet proper service. 1572 Although the flow identification methods described in IEEE 802.1CB 1573 [IEEE8021CB] are flexible, and in fact, include IP 5-tuple 1574 identification methods, the baseline TSN standards assume that every 1575 Ethernet frame belonging to a TSN stream (i.e. DetNet flow) carries 1576 a multicast destination MAC address that is unique to that flow 1577 within the bridged network over which it is carried. Furthermore, 1578 IEEE 802.1CB [IEEE8021CB] describes three methods by which a packet 1579 sequence number can be encoded in an Ethernet frame. 1581 Ensuring that the proper Ethernet VLAN tag priority and destination 1582 MAC address are used on a DetNet/TSN packet may require further 1583 clarification of the customary L2/L3 transformations carried out by 1584 routers and edge label switches. Edge nodes may also have to move 1585 sequence number fields among Layer 2, PW, and IPv6 encapsulations. 1587 8.6. Interworking between MPLS- and IPv6-based encapsulations 1589 [Editor's note: add considerations for interworking between MPLS- 1590 based and native IPv6-based DetNet encapsuations.] 1592 8.7. IPv4 considerations 1594 [Editor's note: The fact is that there are and will be deployments 1595 using IPv4. Neglecting it entirely is not feasible.] 1597 9. Time synchronization 1599 Comment #39 SB> This section should point the reader to RFC8169 1600 (residence time in MPLS n/w. We need to consider if we need to 1601 introduce the same concept in IP. 1603 Discussion: Agree. For IP we could reference to PTPv2 or v3 over 1604 UDP/IP, since it measures residence time among other things. 1606 [Editor's note: describe a bit of issues and deployment 1607 considerations related to time-synchronization within DetNet. Refer 1608 to DT discussion and the slides that summarize different approaches 1609 and rough synchronization performance numbers. Finally, scope time- 1610 synchronization solution outside data plane.] 1612 When DetNet is used, there is an underlying assumption that the 1613 applicaton(s) require clock synchronization such as the Precision 1614 Time Protocol (PTP) [IEEE1588]. The relay nodes may or may not 1615 utilize clock synchronization in order to provide zero congestion 1616 loss and controlled latency delivery. In either case, there are a 1617 few possible approaches of how synchronization protocol packets are 1618 forwarded and handled by the network: 1620 o PTP packets can be sent either as DetNet flows or as high-priority 1621 best effort packets. Using DetNet for PTP packets requires 1622 careful consideration to prevent unwanted interactions between 1623 clock-synchronized network nodes and the packets that synchronize 1624 the clocks. 1626 o PTP packets are sent as a normal DetNet flow through network nodes 1627 that are not time-synchronized: in this approach PTP traffic is 1628 forwarded as a DetNet flow, and as such it is forwarded in a way 1629 that allows a low delay variation. However, since intermediate 1630 nodes do not take part in the synchronization protocol, this 1631 approach provides a relatively low degree of accuracy. 1633 o PTP with on-path support: in this approach PTP packets are sent as 1634 ordinary or as DetNet flows, and intermediate nodes take part in 1635 the protocol as Transparent Clocks or Boundary Clocks [IEEE1588]. 1636 The on-path PTP support by intermediate nodes provides a higher 1637 degree of accuracy than the previous approach. The actual 1638 accuracy depends on whether all intermediate nodes are PTP- 1639 capable, or only a subset of them. 1641 o Time-as-a-service: in this approach accurate time is provided as- 1642 a-service to the DetNet source and destination, as well as the 1643 intermediate nodes. Since traffic between the source and 1644 destination is sent over a provider network, if the provider 1645 supports time-as-a-service, then accurate time can be provided to 1646 both the source and the destination of DetNet traffic. This 1647 approach can potentially provide the highest degree of accuracy. 1649 It is expected that the latter approach will be the most common one, 1650 as it provides the highest degree of accuracy, and creates a layer 1651 separation between the DetNet data and the synchronization service. 1653 It should be noted that in all four approaches it is not recommended 1654 to use replication and elimination for synchronization packets; the 1655 replication/elimination approach may in some cases reduce the 1656 synchronization accuracy, since the observed path delay will be 1657 bivalent. 1659 Comment #40 SB> I am not sure why we sould not use PREP. We should 1660 explain to the reader. 1662 Discussion: Agree that a this can be opened a bit more in detail. 1663 The issue is explained briefly in the last sentence but it could 1664 be more clear. 1666 10. Management and control considerations 1668 [Editor's note: This section needs to be different for MPLS and IPv6 1669 solutions. Most solutions are technology dependant,] 1671 While management plane and control planes are traditionally 1672 considered separately, from the Data Plane perspective there is no 1673 practical difference based on the origin of flow provisioning 1674 information. This document therefore does not distinguish between 1675 information provided by a control plane protocol, e.g., RSVP-TE 1676 [RFC3209] and [RFC3473], or by a network management mechanisms, e.g., 1677 RestConf [RFC8040] and YANG [RFC7950]. 1679 [Editor's note: This section is a work in progress. discuss here 1680 what kind of enhancements are needed for DetNet and specifically for 1681 PREF and DetNet zero congest loss and latency control. Need to cover 1682 both traffic control (queuing) and connection control (control 1683 plane).] 1685 10.1. MPLS-based data plane 1687 10.1.1. S-Label assignment and distribution 1689 [Editor's note: Outdated and MPLS specific.. and needs more work.] 1691 The DetNet S-Label distribution follows the same mechanisms specified 1692 for XYZ . The details of the control plane protocol solution required 1693 for the label distribution and the management of the label number 1694 space are out of scope of this document. 1696 10.1.2. Explicit routes 1698 [Editor's note: Outdated.. and needs more work.] 1700 [TBD: based on MPLS TE and possibly IPv6 SR] 1702 10.2. IPv6-based data plane 1704 10.2.1. Flow Label assignment and distribution 1706 [Editor's note: Outdated and IPv6 Specific.. and needs more work.] 1708 The IPv6 Flow Label distribution and the label number space are out 1709 of scope of this document. However, it should be noted that the 1710 combination of the IPv6 source address and the IPv6 Flow Label is 1711 assumed to be unique within the DetNet-enabled network. Therefore, 1712 as long as each node is able to assign unique Flow Labels for the 1713 source address(es) it is using the DetNet-enabled network wide flow 1714 identification uniqueness is guaranteed. 1716 10.2.2. Explicit routes 1718 [Editor's note: Outdated.. and needs more work.] 1720 [TBD: What we have there for IPv6 and explicit routes] 1722 10.3. Packet replication and elimination 1724 [Editor's note: Outdated and at the functional level technology 1725 independent.. but needs more work.] 1727 The control plane protocol solution required for managing the PREF 1728 processing is outside the scope of this document. 1730 10.4. Congestion protection and latency control 1732 [TBD] 1734 10.5. Flow aggregation control 1736 [TBD] 1738 11. Security considerations 1740 The security considerations of DetNet in general are discussed in 1741 [I-D.ietf-detnet-architecture] and [I-D.sdt-detnet-security]. Other 1742 security considerations will be added in a future version of this 1743 draft. 1745 12. IANA considerations 1747 TBD. 1749 13. Acknowledgements 1751 The author(s) ACK and NACK. 1753 The following people were part of the DetNet Data Plane Solution 1754 Design Team: 1756 Jouni Korhonen 1758 Janos Farkas 1760 Norman Finn 1761 Balazs Varga 1763 Loa Andersson 1765 Tal Mizrahi 1767 David Mozes 1769 Yuanlong Jiang 1771 Carlos J. Bernardos 1773 The DetNet chairs serving during the DetNet Data Plane Solution 1774 Design Team: 1776 Lou Berger 1778 Pat Thaler 1780 14. References 1782 14.1. Normative references 1784 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1785 Requirement Levels", BCP 14, RFC 2119, 1786 DOI 10.17487/RFC2119, March 1997, 1787 . 1789 [RFC2211] Wroclawski, J., "Specification of the Controlled-Load 1790 Network Element Service", RFC 2211, DOI 10.17487/RFC2211, 1791 September 1997, . 1793 [RFC2212] Shenker, S., Partridge, C., and R. Guerin, "Specification 1794 of Guaranteed Quality of Service", RFC 2212, 1795 DOI 10.17487/RFC2212, September 1997, 1796 . 1798 [RFC2474] Nichols, K., Blake, S., Baker, F., and D. Black, 1799 "Definition of the Differentiated Services Field (DS 1800 Field) in the IPv4 and IPv6 Headers", RFC 2474, 1801 DOI 10.17487/RFC2474, December 1998, 1802 . 1804 [RFC3032] Rosen, E., Tappan, D., Fedorkow, G., Rekhter, Y., 1805 Farinacci, D., Li, T., and A. Conta, "MPLS Label Stack 1806 Encoding", RFC 3032, DOI 10.17487/RFC3032, January 2001, 1807 . 1809 [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition 1810 of Explicit Congestion Notification (ECN) to IP", 1811 RFC 3168, DOI 10.17487/RFC3168, September 2001, 1812 . 1814 [RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V., 1815 and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP 1816 Tunnels", RFC 3209, DOI 10.17487/RFC3209, December 2001, 1817 . 1819 [RFC3270] Le Faucheur, F., Wu, L., Davie, B., Davari, S., Vaananen, 1820 P., Krishnan, R., Cheval, P., and J. Heinanen, "Multi- 1821 Protocol Label Switching (MPLS) Support of Differentiated 1822 Services", RFC 3270, DOI 10.17487/RFC3270, May 2002, 1823 . 1825 [RFC3443] Agarwal, P. and B. Akyol, "Time To Live (TTL) Processing 1826 in Multi-Protocol Label Switching (MPLS) Networks", 1827 RFC 3443, DOI 10.17487/RFC3443, January 2003, 1828 . 1830 [RFC3473] Berger, L., Ed., "Generalized Multi-Protocol Label 1831 Switching (GMPLS) Signaling Resource ReserVation Protocol- 1832 Traffic Engineering (RSVP-TE) Extensions", RFC 3473, 1833 DOI 10.17487/RFC3473, January 2003, 1834 . 1836 [RFC4206] Kompella, K. and Y. Rekhter, "Label Switched Paths (LSP) 1837 Hierarchy with Generalized Multi-Protocol Label Switching 1838 (GMPLS) Traffic Engineering (TE)", RFC 4206, 1839 DOI 10.17487/RFC4206, October 2005, 1840 . 1842 [RFC4385] Bryant, S., Swallow, G., Martini, L., and D. McPherson, 1843 "Pseudowire Emulation Edge-to-Edge (PWE3) Control Word for 1844 Use over an MPLS PSN", RFC 4385, DOI 10.17487/RFC4385, 1845 February 2006, . 1847 [RFC5129] Davie, B., Briscoe, B., and J. Tay, "Explicit Congestion 1848 Marking in MPLS", RFC 5129, DOI 10.17487/RFC5129, January 1849 2008, . 1851 [RFC5462] Andersson, L. and R. Asati, "Multiprotocol Label Switching 1852 (MPLS) Label Stack Entry: "EXP" Field Renamed to "Traffic 1853 Class" Field", RFC 5462, DOI 10.17487/RFC5462, February 1854 2009, . 1856 [RFC6003] Papadimitriou, D., "Ethernet Traffic Parameters", 1857 RFC 6003, DOI 10.17487/RFC6003, October 2010, 1858 . 1860 [RFC6073] Martini, L., Metz, C., Nadeau, T., Bocci, M., and M. 1861 Aissaoui, "Segmented Pseudowire", RFC 6073, 1862 DOI 10.17487/RFC6073, January 2011, 1863 . 1865 [RFC8200] Deering, S. and R. Hinden, "Internet Protocol, Version 6 1866 (IPv6) Specification", STD 86, RFC 8200, 1867 DOI 10.17487/RFC8200, July 2017, 1868 . 1870 14.2. Informative references 1872 [I-D.ietf-6man-segment-routing-header] 1873 Previdi, S., Filsfils, C., Raza, K., Dukes, D., Leddy, J., 1874 Field, B., daniel.voyer@bell.ca, d., 1875 daniel.bernier@bell.ca, d., Matsushima, S., Leung, I., 1876 Linkova, J., Aries, E., Kosugi, T., Vyncke, E., Lebrun, 1877 D., Steinberg, D., and R. Raszuk, "IPv6 Segment Routing 1878 Header (SRH)", draft-ietf-6man-segment-routing-header-08 1879 (work in progress), January 2018. 1881 [I-D.ietf-detnet-architecture] 1882 Finn, N., Thubert, P., Varga, B., and J. Farkas, 1883 "Deterministic Networking Architecture", draft-ietf- 1884 detnet-architecture-04 (work in progress), October 2017. 1886 [I-D.ietf-detnet-dp-alt] 1887 Korhonen, J., Farkas, J., Mirsky, G., Thubert, P., 1888 Zhuangyan, Z., and L. Berger, "DetNet Data Plane Protocol 1889 and Solution Alternatives", draft-ietf-detnet-dp-alt-00 1890 (work in progress), October 2016. 1892 [I-D.sdt-detnet-security] 1893 Mizrahi, T., Grossman, E., Hacker, A., Das, S., 1894 "Deterministic Networking (DetNet) Security 1895 Considerations, draft-sdt-detnet-security, work in 1896 progress", 2017. 1898 [IEEE1588] 1899 IEEE, "IEEE 1588 Standard for a Precision Clock 1900 Synchronization Protocol for Networked Measurement and 1901 Control Systems Version 2", 2008. 1903 [IEEE8021CB] 1904 Finn, N., "Draft Standard for Local and metropolitan area 1905 networks - Seamless Redundancy", IEEE P802.1CB 1906 /D2.1 P802.1CB, December 2015, 1907 . 1910 [IEEE8021Q] 1911 IEEE 802.1, "Standard for Local and metropolitan area 1912 networks--Bridges and Bridged Networks (IEEE Std 802.1Q- 1913 2014)", 2014, . 1915 [RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S. 1916 Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1 1917 Functional Specification", RFC 2205, DOI 10.17487/RFC2205, 1918 September 1997, . 1920 [RFC3985] Bryant, S., Ed. and P. Pate, Ed., "Pseudo Wire Emulation 1921 Edge-to-Edge (PWE3) Architecture", RFC 3985, 1922 DOI 10.17487/RFC3985, March 2005, 1923 . 1925 [RFC5654] Niven-Jenkins, B., Ed., Brungard, D., Ed., Betts, M., Ed., 1926 Sprecher, N., and S. Ueno, "Requirements of an MPLS 1927 Transport Profile", RFC 5654, DOI 10.17487/RFC5654, 1928 September 2009, . 1930 [RFC7551] Zhang, F., Ed., Jing, R., and R. Gandhi, Ed., "RSVP-TE 1931 Extensions for Associated Bidirectional Label Switched 1932 Paths (LSPs)", RFC 7551, DOI 10.17487/RFC7551, May 2015, 1933 . 1935 [RFC7950] Bjorklund, M., Ed., "The YANG 1.1 Data Modeling Language", 1936 RFC 7950, DOI 10.17487/RFC7950, August 2016, 1937 . 1939 [RFC8040] Bierman, A., Bjorklund, M., and K. Watsen, "RESTCONF 1940 Protocol", RFC 8040, DOI 10.17487/RFC8040, January 2017, 1941 . 1943 Appendix A. Example of DetNet data plane operation 1945 [Editor's note: Add a simplified example of DetNet data plane and how 1946 labels etc work in the case of MPLS-based PSN and utilizing PREF. 1947 The figure is subject to change depending on the further DT decisions 1948 on the label handling..] 1950 Appendix B. Example of pinned paths using IPv6 1952 TBD. 1954 Authors' Addresses 1956 Jouni Korhonen (editor) 1957 Nordic Semiconductor 1959 Email: jouni.nospam@gmail.com 1961 Loa Andersson 1962 Huawei 1964 Email: loa@pi.nu 1966 Yuanlong Jiang 1967 Huawei 1969 Email: jiangyuanlong@huawei.com 1971 Norman Finn 1972 Huawei 1973 3101 Rio Way 1974 Spring Valley, CA 91977 1975 USA 1977 Email: norman.finn@mail01.huawei.com 1979 Balazs Varga 1980 Ericsson 1981 Konyves Kalman krt. 11/B 1982 Budapest 1097 1983 Hungary 1985 Email: balazs.a.varga@ericsson.com 1986 Janos Farkas 1987 Ericsson 1988 Konyves Kalman krt. 11/B 1989 Budapest 1097 1990 Hungary 1992 Email: janos.farkas@ericsson.com 1994 Carlos J. Bernardos 1995 Universidad Carlos III de Madrid 1996 Av. Universidad, 30 1997 Leganes, Madrid 28911 1998 Spain 2000 Phone: +34 91624 6236 2001 Email: cjbc@it.uc3m.es 2002 URI: http://www.it.uc3m.es/cjbc/ 2004 Tal Mizrahi 2005 Marvell 2006 6 Hamada st. 2007 Yokneam 2008 Israel 2010 Email: talmi@marvell.com 2012 Lou Berger 2013 LabN Consulting, L.L.C. 2015 Email: lberger@labn.net