idnits 2.17.1 draft-ietf-detnet-dp-sol-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (March 5, 2018) is 2241 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'TBD' is mentioned on line 1694, but not defined == Unused Reference: 'RFC6073' is defined on line 1818, but no explicit reference was found in the text == Outdated reference: A later version (-26) exists of draft-ietf-6man-segment-routing-header-08 == Outdated reference: A later version (-13) exists of draft-ietf-detnet-architecture-04 Summary: 0 errors (**), 0 flaws (~~), 5 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 DetNet J. Korhonen, Ed. 3 Internet-Draft Nordic 4 Intended status: Standards Track L. Andersson 5 Expires: September 6, 2018 Y. Jiang 6 N. Finn 7 Huawei 8 B. Varga 9 J. Farkas 10 Ericsson 11 CJ. Bernardos 12 UC3M 13 T. Mizrahi 14 Marvell 15 L. Berger 16 LabN 17 March 5, 2018 19 DetNet Data Plane Encapsulation 20 draft-ietf-detnet-dp-sol-02 22 Abstract 24 This document specifies Deterministic Networking data plane 25 encapsulation solutions. The described data plane solutions can be 26 applied over either IP or MPLS Packet Switched Networks. 28 Status of This Memo 30 This Internet-Draft is submitted in full conformance with the 31 provisions of BCP 78 and BCP 79. 33 Internet-Drafts are working documents of the Internet Engineering 34 Task Force (IETF). Note that other groups may also distribute 35 working documents as Internet-Drafts. The list of current Internet- 36 Drafts is at https://datatracker.ietf.org/drafts/current/. 38 Internet-Drafts are draft documents valid for a maximum of six months 39 and may be updated, replaced, or obsoleted by other documents at any 40 time. It is inappropriate to use Internet-Drafts as reference 41 material or to cite them other than as "work in progress." 43 This Internet-Draft will expire on September 6, 2018. 45 Copyright Notice 47 Copyright (c) 2018 IETF Trust and the persons identified as the 48 document authors. All rights reserved. 50 This document is subject to BCP 78 and the IETF Trust's Legal 51 Provisions Relating to IETF Documents 52 (https://trustee.ietf.org/license-info) in effect on the date of 53 publication of this document. Please review these documents 54 carefully, as they describe your rights and restrictions with respect 55 to this document. Code Components extracted from this document must 56 include Simplified BSD License text as described in Section 4.e of 57 the Trust Legal Provisions and are provided without warranty as 58 described in the Simplified BSD License. 60 Table of Contents 62 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 63 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 64 2.1. Terms used in this document . . . . . . . . . . . . . . . 4 65 2.2. Abbreviations . . . . . . . . . . . . . . . . . . . . . . 5 66 3. Requirements language . . . . . . . . . . . . . . . . . . . . 6 67 4. DetNet data plane overview . . . . . . . . . . . . . . . . . 6 68 4.1. DetNet data plane encapsulation requirements . . . . . . 8 69 4.2. Packet replication and elimination considerations . . . . 10 70 4.3. Packet reordering considerations . . . . . . . . . . . . 10 71 5. DetNet encapsulation . . . . . . . . . . . . . . . . . . . . 10 72 5.1. End-system specific considerations . . . . . . . . . . . 10 73 5.2. DetNet domain specific considerations . . . . . . . . . . 12 74 5.2.1. DetNet Bridging Service . . . . . . . . . . . . . . . 13 75 5.2.2. DetNet Routing Service . . . . . . . . . . . . . . . 14 76 5.3. DetNet Inter-Working Function (DN-IWF) . . . . . . . . . 16 77 5.3.1. Networks with multiple technology segments . . . . . 16 78 5.3.2. DN-IWF related considerations . . . . . . . . . . . . 17 79 6. MPLS-based DetNet data plane solution . . . . . . . . . . . . 18 80 6.1. DetNet specific packet fields . . . . . . . . . . . . . . 18 81 6.2. Data plane encapsulation . . . . . . . . . . . . . . . . 18 82 6.3. DetNet control word . . . . . . . . . . . . . . . . . . . 19 83 6.4. Flow identification . . . . . . . . . . . . . . . . . . . 20 84 6.5. Service layer considerations . . . . . . . . . . . . . . 20 85 6.5.1. Edge node processing . . . . . . . . . . . . . . . . 21 86 6.5.2. Relay node processing . . . . . . . . . . . . . . . . 22 87 6.5.3. End system processing . . . . . . . . . . . . . . . . 24 88 6.6. Transport node considerations . . . . . . . . . . . . . . 24 89 6.6.1. Congestion protection . . . . . . . . . . . . . . . . 24 90 6.6.2. Explicit routes . . . . . . . . . . . . . . . . . . . 24 91 7. IPv6-based DetNet data plane solution . . . . . . . . . . . . 24 92 7.1. Data plane encapsulation . . . . . . . . . . . . . . . . 24 93 7.2. DetNet destination option . . . . . . . . . . . . . . . . 26 94 7.3. Flow identification . . . . . . . . . . . . . . . . . . . 27 95 7.4. Service layer considerations . . . . . . . . . . . . . . 27 96 7.4.1. Edge node processing . . . . . . . . . . . . . . . . 28 97 7.4.2. Relay node processing . . . . . . . . . . . . . . . . 30 98 7.4.3. End system processing . . . . . . . . . . . . . . . . 30 99 7.5. Transport node processing . . . . . . . . . . . . . . . . 30 100 7.5.1. Congestion protection . . . . . . . . . . . . . . . . 30 101 7.5.2. Explicit routes . . . . . . . . . . . . . . . . . . . 31 102 8. Other DetNet data plane considerations . . . . . . . . . . . 31 103 8.1. Class of Service . . . . . . . . . . . . . . . . . . . . 31 104 8.2. Quality of Service . . . . . . . . . . . . . . . . . . . 31 105 8.3. Cross-DetNet flow resource aggregation . . . . . . . . . 33 106 8.4. Bidirectional traffic . . . . . . . . . . . . . . . . . . 34 107 8.5. Layer 2 addressing and QoS Considerations . . . . . . . . 34 108 8.6. Interworking between MPLS- and IPv6-based encapsulations 35 109 8.7. IPv4 considerations . . . . . . . . . . . . . . . . . . . 35 110 9. Time synchronization . . . . . . . . . . . . . . . . . . . . 35 111 10. Management and control considerations . . . . . . . . . . . . 37 112 10.1. MPLS-based data plane . . . . . . . . . . . . . . . . . 37 113 10.1.1. S-Label assignment and distribution . . . . . . . . 37 114 10.1.2. Explicit routes . . . . . . . . . . . . . . . . . . 37 115 10.2. IPv6-based data plane . . . . . . . . . . . . . . . . . 37 116 10.2.1. Flow Label assignment and distribution . . . . . . . 37 117 10.2.2. Explicit routes . . . . . . . . . . . . . . . . . . 38 118 10.3. Packet replication and elimination . . . . . . . . . . . 38 119 10.4. Congestion protection and latency control . . . . . . . 38 120 10.5. Flow aggregation control . . . . . . . . . . . . . . . . 38 121 11. Security considerations . . . . . . . . . . . . . . . . . . . 38 122 12. IANA considerations . . . . . . . . . . . . . . . . . . . . . 38 123 13. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 38 124 14. References . . . . . . . . . . . . . . . . . . . . . . . . . 39 125 14.1. Normative references . . . . . . . . . . . . . . . . . . 39 126 14.2. Informative references . . . . . . . . . . . . . . . . . 41 127 Appendix A. Example of DetNet data plane operation . . . . . . . 42 128 Appendix B. Example of pinned paths using IPv6 . . . . . . . . . 43 129 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 43 131 1. Introduction 133 Deterministic Networking (DetNet) is a service that can be offered by 134 a network to DetNet flows. DetNet provides these flows extremely low 135 packet loss rates and assured maximum end-to-end delivery latency. 136 General background and concepts of DetNet can be found in 137 [I-D.ietf-detnet-architecture]. 139 This document specifies the DetNet data plane and the on-wire 140 encapsulation of DetNet flows. The specified encapsulation provides 141 the building blocks to enable the DetNet service layer functions and 142 allow flow identification as described in the DetNet Architecture. 143 Two data plane definitions are given. 145 1. MPLS-based: The enacapsulation resembles PseudoWires (PW) with an 146 MPLS Packet Switched Network (PSN) [RFC3985][RFC4385]. 148 2. Native-IP-based: The encapsulating protocol is IPv6 and the 149 solution relies on IP header fields, existing and DetNet specific 150 IPv6 eaxtension header options [RFC8200]. 152 [Editor's note: MPLS- and IPv6-based solutions are likely to be 153 split into different documents.] 155 It is worth noting that while MPLS-based solution can transport IP 156 packets a native-IP solution is meant for deployments where the 157 DetNet service layer functions are provided at the IP-layer rather 158 than the underlying transport network. The primary reason for this 159 is the benefit gained by enabling the use of a normal application 160 stack, where transport protocols such as TCP or UDP are directly 161 encapsulated in IP. 163 The DetNet transport layer functionality that provides congestion 164 protection for DetNet flows is assumed to be in place in a DetNet 165 node. 167 Furthermore, this document also describes how DetNet flows are 168 identified, how a DetNet Relay/Edge/Transit nodes work, and how the 169 Packet Replication and Elimination function (PREF) is implemented 170 with the two data plane solutions. 172 This document does not define the associated control plane functions, 173 or Operations, Administration, and Maintenance (OAM). It also does 174 not specify traffic handling capabilities required to deliver 175 congestion protection and latency control for DetNet flows at the 176 DetNet transport layer. 178 2. Terminology 180 2.1. Terms used in this document 182 This document uses the terminology established in the DetNet 183 architecture [I-D.ietf-detnet-architecture] and the DetNet Data Plane 184 Solution Alternatives [I-D.ietf-detnet-dp-alt]. 186 T-Label A label used to identify the LSP used to transport a 187 DetNet flow across an MPLS PSN, e.g., a hop-by-hop 188 label used between label switching routers (LSR). 190 S-Label A DetNet "service" label that is used between DetNet 191 nodes that implment also the DetNet service layer 192 functions. An S-Label is also used to identify a 193 DetNet flow at DetNet service layer. 195 Flow Label IPv6 header field that is used to identify a DetNet 196 flow (together with the source IP address field). 198 Local-ID A DetNet Edge and Relay node internal construct that 199 uniquely identifies a DetNet flow within a node and 200 never appear on-wire. It may be used to select proper 201 forwarding and/or DetNet specific service function. 203 PREF A Packet Replication and Elimination Function (PREF) 204 does the replication and elimination processing of 205 DetNet flow packets in edge or relay nodes. The 206 replication function is essentially the existing 1+1 207 protection mechanism. The elimination function reuses 208 and extends the existing duplicate detection mechanism 209 to operate over multiple (separate) DetNet member flows 210 of a DetNet compound flow. 212 DetNet Control Word A control word used for sequencing and 213 identifying duplaicate packets at the DetNet service 214 layer. 216 2.2. Abbreviations 218 The following abbreviations used in this document: 220 AC Attachment Circuit. 222 CE Customer Edge equipment. 224 CoS Class of Service. 226 CW Control Word. 228 d-CW DetNet Control Word. 230 DetNet Deterministic Networking. 232 DF DetNet Flow. 234 L2VPN Layer 2 Virtual Private Network. 236 LSR Label Switching Router. 238 MPLS Multiprotocol Label Switching. 240 MPLS-TP Multiprotocol Label Switching - Transport Profile. 242 MS-PW Multi-Segment PseudoWire (MS-PW). 244 NSP Native Service Processing. 246 OAM Operations, Administration, and Maintenance. 248 PE Provider Edge. 250 PREF Packet Replication and Elimination Function. 252 PSN Packet Switched Network. 254 PW PseudoWire. 256 QoS Quality of Service. 258 TSN Time-Sensitive Network. 260 3. Requirements language 262 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 263 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 264 document are to be interpreted as described in [RFC2119]. 266 4. DetNet data plane overview 268 This document describes how to use IP and/or MPLS to support a data 269 plane method of flow identification and packet formwarding over 270 layer-3. Two different cases are covered: (i) the inter-connect 271 scenario, in which IEEE802.1 TSN is routed over a layer-3 network 272 (i.e., to enlarge the layer-2 domain), and (ii) native connectivity 273 between DetNet-aware end systems. 275 Figure 1 illustrates how DetNet can provide services for IEEE 276 802.1TSN end systems over a DetNet enabled network. The edge nodes 277 insert and remove required DetNet data plane encapsulation. The 'X' 278 in the edge and relay nodes represents a potential DetNet flow packet 279 replication and elimination point. This conceptually parallels L2VPN 280 services, and could leverage existing related solutions as discussed 281 below. 283 TSN |<---------- End to End DetNet Service ------>| TSN 284 Service | Transit Transit | Service 285 TSN (AC) | |<-Tunnel->| |<-Tnl->| | (AC) TSN 286 End | V V 1 V V 2 V V | End 287 System | +--------+ +--------+ +--------+ | System 288 +---+ | | E1 |==========| R1 |=======| E2 | | +---+ 289 | |--|----|._X_....|..DetNet..|.._ _...|..DF3..|...._X_.|---|---| | 290 |CE1| | | \ | Flow 1 | X | | / | | |CE2| 291 | | | \_.|...DF2....|._/ \_..|..DF4..|._/ | | | 292 +---+ | |==========| |=======| | +---+ 293 ^ +--------+ +--------+ +--------+ ^ 294 | Edge Node Relay Node Edge Node | 295 | | 296 |<----- Emulated Time Sensitive Networking (TSN) Service ---->| 298 Figure 1: IEEE 802.1TSN over DetNet 300 Figure 2 illustrates how end to end MPLS-based DetNet service can be 301 provided. In this case, the end systems are able to send and receive 302 DetNet flows. For example, an end system sends data encapsulated in 303 MPLS. Like earlier the 'X' in the end systems, edge and relay nodes 304 represents potential DetNet flow packet replication and elimination 305 points. Here the relay nodes may change the underlying transport, 306 for example tunneling IP over MPLS, or simply interconnect network 307 segments. 309 DetNet DetNet 310 Service Transit Transit Service 311 DetNet | |<-Tnl->| |<-Tnl->| | DetNet 312 End | V 1 V V 2 V | End 313 System | +--------+ +--------+ +--------+ | System 314 +---+ | | R1 |=======| R2 |=======| R3 | | +---+ 315 | X...DFa...|._X_....|..DF1..|.__ ___.|..DF3..|...._X_.|.DFa..|.X | 316 |CE1|========| \ | | X | | / |======|CE2| 317 | | | | \_.|..DF2..|._/ \__.|..DF4..|._/ | | | | 318 +---+ | |=======| |=======| | +---+ 319 ^ +--------+ +--------+ +--------+ ^ 320 | Relay Node Relay Node Relay Node | 321 | | 322 |<--------------- End to End DetNet Service -------------->| 324 Figure 2: MPLS-Based Native DetNet 326 Figure 3 illustrates how end to end IP-based DetNet service can be 327 provided. In this case, the end systems are able to send and receive 328 DetNet flows. [Editor's note: TBD] 329 NOTE: This figures is TBD 331 DetNet DetNet 332 Service Transit Transit Service 333 DetNet | |<-Tnl->| |<-Tnl->| | DetNet 334 End | V 1 V V 2 V | End 335 System | +--------+ +--------+ +--------+ | System 336 +---+ | | R1 |=======| R2 |=======| R3 | | +---+ 337 | X...DFa...| | | | | | .|.X | 338 | H1|========| | | | | |======| H2| 339 | | | | | | | | | | | | 340 +---+ | |=======| |=======| | +---+ 341 ^ +--------+ +--------+ +--------+ ^ 342 | Relay Node Relay Node Relay Node | 343 | | 344 |<--------------- End to End DetNet Service -------------->| 346 Figure 3: IP-Based Native DetNet 348 4.1. DetNet data plane encapsulation requirements 350 Two major groups of scenarios can be distinguished which require flow 351 identification during transport: 353 1. DetNet function related scenarios: 355 * Congestion protection and latency control: usage of allocated 356 resources (queuing, policing, shaping). 358 * Explicit routes: select/apply the flow specific path. 360 * Service protection: recognize DetNet compound and member flows 361 for replication an elimination. 363 Comment #12 I am not sure whether the correct architectural 364 construct is flow or flow group. Flow suggests that sharing/ 365 aggregation is not allowed but whether this is allowed or not 366 is an application specific issue. 368 Discussion: Agree that a flow group would be a better 369 characterization. 371 Comment #13 I think that there needs to be some clarification as 372 to whether FG is is understood by the DN system exclusively or 373 whether there is an expectation that it is understood by the 374 underlay. 376 Discussion: Agree that more detail is needed here. DetNet aware 377 nodes need to understand flow groups. Underlay needs to be 378 aware of flow groups at the resource allocation level. 380 2. OAM function related scenarios: 382 * troubleshooting (e.g., identify misbehaving flows, etc.) 384 * recognize flow(s) for analytics (e.g., increase counters, 385 etc.) 387 * correlate events with flows (e.g., volume above threshold, 388 etc.) 390 * etc. 392 Each DetNet node (edge, relay and transit) use an internal/ 393 implementation specific local-ID of the DetNet-(compound)-flow in 394 order to accomplish its role during transport. Recognizing the 395 DetNet flow is more relaxed for edge and relay nodes, as they are 396 fully aware of both the DetNet service and transport layers. The 397 primary DetNet role of intermediate transport nodes is limited to 398 ensuring congestion protection and latency control for the above 399 listed DetNet functions. 401 The DetNet data plane allows for the aggregation of DetNet flows, 402 e.g., via MPLS hierarchical LSPs, to improved scaling. When DetNet 403 flows are aggregated, transit nodes may have limited ability to 404 provide service on per-flow DetNet identifiers. Therefore, 405 identifying each individual DetNet flow on a transit node may not be 406 achieved in some network scenarios, but DetNet service can still be 407 assured in these scenarios through resource allocation and control. 409 Comment #14 You could introduce the concept of a flow group 410 identified into the packet. You may also include a flow id at a 411 lower layer. 413 Discussion: Agree on the identification properties. Adding a 414 specific id into actual on-wire formats is not necessarily needed. 416 On each DetNet node dealing with DetNet flows, an internal local-ID 417 is assumed to determine what local operation a packet goes through. 418 Therefore, local-IDs has to be unique on each edge and relay nodes. 419 Local-ID is unambiguously bound to the DetNet flow. 421 4.2. Packet replication and elimination considerations 423 DetNet service layer introduces packet replication and elimination 424 functionality (PREF) for use in DetNet edge and relay node and end 425 system packet processing. PREF MAY be enabled in a DetNet node and 426 the required processing is only applied to packets with a positive 427 flow identification at the DetNet service layer. PREF utilizes a 428 sequence number carried within a DetNet flow packets. 430 At a DetNet node level the output of the PREF elimination function is 431 always a single packet. The output of the PREF replication function 432 at a DetNet node level is always one or more packets (i.e., 1:M 433 replication). The replicated packets MUST share the same d-CW i.e., 434 the sequence number is the same for each member flow of the compound 435 flow. The location and mechanism on the packet processing pipeline 436 used for replication is implementation specific. 438 The complex part of the DetNet PREF processing is tracking the 439 history of received packets for multiple DetNet member flows. These 440 ingress DetNet member flows (to a node) MUST have the same local-ID 441 if they belong to the same DetNet (compound) flow and share the same 442 sequence number counter and the history information. The location of 443 the packet elimination on the packet processing pipeline is 444 implementation specific. 446 4.3. Packet reordering considerations 448 DetNet service layer introduces also packet reordering functionality 449 for use in DetNet edge and relay node and end system packet 450 processing. The reordering functionality MAY be enabled in a DetNet 451 node. The reordeing functionality relies on a presence of sequence 452 numbers in a DetNet (compound) flows. The reordering processing is 453 only applied to packets with a positive flow identification at the 454 DetNet service layer. 456 5. DetNet encapsulation 458 5.1. End-system specific considerations 460 Data-flows requiring DetNet service are generated and terminated on 461 end-systems. Encapsulation depends on application and its 462 preferences. In a DetNet (or even a TSN) domain the DN (TSN) 463 functions use at most two flow parameters, namely Flow-ID and 464 Seq.Number. However, an application may exchange further flow 465 related parameters (e.g., time-stamp), which are not considered by DN 466 functions. 468 Two types of end-systems are distinguished: 470 o L3 (IP) end-system: application over L3 472 o L2 (Ethernet) end-system: application directly over L2 474 In case of Ethernet end-systems the application data is encapsulated 475 directly in L2. From the DN domain perspective no upper layer 476 protocols are visible. The Data-flow uses only Ethernet tag(s) and 477 further flow specific parameters (if needed) are hidden inside the 478 PDU. 480 The IP end-system scenario is different. Data-flows are encapsulated 481 directly in L3 (i.e., IP) and the application may use further upper 482 layer protocols (e.g., RTP). Many valid combinations exist, and it 483 may be application specific how the IP header fields are used. Also, 484 usage of further upper layer protocols depends on application 485 requirements (e.g., time-stamp). Some examples for encoding of Flow- 486 ID or Seq.Number attributes: IP address, IPv6-Flow-label, L4 ports, 487 RTP-header, etc. 489 As a general rule, DetNet domains MUST be capable to forward any 490 Data-flows and the DetNet domain MUST NOT intend to mandate end- 491 system encapsulation format. 493 Furthermore, no application-level-proxy function is envisioned inside 494 the DetNet domain, so end-systems peer with end-systems using the 495 same application encapsulation format (see figure below): 497 o L3 end-systems peer with L3 end-systems and 499 o L2 end-systems peer with L2 end-systems 500 +-----+ 501 | X | +-----+ 502 +-----+ | X | 503 | Eth | ________ +-----+ 504 +-----+ _____ / \ | Eth | 505 \ / \__/ \___ +-----+ 506 \ / \ / 507 0======== tunnel-1 ========0_ 508 | \ 509 \ | 510 0========= tunnel-2 =========0 511 / \ __/ \ 512 +-----+ \__ DetNet domain / \ 513 | X | \ __ / +-----+ 514 +-----+ \_______/ \_____/ | X | 515 | IP | +-----+ 516 +-----+ | IP | 517 +-----+ 519 Figure 4: End-systems and the DetNet domain 521 5.2. DetNet domain specific considerations 523 From connection type perspective three scenarios are distinguished: 525 1. Directly attached: end-system is directly connected to an edge 526 node 528 2. Indirectly attached: end-system is behind a (L2-TSN / L3-DetNet) 529 sub-net 531 3. DN integrated: end-system is part of the DetNet domain 533 L3 end-systems may use any of these connection types, however L2 end- 534 systems may use only the first two (directly or indirectly attached). 535 DetNet domain MUST allow communication between any end-systems of the 536 same type (L2-L2, L3-L3), independent of their connection type and 537 DetNet capability. However directly attached and indirectly attached 538 end-systems have no knowledge about the DetNet domain and its 539 encapsulation format at all. See the figure below for L3 end-system 540 scenarios. 542 ____+----+ 543 +----+ _____ / | ES3| 544 | ES1|____ / \__/ +----+___ 545 +----+ \ / \ 546 + | 547 ____ \ _/ 548 +----+ __/ \ +__ DetNet domain / 549 | ES2|____/ L2/L3 |___/ \ __ __/ 550 +----+ \_______/ \_______/ \___/ 552 Figure 5: Connection types of L3 end-systems 554 5.2.1. DetNet Bridging Service 556 The simplest DetNet service is to provide bridging (i.e., tunneling 557 for L2), where the connected hosts are in the same broadcast (BC) 558 domain. Forwarding over the DetNet domain is based on L2 (MAC) 559 addresses (i.e. dst-MAC), so L2 headers MUST be kept. For both IP 560 and MPLS PSN a DetNet specific tunnel encapsulation MUST be 561 introduced. 563 +------+ 564 | X | 565 +------+ +------+ 566 | X | | IP | 567 +------+ +------+ 568 End+system | L2 | | L2 | 569 +-----+======+-+======+--+======+-+======++ 570 DetNet tunnel | IP | | MPLS | 571 +------+ +------+ 572 | L2 | | L2 | 573 +------+ +------+ 575 Examples 577 +------+ +------+ 578 | X | | X | 579 +------+ +------+ +------+ +------+ 580 | X | | X | | IP | | IP | 581 +------+ +------+ +------+ +------+ 582 | L2 | | L2 | | L2 | | L2 | 583 ++======+-+======+--+======+-+======++ 584 | IP | | MPLS | | IP | | MPLS | 585 +------+ +------+ +------+ +------+ 586 | L2 | | L2 | | L2 | | L2 | 587 +------+ +------+ +------+ +------+ 589 Figure 6: Encapsulation format for DetNet Bridging 591 As shown on the figure both L2 and L3 end-systems can be served by 592 such a DetNet Bridging service. 594 5.2.2. DetNet Routing Service 596 DetNet Routing service provides routing, therefore available only for 597 L3 hosts that are in different BC domains. Forwarding over the 598 DetNet domain is based on L3 (IP) addresses (i.e. dst-IP). 600 5.2.2.1. MPLS PSN 602 In case of an MPLS PSN at the ingress/egress (i.e., PE nodes of 603 DetNet domain) the IP packets are encapsulated in MPLS. The data- 604 flow IP header MUST be preserved as-is. 606 +------+ +------+ 607 | X | | X | 608 +------+ +------+ 609 End+system | IP | | IP | 610 -----+------+-------+======+--- --+======+-- 611 DetNet | MPLS | | MPLS | 612 +------+ +------+ 613 | L2 | | L2 | 614 +------+ +------+ 616 Figure 7: Encapsulation format for DetNet Routing in MPLS PSN for L3 617 end-systems 619 5.2.2.2. IP PSN 621 In case of an IP PSN the same tunneling concept can be used as for an 622 MPLS PSN, but the tunnel is constructed by a new IP header (and 623 possible upper layer fields). The data-flow IP header MUST be 624 preserved as-is. 626 +------+ +------+ 627 | X | | X | 628 +======+ +------+ 629 End-system | IP | | IP | 630 -----+------+-------+======+--- --+======+-- 631 DetNet | IP | | IP | 632 +------+ +------+ 633 | L2 | | L2 | 634 +------+ +------+ 636 Figure 8: Encapsulation format for DetNet Routing in IP PSN for L3 637 end-systems 639 DetNet IP header contains the IP addresses of the ingress/egress PE 640 nodes of DetNet domain. The End-system IP header contains the IP 641 addresses of the end-systems. 643 Note: In case of IP PSN one may consider avoiding the additional IP 644 encapsulation, however there are many issues with such an approach. 645 First, the DetNet nodes MUST be able to extract from the IP header 646 (and maybe upper layers) the attributes required by DetNet functions 647 (i.e. Flow-ID, Seq.Number). The challenge is that encoding of those 648 attributes may be application specific, so DetNet nodes MUST be 649 prepared to handle all application specific formats. Second, adding 650 further fields (e.g., explicit path information) to an existing IP 651 header may be impossible (e.g., due to security/encryption). 653 Furthermore, DetNet domain IP-header format may collide with IP- 654 header format used by the source of a flow. Implementing such an 655 approach requires that source encapsulation is in-line with DetNet 656 domain encapsulation format, however we do not intend to mandate end- 657 systems' encapsulation format (see former text: As a general rule, 658 DetNet domains MUST be capable to forward any Data-flows and the 659 DetNet domain MUST NOT intend to mandate end-system encapsulation 660 format.). 662 5.3. DetNet Inter-Working Function (DN-IWF) 664 5.3.1. Networks with multiple technology segments 666 There are network scenarios, where the DetNet domain contains 667 multiple technology segments (IP, MPLS) and all those segments are 668 under the same administrative control. Furthermore, DetNet nodes may 669 be interconnected via TSN segments. 671 An important aspect of DetNet network design is placement of DetNet 672 functions across the domain. Designs based on segment-by-segment 673 optimization can provide only suboptimal solutions. In order to 674 achieve global optimum Inter-Working Functions (DN-IWF) can be placed 675 at segment border nodes, which stich together DetNet flows across 676 connected segments. 678 DN-IWF may ensure that flow attributes are correlated across segment 679 borders. For example, there are two DetNet functions which require 680 Seq.Numbers: (1) Elimination: removes duplications from flows and (2) 681 IOD: ensures in-order-delivery of packet in a flow. Stitching flows 682 together and correlating attributes means for example that 683 replication of packets can happen in one segment and elimination of 684 duplicates in a different one. 686 ______ 687 _____ / \__ 688 ____ / \__/ \___ ______ 689 +----+ __/ +======+ +==+ \ +----+ 690 |src |__/ Seg1 ) | | \ Seg3 \____| dst| 691 +----+ \_______+ \ Segment-2 | \+_____/ +----+ 692 \======+__ _+===/ 693 \ __ __/ 694 \_______/ \___/ 696 +------------+ 697 +------------------E----+ | | 698 +----+ | | +----R---+ | +----+ 699 |src |-------R +---+ | E----------+ dst| 700 +----+ | | E--------+ +----+ 701 +--------------R | 702 +-----------------+ 704 Figure 9: Optimal replication and elimination placement across 705 technology segments example 707 5.3.2. DN-IWF related considerations 709 The ultimate goal of DN-IWF is to (1) match and (2) translate segment 710 specific flow attributes. The DN-IWF ensures that segment specific 711 attributes comprise per domain unique attributes for the whole DetNet 712 domain. This characteristic can ensure that DetNet functions can be 713 based on per domain attributes and not per segment attributes. 715 The two DetNet specific attributes have the following 716 characteristics: 718 o Flow-ID: it is same in all packets of a flow 720 o Seq.Number: it is different packet-by-packet 722 For the Flow-ID the DN-IWF can implement a static mapping. The 723 situation is more complicated for Seq.Number as it is different 724 packet-by-packet, so it may need more sophisticated translation 725 unless its format is exactly the same in the two technology segments. 726 In this later case the DN-IWF can simple copy the Seq.Number field 727 between the tunneling encapsulation of the two technology segments. 729 In case of three technology segments (IP, MPLS and TSN) three DN-IWF 730 functions can be specified. In the rest of this section the focus is 731 on the (1) IP - MPLS network scenario. Note: the use-cases are out- 732 of-scope for (2) TSN - IP, (3) TSN - MPLS. Note2: incompatible 733 format of Seq.Number with TSN. 735 Simplest implementation of DN-IWF is provided if the flow attributes 736 have the same format. Such a common denominator of the tunnel 737 encapsulation format is the pseudowire encapsulation over both IP and 738 MPLS. 740 Placeholder 742 Figure 10: FIGURE Placeholder PW over X 744 6. MPLS-based DetNet data plane solution 746 6.1. DetNet specific packet fields 748 The DetNet data plane encapsulation MUST include two DetNet specific 749 information elements in each packet of a DetNet flow: (1) a flow 750 identification and (2) a sequence number. 752 The DetNet data plane encapsulation may consists further elements 753 used for overlay tunneling, to distinguish between DetNet member 754 flows of the same DetNet compound flow or to support OAM functions. 756 6.2. Data plane encapsulation 758 Figure 11 illustrates a DetNet data plane MPLS encapsulation. The 759 MPLS-based encapsulation of the DetNet flows is a good fit for the 760 Layer-2 interconnect deployment cases (see Figure 1). Furthermore, 761 end to end DetNet service i.e., native DetNet deployment (see 762 Figure 2) is also possible if DetNet end systems are capable of 763 initiating and termination MPLS encapsulated packets. Transport of 764 IP encapsulated DetNet flows, see Section 7, over MPLS-based DetNet 765 data plane is also possible. Interworking between PW- and IPv6-based 766 encapsulations is discussed further in Section 8.6. 768 The MPLS-based DetNet data plane encapsulation consists of: 770 o DetNet control word (d-CW) containing sequencing information for 771 packet replication and duplicate elimination purposes. There MUST 772 a separate sequence number space for each DetNet flow. 774 o DetNet Label that identifies a DetNet flow within a DetNet Edge or 775 a Relay node. The DetNet label MUST be at the bottom of the label 776 stack. 778 o An optional DetNet service lable (S-Label) that represents DetNet 779 Service LSP used between DetNet Egde and/or Relay nodes. One 780 possible use of an S-Label is to identify DetNet member flows used 781 to provide protection to a DetNet compound flow, perhaps even when 782 both LSPs appear on the same link for some reason. 784 One or more MPLS transport LSP label(s) (T-label) which may be a hop- 785 by-hop label used between LSR and MUST appear higher in the label 786 stack than S-labels. A top of stack T-label may be PHPed before 787 arriving at a DetNet node. In general T-labels should be considered 788 to be part of the underlying transport network rather the actual 789 DetNet data plane encapsulation. 791 DetNet MPLS-based encapsulation 793 +---------------------------------+ 794 | | 795 | DetNet Flow | 796 | Payload Packet | 797 | | 798 +---------------------------------+ <--\ 799 | DetNet Control Word | | 800 +---------------------------------+ +--> DetNet data plane 801 | S-Label | | MPLS encapsulation 802 +---------------------------------+ <--/ 803 | T-Label(s) | 804 +---------------------------------+ 805 | Data-Link | 806 +---------------------------------+ 807 | Physical | 808 +---------------------------------+ 810 Figure 11: Encapsulation of a DetNet flow in an MPLS(-TP) PSN 812 6.3. DetNet control word 814 A DetNet control word (d-CW) conforms to the Generic PW MPLS Control 815 Word (PWMCW) defined in [RFC4385] and is illustrated in Figure 12. 816 The upper nibble of the d-CW MUST be set to zero (0). The effective 817 sequence number bit length is between 0 and 28 bits, and configured 818 either by a control plane or manually for each DetNet flow. The 819 sequence number is aligned to the right (least significant bits) and 820 unused bits MUST be set to zero (0). Each DetNet flow MUST have its 821 own sequence number counter. The sequence number is incremented by 822 one for each new packet. 824 The d-CW MUST always be present in a packet. In a case the sequence 825 number is not used (e.g., for DetNet-t-flows) the control plane or 826 the manual configuration has to define zero (0) bit length seqeunce 827 number and the value of the sequence number MUST be set to zero (0). 829 0 1 2 3 830 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 831 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 832 |0 0 0 0| Sequence Number | 833 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 835 Figure 12: DetNet Control Word 837 6.4. Flow identification 839 DetNet flow identification at a DetNet service layer is realized by 840 an S-label. It maps a Detnet flow to a specific d-CW in a DetNet 841 node. The S-label used for flow identification MUST be bottom label 842 of the label stack for a DetNet-s- or DetNet-st-flow and MUST precede 843 the d-CW. 845 An S-label for a single DetNet flow does not need to be unique DetNet 846 domain wide. As long as two or more different DetNet flows do not 847 errorneously map to a same d-CW in a DetNet node the labels may vary. 849 6.5. Service layer considerations 851 [Editor's note: quite a bit of unfinished and old text in the 852 following sections.] 854 The edge and relay node internal procedures of the PREF are 855 implementation specific. The order of a packet elimination or 856 replication is out of scope in this specification. However, care 857 should be taken that the replication function does not actually 858 loopback packets as "replicas". Looped back packets include 859 artificial delay when the node that originally initiated the packet 860 receives it again. Also, looped back packets may make the network 861 condition to look healthier than it actually is (in some cases link 862 failures are not reflected properly because looped back packets make 863 the situation appear better than it actually is). 865 Comment #29: SB> There needs to be some text about preventing a node 866 ever receiving its own replicated packets. Indeed that would 867 suggest that the flow id should be changed and replication should 868 only take place on configured flow IDs. I have a feeling that 869 this would all be a lot safer if replication only happened at 870 ingress and we managed the diversity of the paths. 872 Discussion: Agree on hardening the loopback text considerations. 874 6.5.1. Edge node processing 876 TBD. 878 [Editor's note: Since we are not defining the inner workings and 879 implementation of the DetNet Egde node - rather only what goes in and 880 what comes out, and of course the on-wire details, then the figures 881 shown in the coming section would not need to detail the inner 882 architecture of a DetNet Node.] 884 +---------------------------------------+ 885 | DetNet Edge Node | 886 +---------------------------------------+ 887 | | | | 888 | | | | 889 Client AC | +---o <-------> o o<----------> 890 | Elim. | | | | 891 <---------->o <-------| | +-------------+ 892 | | | | | 893 | +---o <-------> o | 894 | | | o<----------> 895 | | +-> o | 896 +-------------+ | +-------------+ 897 | | | | | 898 Client AC | | Repl. | | | 899 <---------->o o <-----X-> o o<----------> 900 | | Elim. | | 901 +-------------+ +-------------+ 902 | | | | 903 Client AC | | | | 904 <---------->o o <-------> o o<----------> 905 | | | | 906 +---------------------------------------+ 908 Figure 13: DetNet Edge Node processing 910 An edge node participates to the packet replication and duplication 911 elimination. Required processing is done within an extended 912 forwarder function. In the case the native service processing (NSP) 913 is IEEE 802.1CB [IEEE8021CB] capable, the packet replication and 914 duplicate elimination MAY entirely be done in the NSP and bypassing 915 the DetNet flow encapsulation and logic entirely, and thus is able to 916 operate over unmodified implementation and deployment. The NSP 917 approach works only between edge nodes and cannot make use of relay 918 nodes (see Section 6.5.2). 920 Comment #31 SB> This would be a fine way to operate the PW system - 921 edge to edge. 923 Discussion: When it comes to use of NSPs, agree. Also for "island 924 interconnect" this is a fine. However, when there is a need to do 925 PREF in a middle, plain edge to edge is not enough. 927 The DetNet-aware extended forwarder selects the egress DetNet member 928 flow based on the DetNet forwarding rules. In both "normal AC" and 929 "Packet AC" cases there may be no DetNet encapsulation header 930 available yet as it is the case with relay nodes (see Section 6.5.2). 931 It is the responsibility of the extended forwarder within the edge 932 node to push the DetNet specific encapsulation (if not already 933 present) to the packet before forwarding it to the appropriate egress 934 DetNet member flow instance(s). 936 Comment #32 SB> I am not convinced of the wisdom of having a mid- 937 point node convert a flow into a DN flow, which is what you are 938 implying here. This seems like an ingress function. 940 Discussion: OK. The text here has issues and seems to mix relay and 941 edge. 943 The extended forwarder MAY copy the sequencing information from the 944 native DetNet packet into the DetNet sequence number field and vice 945 versa. If there is no existing sequencing information available in 946 the native packet or the forwarder chose not to copy it from the 947 native packet, then the extended forwarder MUST maintain a sequence 948 number counter for each DetNet flow (indexed by the DetNet flow 949 identification). 951 6.5.2. Relay node processing 953 TBD. 955 A DetNet Relay node participates to the packet replication and 956 duplication elimination. This processing is done within an extended 957 forwarder function. Whether an ingress DetNet member flow receives 958 DetNet specific processing depends on how the forwarding is 959 programmed. For some DetNet member flows the relay node can act as a 960 normal relay node and for some apply the DetNet specific processing 961 (i.e., PREF). 963 Comment #34 SB> Again relay node is not a normal term, so am not 964 sure what it does in the absence of a PREF function. 966 Discussion: Relay node was a DetNet aware S-PE originally, which is 967 not explicitly stated here anymore, thus slightly confusing text 968 here. The text here needs to clarify the roles of PREF and 969 switching functions. A DetNet relay is described in the 970 architecture document. However, there is definitely room for 971 termonilogy and text improvements. 973 It is also possible to treat the relay node as a transit node, see 974 Section 8.3. Again, this is entirely up to how the forwarding has 975 been programmed. 977 The DetNet-aware forwarder selects the egress DetNet member flow 978 segment based on the flow identification. The mapping of ingress 979 DetNet member flow segment to egress DetNet member flow segment may 980 be statically or dynamically configured. Additionally the DetNet- 981 aware forwarder does duplicate frame elimination based on the flow 982 identification and the sequence number combination. The packet 983 replication is also done within the DetNet-aware forwarder. During 984 elimination and the replication process the sequence number of the 985 DetNet member flow MUST be preserved and copied to the egress DetNet 986 member flow. 988 +---------------------------------------+ 989 | DetNet Relay Node | 990 Ingress +---------------------------------------+ 991 | | | | Egress 992 | o---------> o--+ Elim. | 993 ----------->o | | +--------->o-----------> 994 | | +-----> o--+ | 995 Ingress +-------------+ | +-------------+ 996 | | | | | Egress 997 | | | | | 998 ----------->o o --+ +-> o o-----------> 999 | | | | | 1000 Ingress +-------------+ | +-------------+ 1001 | | | | | Egress 1002 | | Repl. | | | 1003 ----------->o o ------+-> o o-----------> 1004 | | | | 1005 Ingress +-------------+ +-------------+ 1006 | | | | Egress 1007 | | | | 1008 ----------->o o --------> o o-----------> 1009 | | | | 1010 +---------------------------------------+ 1012 Figure 14: DetNet Relay Node processing 1014 Comment #35 SB> Somewhere in the dp document there needs to be a 1015 note of the requirement for interfaces to do fast exchange of 1016 counter state, and a note to those planning the network and 1017 designing the control plane that they need to provide support for 1018 this. 1020 Discussion: We kinf of agree but also think the above exchange or 1021 synchronization of counter states is not in our scope to solve. 1023 6.5.3. End system processing 1025 TBD. 1027 6.6. Transport node considerations 1029 6.6.1. Congestion protection 1031 TBD. 1033 6.6.2. Explicit routes 1035 TBD. 1037 7. IPv6-based DetNet data plane solution 1039 7.1. Data plane encapsulation 1041 Figure 15 illustrates a DetNet native IPv6 encapsulation. The native 1042 IPv6 encapsulation is meant for end to end Detnet service use cases, 1043 where the end stations are DetNet-aware (see Figure 3). Technically 1044 it is possible to use the IPv6 encapsulation to tunnel any traffic 1045 over a DetNet enabled network, which would make native IPv6 1046 encapsulation also a valid data plane choice for an interconnect use 1047 case (see Figure 1). 1049 The native IPv6-based DetNet data plane encapsulation consists of: 1051 o IPv6 header as the transport protocol. 1053 o IPv6 header Flow Label that is used to help to identify a DetNet 1054 flow (i.e., roughly an equivalent to an S-Label for the MPLS 1055 encapsulation). A Flow Label together with the IPv6 source 1056 address uniquely identifies a DetNet flow. 1058 Comment #21 SB> Have we validated that it is unconditionally safe to 1059 make this assumption about the use of the FL? 1061 Discussion: RFC6437 does not restrict such use and DetNet DP 1062 solution can always define their own use of flow label. It should 1063 be noted that a DetNet aware node will always contain new code and 1064 is not a load balancer. 1066 o Zero, one or two DetNet Destination Options containing sequencing 1067 information for packet replication and duplicate elimination 1068 function (PREF), and/or packet reordering purposes. The DetNet 1069 Destination Option is equivalent to the DetNet Control Word. If 1070 PREF or packet reordering is not needed for the DetNet flow then 1071 no DetNet Destination Option is inserted into the IPv6 header. 1073 A DetNet-aware end station (a host) or an intermediate Detnet node 1074 initiating an (or adding a tunnelling) IPv6 packet is responsible for 1075 setting the Flow Label, adding the optional DetNet Destination 1076 Option(s) for DetNet-s- or DetNet-st-flows, and possibly adding a 1077 routing header such as the segment routing option (e.g., for pre- 1078 defined paths [I-D.ietf-6man-segment-routing-header]). If a routing 1079 header is inserted into the IPv6 packet for DetNet-s- or DetNet-st- 1080 flows then a second instance of the DetNet Destination Option MUST be 1081 added before the routing header (see Section 4.1 of [RFC8200]). 1083 A DetNet-aware end station (a host) or an intermediate node receiving 1084 an IPv6 packet destined to it and containing a DetNet Destination 1085 Option does the appropriate processing of the packet. This may 1086 involve packet duplication and elimination (PREF processing), 1087 terminating a tunnel or delivering the packet to the upper layers/ 1088 Applications. 1090 +---------------------------------+ 1091 | | 1092 | DetNet Flow | 1093 | Payload | 1094 | | 1095 /---------------------------------\ 1096 H Optional DetNet DstOpt Hdr H 1097 \---------------------------------/ 1098 | IPv6 header | 1099 | (with set Flow label) | 1100 +---------------------------------+ 1102 Figure 15: Encapsulation of a native IPv6 DetNet-s- or DetNet-st-flow 1103 without a routing header 1105 Figure 16 illustrates an IPv6 packet for the case where a routing 1106 header has been added into the packet by a DetNet-aware end system 1107 (again assuming DetNet-s- or DetNet-st-flows). Note that the use of 1108 routing header such as the one with the segment routing option is not 1109 mandatory for explicit routes. Similar functionality can be arranged 1110 using other means as well (e.g., using policy routing or layer-2 1111 means). 1113 +---------------------------------+ 1114 | | 1115 | DetNet Flow | 1116 | Payload | 1117 | | 1118 /---------------------------------\ 1119 H DetNet DstOpt Hdr H 1120 \---------------------------------/ 1121 | Routing header | 1122 /---------------------------------\ 1123 H DetNet DstOpt Hdr H 1124 \---------------------------------/ 1125 | IPv6 header | 1126 | (with set Flow label) | 1127 +---------------------------------+ 1129 Figure 16: Encapsulation of a native IPv6 DetNet-s- or DetNet-st- 1130 flows with routing header 1132 IPv6 extension headers can only be inserted by a node that initiated 1133 the IPv6 packet. IPv6 extension headers, except for the Hop-by-Hop 1134 Option headers, can only be processed by an IPv6 node that is 1135 identified by the Destination Address field of the IPv6 header (see 1136 Section 0 of [RFC8200]. Therefore, if a DetNet-aware end system only 1137 inserted the DetNet Destination Option into the IPv6 but e.g., a 1138 DetNet Edge node is configured to enforce an explicit route for the 1139 IPv6 packet using a source routing header, then it has no other 1140 possibility than add an outer tunneling IPv6 header with required 1141 extension headers in it. The processing of IPv6 packets in a DetNet 1142 Edge node is discussed further in Section 7.4.1. 1144 7.2. DetNet destination option 1146 A DetNet flow must carry sequencing information for packet 1147 replication and elimination function (PREF) purposes. This document 1148 specifies a new IPv6 Destination Option: the DetNet Destination 1149 Option for that purpose. The format of the option is illustrated in 1150 Figure 17. 1152 0 1 2 3 1153 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1154 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1155 | TBD1 | 4 | 0 | 28 bit sequence 1156 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1157 number | 1158 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1160 Figure 17: DetNet Destination Option 1162 The Option Type for the DetNet Destination Option is set to TBD1. 1163 [To be removed from the final version of the document: The Option 1164 Type MUST have the two most significant bits set to 10b] 1166 If an IPv6 packet gets dropped due the DetNet Service layer 1167 processing based on the DetNet Destination Option an ICMPv6 packet of 1168 any type MUST NOT be sent back to the source of the packet. 1170 7.3. Flow identification 1172 The DetNet flow identification is based on the IPv6 Flow Label and 1173 the source address combination. The two fields uniquelly identify 1174 the end to end native IPv6 encapsulated DetNet flow. Obviously, the 1175 identification fails if any intermediate node modifies either the 1176 source address or the Flow Label. 1178 Comment #27 SB> See earlier. If there are enough IPv6 addresses to 1179 address video fragments, why not DN flows? Then this problem goes 1180 away. 1182 Discussion: See the earlier comment #25 discussion. If nodes get 1183 their addressies via DHCPv6 basically ruins this mechanism. Also 1184 the assumption for this to work is that the node has a full /64 to 1185 use, which is not always the case. Otherwise the idea is just 1186 fine. 1188 7.4. Service layer considerations 1190 [Editor's note: this section is TBD. It will detail the PREF 1191 functionality.] 1193 o PREF - requires both flow identification and sequence numbering. 1195 o Packet reordeing - requires both flow identification and sequence 1196 numbering. 1198 A DetNet service layer processing can be done at each DetNet node 1199 that matches the IPv6 header's Destination Address. Then, if the 1200 DetNet flow identification provides a positive match for the DetNet 1201 flow that the node has a service layer state installed e.g., for PREF 1202 or packet reordering purposes, further service layer processing takes 1203 place. In a case of PREF or packet reordering that means processing 1204 the DetNet Destination Option for the identified DetNet flow. 1206 7.4.1. Edge node processing 1208 [Editor's note: This is the start of the IPv6 handling text - there 1209 are errors and bad language. The founding assumption is the use of 1210 source routing when intermediate nodes (relays/edges) need to modify 1211 packets. This is due the text in RFC8200 and the fact that without 1212 hph options only routing+dsthdr is usable with intermediates under 1213 strict RFC8200.. ] 1215 [Editor's note: Regrading the source routing and the "example" SRv6 1216 approach. Current text is based on the assumption that intermediates 1217 cannot add/delete extension headers such as the SRv6. That said 1218 adding adding a header implies adding a tunneling outer IPv6 header 1219 and deleting a header implies a tunnel decapsulation. This is not 1220 probably desired due to the involved overhead and to be discussed 1221 whether it is possible/acceptable to just "process" the Application 1222 flow packets.] 1224 For a DetNet Edge node there are several scenarios that involve 1225 modifications to the DetNet flow IPv6 packets. The assumption is 1226 that a DetNet-aware end system has always set the IPv6 header flow 1227 label properly for the flow identification purposes. A DetNet- or 1228 DetNet-t-flow does not include the DetNet Destination Option. 1229 Following cases have been identified: 1231 1. A DetNet App-flow or a DetNet-t-flow packet arrives at an ingress 1232 DetNet Edge node and DetNet service layer functions are done only 1233 at DetNet Edge nodes. Possible explicit routes between edge 1234 nodes are arranged by other than IPv6 specific means. 1236 2. A DetNet App-flow or a DetNet-t-flow packet arrives at an ingress 1237 DetNet Edge node and multiple DetNet Relay nodes may process 1238 DetNet flow packets before reaching an egress DetNet Edge node. 1239 Explicit routes between edge nodes has to be arranged by IPv6 1240 specific means. 1242 3. A DetNet-s- or a DetNet-st-flow packet arrives at an ingress 1243 DetNet Edge node and DetNet service layer functions are done only 1244 at DetNet Edge nodes. Possible explicit routes between edge 1245 nodes are arranged by other than IPv6 specific means. 1247 4. A DetNet-s- or a DetNet-st-flow packet arrives at an ingress 1248 DetNet Edge node and multiple DetNet Relay nodes may process 1249 DetNet flow packets before reaching an egress DetNet Edge node. 1250 Explicit routes between edge nodes has to be arranged by IPv6 1251 specific means. 1253 A generic DetNet IPv6 encapsulation for a DetNet flow packet between 1254 DetNet Edge nodes is shown in Figure 18. Essentially every time an 1255 igress DetNet Edge node has to insert something into the DetNet flow 1256 packet it has to add an outer tunneling IPv6 header, which then 1257 contain possible additional extension headers. 1259 +---------------------------------+ 1260 | | 1261 | DetNet Flow | 1262 | Payload | 1263 | | 1264 +---------------------------------+ 1265 | Optional DetNet DstOpt Hdr (1) | 1266 +---------------------------------+ 1267 | Inner IPv6 header | 1268 | (with set Flow label) (1) | 1269 +---------------------------------+ <--+ 1270 | Optional Routing header | | 1271 +---------------------------------+ | 1272 | Optional DetNet DstOpt Hdr (2) | +-- Added/removed by 1273 +---------------------------------+ | DetNet Edge node 1274 | Outer IPv6 header | | 1275 | (with set Flow label) (2) | | 1276 +---------------------------------+ <--+ 1278 Figure 18: Encapsulation of a DetNet-flow IPv6 packet at the DetNet 1279 Edge node 1281 7.4.1.1. Ingress DetNet Edge node processing 1283 Case 1) MAY require an addition of the DetNet Destination Option if 1284 packet reordering is requested at the egress DetNet Edge node. 1285 Otherwise, no modifications except rewriting the IPv6 header flow 1286 label to the packet is done. If modifications are required then: 1288 o The outer IPv6 header is added with the Source Address set to the 1289 ingress DetNet Edge node address and the Destination Address set 1290 to the egress DetNet Edge node address. 1292 o The flow label of the outer IPv6 header SHOULD be set to a value 1293 maintained by the edge node. 1295 o The DetNet Destination Option with the edge node managed per 1296 DetNet flow sequence number value is inserted into the outer IPv6 1297 header. 1299 Case 2) requires an addition of the DetNet Destination Option unless 1300 neither packet reordeing or PREF is enable at any DetNet Edge/Relay 1301 node. A source routing header has to be added for the explicit route 1302 purposes. An example of the source routing header is the Segment 1303 Routing header. The following modifications to DetNet flow IPv6 1304 packets are required: 1306 o An outer IPv6 header is added with the Source Address set to the 1307 ingress DetNet Edge node address and the Destination Address set 1308 to the egress DetNet Edge node address. 1310 o The flow label of the outer IPv6 header SHOULD be set to a value 1311 maintained by the edge node. 1313 o The DetNet Destination Option with the edge node managed per 1314 DetNet flow sequence number value MAY be inserted into the outer 1315 IPv6 header. 1317 o A source routing header with addresses of those DetNet Relay nodes 1318 that must be traversed is inserted into the outer IPv6 header. 1320 Case 3) ...[Editor's note: is it OK if the sequece number added here 1321 by the edge node has only local significance between the edge nodes 1322 and not end to end between end systems? ] 1324 Case 4) ... 1326 7.4.1.2. Engress DetNet Edge node processing 1328 7.4.2. Relay node processing 1330 TBD. 1332 7.4.3. End system processing 1334 TBD. 1336 7.5. Transport node processing 1338 7.5.1. Congestion protection 1339 7.5.2. Explicit routes 1341 8. Other DetNet data plane considerations 1343 8.1. Class of Service 1345 Class and quality of service, i.e., CoS and QoS, are terms that are 1346 often used interchangeably and confused. In the context of DetNet, 1347 CoS is used to refer to mechanisms that provide traffic forwarding 1348 treatment based on aggregate group basis and QoS is used to refer to 1349 mechanisms that provide traffic forwarding treatment based on a 1350 specific DetNet flow basis. Examples of existing network level CoS 1351 mechanisms include DiffServ which is enabled by IP header 1352 differentiated services code point (DSCP) field [RFC2474] and MPLS 1353 label traffic class field [RFC5462], and at Layer-2, by IEEE 802.1p 1354 priority code point (PCP). 1356 CoS for DetNet flows carried in PWs and MPLS is provided using the 1357 existing MPLS Differentiated Services (DiffServ) architecture 1358 [RFC3270]. Both E-LSP and L-LSP MPLS DiffServ modes MAY be used to 1359 support DetNet flows. The Traffic Class field (formerly the EXP 1360 field) of an MPLS label follows the definition of [RFC5462] and 1361 [RFC3270]. The Uniform, Pipe, and Short Pipe DiffServ tunneling and 1362 TTL processing models are described in [RFC3270] and [RFC3443] and 1363 MAY be used for MPLS LSPs supporting DetNet flows. MPLS ECN MAY also 1364 be used as defined in ECN [RFC5129] and updated by [RFC5462]. 1366 CoS for DetNet flows carried in IPv6 is provided using the standard 1367 differentiated services code point (DSCP) field [RFC2474] and related 1368 mechanisms. The 2-bit explicit congestion notification (ECN) 1369 [RFC3168] field MAY also be used. 1371 One additional consideration for DetNet nodes which support CoS 1372 services is that they MUST ensure that the CoS service classes do not 1373 impact the congestion protection and latency control mechanisms used 1374 to provide DetNet QoS. This requirement is similar to requirement 1375 for MPLS LSRs to that CoS LSPs do not impact the resources allocated 1376 to TE LSPs via [RFC3473]. 1378 8.2. Quality of Service 1380 Quality of Service (QoS) mechanisms for flow specific traffic 1381 treatment typically includes a guarantee/agreement for the service, 1382 and allocation of resources to support the service. Example QoS 1383 mechanisms include discrete resource allocation, admission control, 1384 flow identification and isolation, and sometimes path control, 1385 traffic protection, shaping, policing and remarking. Example 1386 protocols that support QoS control include Resource ReSerVation 1387 Protocol (RSVP) [RFC2205] (RSVP) and RSVP-TE [RFC3209] and [RFC3473]. 1388 The existing MPLS mechanisms defined to support CoS [RFC3270] can 1389 also be used to reserve resources for specific traffic classes. 1391 In addition to explicit routes, and packet replication and 1392 elimination, described in Section 6 above, DetNet provides zero 1393 congestion loss and bounded latency and jitter. As described in 1394 [I-D.ietf-detnet-architecture], there are different mechanisms that 1395 maybe used separately or in combination to deliver a zero congestion 1396 loss service. These mechanisms are provided by the either the MPLS 1397 or IP layers, and may be combined with the mechanisms defined by the 1398 underlying network layer such as 802.1TSN. 1400 A baseline set of QoS capabilities for DetNet flows carried in PWs 1401 and MPLS can provided by MPLS with Traffic Engineering (MPLS-TE) 1402 [RFC3209] and [RFC3473]. TE LSPs can also support explicit routes 1403 (path pinning). Current service definitions for packet TE LSPs can 1404 be found in "Specification of the Controlled Load Quality of 1405 Service", [RFC2211], "Specification of Guaranteed Quality of 1406 Service", [RFC2212], and "Ethernet Traffic Parameters", [RFC6003]. 1407 Additional service definitions are expected in future documents to 1408 support the full range of DetNet services. In all cases, the 1409 existing label-based marking mechanisms defined for TE-LSPs and even 1410 E-LSPs are use to support the identification of flows requiring 1411 DetNet QoS. 1413 QoS for DetNet flows carried in IPv6 MUST be provided locally by the 1414 DetNet-aware hosts and routers supporting DetNet flows. Such support 1415 will leverage the underlying network layer such as 802.1TSN. The 1416 traffic control mechanisms used to deliver QoS for IP encapsulated 1417 DetNet flows are expected to be defined in a future document. From 1418 an encapsulation perspective, and as defined in Section 7, the 1419 combination of the Flow Label together with the IP source address 1420 uniquely identifies a DetNet flow. 1422 Packets that are marked with a DetNet Class of Service value, but 1423 that have not been the subject of a completed reservation, can 1424 disrupt the QoS offered to properly reserved DetNet flows by using 1425 resources allocated to the reserved flows. Therefore, the network 1426 nodes of a DetNet network MUST: 1428 o Defend the DetNet QoS by discarding or remarking (to a non-DetNet 1429 CoS) packets received that are not the subject of a completed 1430 reservation. 1432 o Not use a DetNet reserved resource, e.g. a queue or shaper 1433 reserved for DetNet flows, for any packet that does not carry a 1434 DetNet Class of Service marker. 1436 8.3. Cross-DetNet flow resource aggregation 1438 The ability to aggregate individual flows, and their associated 1439 resource control, into a larger aggregate is an important technique 1440 for improving scaling of control in the data, management and control 1441 planes. This document identifies the traffic identification related 1442 aspects of aggregation of DetNet flows. The resource control and 1443 management aspects of aggregation (including the queuing/shaping/ 1444 policing implications) will be covered in other documents. The data 1445 plane implications of aggregation are independent for PW/MPLS and IP 1446 encapsulated DetNet flows. 1448 DetNet flows transported via MPLS can leverage MPLS-TE's existing 1449 support for hierarchical LSPs (H-LSPs), see [RFC4206]. H-LSPs are 1450 typically used to aggregate control and resources, they may also be 1451 used to provide OAM or protection for the aggregated LSPs. Arbitrary 1452 levels of aggregation naturally falls out of the definition for 1453 hierarchy and the MPLS label stack [RFC3032]. DetNet nodes which 1454 support aggregation (LSP hierarchy) map one or more LSPs (labels) 1455 into and from an H-LSP. Both carried LSPs and H-LSPs may or may not 1456 use the TC field, i.e., L-LSPs or E-LSPs. Such nodes will need to 1457 ensure that traffic from aggregated LSPs are placed (shaped/policed/ 1458 enqueued) onto the H-LSPs in a fashion that ensures the required 1459 DetNet service is preserved. 1461 DetNet flows transported via IP have more limited aggregation 1462 options, due to the available traffic flow identification fields of 1463 the IP solution. One available approach is to manage the resources 1464 associated with a DSCP identified traffic class and to map (remark) 1465 individually controlled DetNet flows onto that traffic class. This 1466 approach also requires that nodes support aggregation ensure that 1467 traffic from aggregated LSPs are placed (shaped/policed/enqueued) in 1468 a fashion that ensures the required DetNet service is preserved. 1470 Comment #38 SB> I am sure we can do better than this with SR, or the 1471 use of routing techniques that map certain addresses to certain 1472 paths. 1474 Discussion: -- 1476 In both the MPLS and IP cases, additional details of the traffic 1477 control capabilities needed at a DetNet-aware node may be covered in 1478 the new service descriptions mentioned above or in separate future 1479 documents. Management and control plane mechanisms will also need to 1480 ensure that the service required on the aggregate flow (H-LSP or 1481 DSCP) are provided, which may include the discarding or remarking 1482 mentioned in the previous sections. 1484 8.4. Bidirectional traffic 1486 Some DetNet applications generate bidirectional traffic. Using MPLS 1487 definitions [RFC5654] there are associated bidirectional flows, and 1488 co-routed bidirectional flows. MPLS defines a point-to-point 1489 associated bidirectional LSP as consisting of two unidirectional 1490 point-to-point LSPs, one from A to B and the other from B to A, which 1491 are regarded as providing a single logical bidirectional transport 1492 path. This would be analogous of standard IP routing, or PWs running 1493 over two reciprocal unidirection LSPs. MPLS defines a point-to-point 1494 co-routed bidirectional LSP as an associated bidirectional LSP which 1495 satisfies the additional constraint that its two unidirectional 1496 component LSPs follow the same path (in terms of both nodes and 1497 links) in both directions. An important property of co-routed 1498 bidirectional LSPs is that their unidirectional component LSPs share 1499 fate. In both types of bidirectional LSPs, resource allocations may 1500 differ in each direction. The concepts of associated bidirectional 1501 flows and co-routed bidirectional flows can be applied to DetNet 1502 flows as well whether IPv6 or MPLS is used. 1504 While the IPv6 and MPLS data planes must support bidirectional DetNet 1505 flows, there are no special bidirectional features with respect to 1506 the data plane other than need for the two directions take the same 1507 paths. Fate sharing and associated vs co-routed bidirectional flows 1508 can be managed at the control level. Note, that there is no stated 1509 requirement for bidirectional DetNet flows to be supported using the 1510 same IPv6 Flow Labels or MPLS Labels in each direction. Control 1511 mechanisms will need to support such bidirectional flows for both 1512 IPv6 and MPLS, but such mechanisms are out of scope of this document. 1513 An example control plane solution for MPLS can be found in [RFC7551]. 1515 8.5. Layer 2 addressing and QoS Considerations 1517 The Time-Sensitive Networking (TSN) Task Group of the IEEE 802.1 1518 Working Group have defined (and are defining) a number of amendments 1519 to IEEE 802.1Q [IEEE8021Q] that provide zero congestion loss and 1520 bounded latency in bridged networks. IEEE 802.1CB [IEEE8021CB] 1521 defines packet replication and elimination functions that should 1522 prove both compatible with and useful to, DetNet networks. 1524 As is the case for DetNet, a Layer 2 network node such as a bridge 1525 may need to identify the specific DetNet flow to which a packet 1526 belongs in order to provide the TSN/DetNet QoS for that packet. It 1527 also will likely need a CoS marking, such as the priority field of an 1528 IEEE Std 802.1Q VLAN tag, to give the packet proper service. 1530 Although the flow identification methods described in IEEE 802.1CB 1531 [IEEE8021CB] are flexible, and in fact, include IP 5-tuple 1532 identification methods, the baseline TSN standards assume that every 1533 Ethernet frame belonging to a TSN stream (i.e. DetNet flow) carries 1534 a multicast destination MAC address that is unique to that flow 1535 within the bridged network over which it is carried. Furthermore, 1536 IEEE 802.1CB [IEEE8021CB] describes three methods by which a packet 1537 sequence number can be encoded in an Ethernet frame. 1539 Ensuring that the proper Ethernet VLAN tag priority and destination 1540 MAC address are used on a DetNet/TSN packet may require further 1541 clarification of the customary L2/L3 transformations carried out by 1542 routers and edge label switches. Edge nodes may also have to move 1543 sequence number fields among Layer 2, PW, and IPv6 encapsulations. 1545 8.6. Interworking between MPLS- and IPv6-based encapsulations 1547 [Editor's note: add considerations for interworking between MPLS- 1548 based and native IPv6-based DetNet encapsuations.] 1550 8.7. IPv4 considerations 1552 [Editor's note: The fact is that there are and will be deployments 1553 using IPv4. Neglecting it entirely is not feasible.] 1555 9. Time synchronization 1557 Comment #39 SB> This section should point the reader to RFC8169 1558 (residence time in MPLS n/w. We need to consider if we need to 1559 introduce the same concept in IP. 1561 Discussion: Agree. For IP we could reference to PTPv2 or v3 over 1562 UDP/IP, since it measures residence time among other things. 1564 [Editor's note: describe a bit of issues and deployment 1565 considerations related to time-synchronization within DetNet. Refer 1566 to DT discussion and the slides that summarize different approaches 1567 and rough synchronization performance numbers. Finally, scope time- 1568 synchronization solution outside data plane.] 1570 When DetNet is used, there is an underlying assumption that the 1571 applicaton(s) require clock synchronization such as the Precision 1572 Time Protocol (PTP) [IEEE1588]. The relay nodes may or may not 1573 utilize clock synchronization in order to provide zero congestion 1574 loss and controlled latency delivery. In either case, there are a 1575 few possible approaches of how synchronization protocol packets are 1576 forwarded and handled by the network: 1578 o PTP packets can be sent either as DetNet flows or as high-priority 1579 best effort packets. Using DetNet for PTP packets requires 1580 careful consideration to prevent unwanted interactions between 1581 clock-synchronized network nodes and the packets that synchronize 1582 the clocks. 1584 o PTP packets are sent as a normal DetNet flow through network nodes 1585 that are not time-synchronized: in this approach PTP traffic is 1586 forwarded as a DetNet flow, and as such it is forwarded in a way 1587 that allows a low delay variation. However, since intermediate 1588 nodes do not take part in the synchronization protocol, this 1589 approach provides a relatively low degree of accuracy. 1591 o PTP with on-path support: in this approach PTP packets are sent as 1592 ordinary or as DetNet flows, and intermediate nodes take part in 1593 the protocol as Transparent Clocks or Boundary Clocks [IEEE1588]. 1594 The on-path PTP support by intermediate nodes provides a higher 1595 degree of accuracy than the previous approach. The actual 1596 accuracy depends on whether all intermediate nodes are PTP- 1597 capable, or only a subset of them. 1599 o Time-as-a-service: in this approach accurate time is provided as- 1600 a-service to the DetNet source and destination, as well as the 1601 intermediate nodes. Since traffic between the source and 1602 destination is sent over a provider network, if the provider 1603 supports time-as-a-service, then accurate time can be provided to 1604 both the source and the destination of DetNet traffic. This 1605 approach can potentially provide the highest degree of accuracy. 1607 It is expected that the latter approach will be the most common one, 1608 as it provides the highest degree of accuracy, and creates a layer 1609 separation between the DetNet data and the synchronization service. 1611 It should be noted that in all four approaches it is not recommended 1612 to use replication and elimination for synchronization packets; the 1613 replication/elimination approach may in some cases reduce the 1614 synchronization accuracy, since the observed path delay will be 1615 bivalent. 1617 Comment #40 SB> I am not sure why we sould not use PREP. We should 1618 explain to the reader. 1620 Discussion: Agree that a this can be opened a bit more in detail. 1621 The issue is explained briefly in the last sentence but it could 1622 be more clear. 1624 10. Management and control considerations 1626 [Editor's note: This section needs to be different for MPLS and IPv6 1627 solutions. Most solutions are technology dependant,] 1629 While management plane and control planes are traditionally 1630 considered separately, from the Data Plane perspective there is no 1631 practical difference based on the origin of flow provisioning 1632 information. This document therefore does not distinguish between 1633 information provided by a control plane protocol, e.g., RSVP-TE 1634 [RFC3209] and [RFC3473], or by a network management mechanisms, e.g., 1635 RestConf [RFC8040] and YANG [RFC7950]. 1637 [Editor's note: This section is a work in progress. discuss here 1638 what kind of enhancements are needed for DetNet and specifically for 1639 PREF and DetNet zero congest loss and latency control. Need to cover 1640 both traffic control (queuing) and connection control (control 1641 plane).] 1643 10.1. MPLS-based data plane 1645 10.1.1. S-Label assignment and distribution 1647 [Editor's note: Outdated and MPLS specific.. and needs more work.] 1649 The DetNet S-Label distribution follows the same mechanisms specified 1650 for XYZ . The details of the control plane protocol solution required 1651 for the label distribution and the management of the label number 1652 space are out of scope of this document. 1654 10.1.2. Explicit routes 1656 [Editor's note: Outdated.. and needs more work.] 1658 [TBD: based on MPLS TE and possibly IPv6 SR] 1660 10.2. IPv6-based data plane 1662 10.2.1. Flow Label assignment and distribution 1664 [Editor's note: Outdated and IPv6 Specific.. and needs more work.] 1666 The IPv6 Flow Label distribution and the label number space are out 1667 of scope of this document. However, it should be noted that the 1668 combination of the IPv6 source address and the IPv6 Flow Label is 1669 assumed to be unique within the DetNet-enabled network. Therefore, 1670 as long as each node is able to assign unique Flow Labels for the 1671 source address(es) it is using the DetNet-enabled network wide flow 1672 identification uniqueness is guaranteed. 1674 10.2.2. Explicit routes 1676 [Editor's note: Outdated.. and needs more work.] 1678 [TBD: What we have there for IPv6 and explicit routes] 1680 10.3. Packet replication and elimination 1682 [Editor's note: Outdated and at the functional level technology 1683 independent.. but needs more work.] 1685 The control plane protocol solution required for managing the PREF 1686 processing is outside the scope of this document. 1688 10.4. Congestion protection and latency control 1690 [TBD] 1692 10.5. Flow aggregation control 1694 [TBD] 1696 11. Security considerations 1698 The security considerations of DetNet in general are discussed in 1699 [I-D.ietf-detnet-architecture] and [I-D.sdt-detnet-security]. Other 1700 security considerations will be added in a future version of this 1701 draft. 1703 12. IANA considerations 1705 TBD. 1707 13. Acknowledgements 1709 The author(s) ACK and NACK. 1711 The following people were part of the DetNet Data Plane Solution 1712 Design Team: 1714 Jouni Korhonen 1716 Janos Farkas 1718 Norman Finn 1719 Balazs Varga 1721 Loa Andersson 1723 Tal Mizrahi 1725 David Mozes 1727 Yuanlong Jiang 1729 Carlos J. Bernardos 1731 The DetNet chairs serving during the DetNet Data Plane Solution 1732 Design Team: 1734 Lou Berger 1736 Pat Thaler 1738 14. References 1740 14.1. Normative references 1742 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1743 Requirement Levels", BCP 14, RFC 2119, 1744 DOI 10.17487/RFC2119, March 1997, 1745 . 1747 [RFC2211] Wroclawski, J., "Specification of the Controlled-Load 1748 Network Element Service", RFC 2211, DOI 10.17487/RFC2211, 1749 September 1997, . 1751 [RFC2212] Shenker, S., Partridge, C., and R. Guerin, "Specification 1752 of Guaranteed Quality of Service", RFC 2212, 1753 DOI 10.17487/RFC2212, September 1997, 1754 . 1756 [RFC2474] Nichols, K., Blake, S., Baker, F., and D. Black, 1757 "Definition of the Differentiated Services Field (DS 1758 Field) in the IPv4 and IPv6 Headers", RFC 2474, 1759 DOI 10.17487/RFC2474, December 1998, 1760 . 1762 [RFC3032] Rosen, E., Tappan, D., Fedorkow, G., Rekhter, Y., 1763 Farinacci, D., Li, T., and A. Conta, "MPLS Label Stack 1764 Encoding", RFC 3032, DOI 10.17487/RFC3032, January 2001, 1765 . 1767 [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition 1768 of Explicit Congestion Notification (ECN) to IP", 1769 RFC 3168, DOI 10.17487/RFC3168, September 2001, 1770 . 1772 [RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V., 1773 and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP 1774 Tunnels", RFC 3209, DOI 10.17487/RFC3209, December 2001, 1775 . 1777 [RFC3270] Le Faucheur, F., Wu, L., Davie, B., Davari, S., Vaananen, 1778 P., Krishnan, R., Cheval, P., and J. Heinanen, "Multi- 1779 Protocol Label Switching (MPLS) Support of Differentiated 1780 Services", RFC 3270, DOI 10.17487/RFC3270, May 2002, 1781 . 1783 [RFC3443] Agarwal, P. and B. Akyol, "Time To Live (TTL) Processing 1784 in Multi-Protocol Label Switching (MPLS) Networks", 1785 RFC 3443, DOI 10.17487/RFC3443, January 2003, 1786 . 1788 [RFC3473] Berger, L., Ed., "Generalized Multi-Protocol Label 1789 Switching (GMPLS) Signaling Resource ReserVation Protocol- 1790 Traffic Engineering (RSVP-TE) Extensions", RFC 3473, 1791 DOI 10.17487/RFC3473, January 2003, 1792 . 1794 [RFC4206] Kompella, K. and Y. Rekhter, "Label Switched Paths (LSP) 1795 Hierarchy with Generalized Multi-Protocol Label Switching 1796 (GMPLS) Traffic Engineering (TE)", RFC 4206, 1797 DOI 10.17487/RFC4206, October 2005, 1798 . 1800 [RFC4385] Bryant, S., Swallow, G., Martini, L., and D. McPherson, 1801 "Pseudowire Emulation Edge-to-Edge (PWE3) Control Word for 1802 Use over an MPLS PSN", RFC 4385, DOI 10.17487/RFC4385, 1803 February 2006, . 1805 [RFC5129] Davie, B., Briscoe, B., and J. Tay, "Explicit Congestion 1806 Marking in MPLS", RFC 5129, DOI 10.17487/RFC5129, January 1807 2008, . 1809 [RFC5462] Andersson, L. and R. Asati, "Multiprotocol Label Switching 1810 (MPLS) Label Stack Entry: "EXP" Field Renamed to "Traffic 1811 Class" Field", RFC 5462, DOI 10.17487/RFC5462, February 1812 2009, . 1814 [RFC6003] Papadimitriou, D., "Ethernet Traffic Parameters", 1815 RFC 6003, DOI 10.17487/RFC6003, October 2010, 1816 . 1818 [RFC6073] Martini, L., Metz, C., Nadeau, T., Bocci, M., and M. 1819 Aissaoui, "Segmented Pseudowire", RFC 6073, 1820 DOI 10.17487/RFC6073, January 2011, 1821 . 1823 [RFC8200] Deering, S. and R. Hinden, "Internet Protocol, Version 6 1824 (IPv6) Specification", STD 86, RFC 8200, 1825 DOI 10.17487/RFC8200, July 2017, 1826 . 1828 14.2. Informative references 1830 [I-D.ietf-6man-segment-routing-header] 1831 Previdi, S., Filsfils, C., Raza, K., Dukes, D., Leddy, J., 1832 Field, B., daniel.voyer@bell.ca, d., 1833 daniel.bernier@bell.ca, d., Matsushima, S., Leung, I., 1834 Linkova, J., Aries, E., Kosugi, T., Vyncke, E., Lebrun, 1835 D., Steinberg, D., and R. Raszuk, "IPv6 Segment Routing 1836 Header (SRH)", draft-ietf-6man-segment-routing-header-08 1837 (work in progress), January 2018. 1839 [I-D.ietf-detnet-architecture] 1840 Finn, N., Thubert, P., Varga, B., and J. Farkas, 1841 "Deterministic Networking Architecture", draft-ietf- 1842 detnet-architecture-04 (work in progress), October 2017. 1844 [I-D.ietf-detnet-dp-alt] 1845 Korhonen, J., Farkas, J., Mirsky, G., Thubert, P., 1846 Zhuangyan, Z., and L. Berger, "DetNet Data Plane Protocol 1847 and Solution Alternatives", draft-ietf-detnet-dp-alt-00 1848 (work in progress), October 2016. 1850 [I-D.sdt-detnet-security] 1851 Mizrahi, T., Grossman, E., Hacker, A., Das, S., 1852 "Deterministic Networking (DetNet) Security 1853 Considerations, draft-sdt-detnet-security, work in 1854 progress", 2017. 1856 [IEEE1588] 1857 IEEE, "IEEE 1588 Standard for a Precision Clock 1858 Synchronization Protocol for Networked Measurement and 1859 Control Systems Version 2", 2008. 1861 [IEEE8021CB] 1862 Finn, N., "Draft Standard for Local and metropolitan area 1863 networks - Seamless Redundancy", IEEE P802.1CB 1864 /D2.1 P802.1CB, December 2015, 1865 . 1868 [IEEE8021Q] 1869 IEEE 802.1, "Standard for Local and metropolitan area 1870 networks--Bridges and Bridged Networks (IEEE Std 802.1Q- 1871 2014)", 2014, . 1873 [RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S. 1874 Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1 1875 Functional Specification", RFC 2205, DOI 10.17487/RFC2205, 1876 September 1997, . 1878 [RFC3985] Bryant, S., Ed. and P. Pate, Ed., "Pseudo Wire Emulation 1879 Edge-to-Edge (PWE3) Architecture", RFC 3985, 1880 DOI 10.17487/RFC3985, March 2005, 1881 . 1883 [RFC5654] Niven-Jenkins, B., Ed., Brungard, D., Ed., Betts, M., Ed., 1884 Sprecher, N., and S. Ueno, "Requirements of an MPLS 1885 Transport Profile", RFC 5654, DOI 10.17487/RFC5654, 1886 September 2009, . 1888 [RFC7551] Zhang, F., Ed., Jing, R., and R. Gandhi, Ed., "RSVP-TE 1889 Extensions for Associated Bidirectional Label Switched 1890 Paths (LSPs)", RFC 7551, DOI 10.17487/RFC7551, May 2015, 1891 . 1893 [RFC7950] Bjorklund, M., Ed., "The YANG 1.1 Data Modeling Language", 1894 RFC 7950, DOI 10.17487/RFC7950, August 2016, 1895 . 1897 [RFC8040] Bierman, A., Bjorklund, M., and K. Watsen, "RESTCONF 1898 Protocol", RFC 8040, DOI 10.17487/RFC8040, January 2017, 1899 . 1901 Appendix A. Example of DetNet data plane operation 1903 [Editor's note: Add a simplified example of DetNet data plane and how 1904 labels etc work in the case of MPLS-based PSN and utilizing PREF. 1905 The figure is subject to change depending on the further DT decisions 1906 on the label handling..] 1908 Appendix B. Example of pinned paths using IPv6 1910 TBD. 1912 Authors' Addresses 1914 Jouni Korhonen (editor) 1915 Nordic Semiconductor 1917 Email: jouni.nospam@gmail.com 1919 Loa Andersson 1920 Huawei 1922 Email: loa@pi.nu 1924 Yuanlong Jiang 1925 Huawei 1927 Email: jiangyuanlong@huawei.com 1929 Norman Finn 1930 Huawei 1931 3101 Rio Way 1932 Spring Valley, CA 91977 1933 USA 1935 Email: norman.finn@mail01.huawei.com 1937 Balazs Varga 1938 Ericsson 1939 Konyves Kalman krt. 11/B 1940 Budapest 1097 1941 Hungary 1943 Email: balazs.a.varga@ericsson.com 1944 Janos Farkas 1945 Ericsson 1946 Konyves Kalman krt. 11/B 1947 Budapest 1097 1948 Hungary 1950 Email: janos.farkas@ericsson.com 1952 Carlos J. Bernardos 1953 Universidad Carlos III de Madrid 1954 Av. Universidad, 30 1955 Leganes, Madrid 28911 1956 Spain 1958 Phone: +34 91624 6236 1959 Email: cjbc@it.uc3m.es 1960 URI: http://www.it.uc3m.es/cjbc/ 1962 Tal Mizrahi 1963 Marvell 1964 6 Hamada st. 1965 Yokneam 1966 Israel 1968 Email: talmi@marvell.com 1970 Lou Berger 1971 LabN Consulting, L.L.C. 1973 Email: lberger@labn.net