idnits 2.17.1 draft-ietf-detnet-dp-sol-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (March 22, 2018) is 2227 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'TBD' is mentioned on line 1439, but not defined == Unused Reference: 'RFC6073' is defined on line 1568, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-6man-segment-routing-header' is defined on line 1585, but no explicit reference was found in the text == Outdated reference: A later version (-26) exists of draft-ietf-6man-segment-routing-header-09 == Outdated reference: A later version (-13) exists of draft-ietf-detnet-architecture-04 Summary: 0 errors (**), 0 flaws (~~), 6 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 DetNet J. Korhonen, Ed. 3 Internet-Draft Nordic 4 Intended status: Standards Track L. Andersson 5 Expires: September 23, 2018 Y. Jiang 6 N. Finn 7 Huawei 8 B. Varga 9 J. Farkas 10 Ericsson 11 CJ. Bernardos 12 UC3M 13 T. Mizrahi 14 Marvell 15 L. Berger 16 LabN 17 March 22, 2018 19 DetNet Data Plane Encapsulation 20 draft-ietf-detnet-dp-sol-04 22 Abstract 24 This document specifies Deterministic Networking data plane 25 encapsulation solutions. The described data plane solutions can be 26 applied over either IP or MPLS Packet Switched Networks. 28 Status of This Memo 30 This Internet-Draft is submitted in full conformance with the 31 provisions of BCP 78 and BCP 79. 33 Internet-Drafts are working documents of the Internet Engineering 34 Task Force (IETF). Note that other groups may also distribute 35 working documents as Internet-Drafts. The list of current Internet- 36 Drafts is at https://datatracker.ietf.org/drafts/current/. 38 Internet-Drafts are draft documents valid for a maximum of six months 39 and may be updated, replaced, or obsoleted by other documents at any 40 time. It is inappropriate to use Internet-Drafts as reference 41 material or to cite them other than as "work in progress." 43 This Internet-Draft will expire on September 23, 2018. 45 Copyright Notice 47 Copyright (c) 2018 IETF Trust and the persons identified as the 48 document authors. All rights reserved. 50 This document is subject to BCP 78 and the IETF Trust's Legal 51 Provisions Relating to IETF Documents 52 (https://trustee.ietf.org/license-info) in effect on the date of 53 publication of this document. Please review these documents 54 carefully, as they describe your rights and restrictions with respect 55 to this document. Code Components extracted from this document must 56 include Simplified BSD License text as described in Section 4.e of 57 the Trust Legal Provisions and are provided without warranty as 58 described in the Simplified BSD License. 60 Table of Contents 62 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 63 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 64 2.1. Terms used in this document . . . . . . . . . . . . . . . 4 65 2.2. Abbreviations . . . . . . . . . . . . . . . . . . . . . . 5 66 3. Requirements language . . . . . . . . . . . . . . . . . . . . 6 67 4. DetNet data plane overview . . . . . . . . . . . . . . . . . 6 68 4.1. DetNet data plane encapsulation requirements . . . . . . 8 69 4.2. Packet replication and elimination considerations . . . . 10 70 4.3. Packet reordering considerations . . . . . . . . . . . . 10 71 5. DetNet encapsulation . . . . . . . . . . . . . . . . . . . . 10 72 5.1. End-system specific considerations . . . . . . . . . . . 10 73 5.2. DetNet domain specific considerations . . . . . . . . . . 12 74 5.2.1. DetNet Bridging Service . . . . . . . . . . . . . . . 13 75 5.2.2. DetNet Routing Service . . . . . . . . . . . . . . . 14 76 5.3. DetNet Inter-Working Function (DN-IWF) . . . . . . . . . 17 77 5.3.1. Networks with multiple technology segments . . . . . 17 78 5.3.2. DN-IWF related considerations . . . . . . . . . . . . 18 79 6. MPLS-based DetNet data plane solution . . . . . . . . . . . . 19 80 6.1. DetNet specific packet fields . . . . . . . . . . . . . . 19 81 6.2. Data plane encapsulation . . . . . . . . . . . . . . . . 19 82 6.3. DetNet control word . . . . . . . . . . . . . . . . . . . 20 83 6.4. Flow identification . . . . . . . . . . . . . . . . . . . 21 84 6.5. Service layer considerations . . . . . . . . . . . . . . 21 85 6.5.1. Edge node processing . . . . . . . . . . . . . . . . 22 86 6.5.2. Relay node processing . . . . . . . . . . . . . . . . 23 87 6.5.3. End system processing . . . . . . . . . . . . . . . . 25 88 6.6. Transport node considerations . . . . . . . . . . . . . . 25 89 6.6.1. Congestion protection . . . . . . . . . . . . . . . . 25 90 6.6.2. Explicit routes . . . . . . . . . . . . . . . . . . . 25 91 7. Simplified IP based DetNet data plane solution . . . . . . . 25 92 8. Other DetNet data plane considerations . . . . . . . . . . . 25 93 8.1. Class of Service . . . . . . . . . . . . . . . . . . . . 25 94 8.2. Quality of Service . . . . . . . . . . . . . . . . . . . 26 95 8.3. Cross-DetNet flow resource aggregation . . . . . . . . . 27 96 8.4. Bidirectional traffic . . . . . . . . . . . . . . . . . . 28 97 8.5. Layer 2 addressing and QoS Considerations . . . . . . . . 29 98 8.6. Interworking between MPLS- and IPv6-based encapsulations 29 99 8.7. IPv4 considerations . . . . . . . . . . . . . . . . . . . 30 100 9. Time synchronization . . . . . . . . . . . . . . . . . . . . 30 101 10. Management and control considerations . . . . . . . . . . . . 31 102 10.1. MPLS-based data plane . . . . . . . . . . . . . . . . . 32 103 10.1.1. S-Label assignment and distribution . . . . . . . . 32 104 10.1.2. Explicit routes . . . . . . . . . . . . . . . . . . 32 105 10.2. IPv6-based data plane . . . . . . . . . . . . . . . . . 32 106 10.2.1. Flow Label assignment and distribution . . . . . . . 32 107 10.2.2. Explicit routes . . . . . . . . . . . . . . . . . . 32 108 10.3. Packet replication and elimination . . . . . . . . . . . 32 109 10.4. Congestion protection and latency control . . . . . . . 33 110 10.5. Flow aggregation control . . . . . . . . . . . . . . . . 33 111 11. Security considerations . . . . . . . . . . . . . . . . . . . 33 112 12. IANA considerations . . . . . . . . . . . . . . . . . . . . . 33 113 13. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 33 114 14. References . . . . . . . . . . . . . . . . . . . . . . . . . 34 115 14.1. Normative references . . . . . . . . . . . . . . . . . . 34 116 14.2. Informative references . . . . . . . . . . . . . . . . . 36 117 Appendix A. Example of DetNet data plane operation . . . . . . . 37 118 Appendix B. Example of pinned paths using IPv6 . . . . . . . . . 38 119 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 38 121 1. Introduction 123 Deterministic Networking (DetNet) is a service that can be offered by 124 a network to DetNet flows. DetNet provides these flows extremely low 125 packet loss rates and assured maximum end-to-end delivery latency. 126 General background and concepts of DetNet can be found in 127 [I-D.ietf-detnet-architecture]. 129 This document specifies the DetNet data plane and the on-wire 130 encapsulation of DetNet flows. The specified encapsulation provides 131 the building blocks to enable the DetNet service layer functions and 132 allow flow identification as described in the DetNet Architecture. 133 Two data plane definitions are given. 135 1. MPLS-based: The enacapsulation resembles PseudoWires (PW) with an 136 MPLS Packet Switched Network (PSN) [RFC3985][RFC4385]. 138 2. Native-IP-based: The encapsulating protocol is IPv6 and the 139 solution relies on IP header fields, existing and DetNet specific 140 IPv6 eaxtension header options [RFC8200]. 142 [Editor's note: MPLS- and IPv6-based solutions are likely to be 143 split into different documents.] 145 It is worth noting that while MPLS-based solution can transport IP 146 packets a native-IP solution is meant for deployments where the 147 DetNet service layer functions are provided at the IP-layer rather 148 than the underlying transport network. The primary reason for this 149 is the benefit gained by enabling the use of a normal application 150 stack, where transport protocols such as TCP or UDP are directly 151 encapsulated in IP. 153 The DetNet transport layer functionality that provides congestion 154 protection for DetNet flows is assumed to be in place in a DetNet 155 node. 157 Furthermore, this document also describes how DetNet flows are 158 identified, how a DetNet Relay/Edge/Transit nodes work, and how the 159 Packet Replication and Elimination function (PREF) is implemented 160 with the two data plane solutions. 162 This document does not define the associated control plane functions, 163 or Operations, Administration, and Maintenance (OAM). It also does 164 not specify traffic handling capabilities required to deliver 165 congestion protection and latency control for DetNet flows at the 166 DetNet transport layer. 168 2. Terminology 170 2.1. Terms used in this document 172 This document uses the terminology established in the DetNet 173 architecture [I-D.ietf-detnet-architecture] and the DetNet Data Plane 174 Solution Alternatives [I-D.ietf-detnet-dp-alt]. 176 T-Label A label used to identify the LSP used to transport a 177 DetNet flow across an MPLS PSN, e.g., a hop-by-hop 178 label used between label switching routers (LSR). 180 S-Label A DetNet "service" label that is used between DetNet 181 nodes that implment also the DetNet service layer 182 functions. An S-Label is also used to identify a 183 DetNet flow at DetNet service layer. 185 Flow Label IPv6 header field that is used to identify a DetNet 186 flow (together with the source IP address field). 188 Local-ID A DetNet Edge and Relay node internal construct that 189 uniquely identifies a DetNet flow within a node and 190 never appear on-wire. It may be used to select proper 191 forwarding and/or DetNet specific service function. 193 PREF A Packet Replication and Elimination Function (PREF) 194 does the replication and elimination processing of 195 DetNet flow packets in edge or relay nodes. The 196 replication function is essentially the existing 1+1 197 protection mechanism. The elimination function reuses 198 and extends the existing duplicate detection mechanism 199 to operate over multiple (separate) DetNet member flows 200 of a DetNet compound flow. 202 DetNet Control Word A control word used for sequencing and 203 identifying duplaicate packets at the DetNet service 204 layer. 206 2.2. Abbreviations 208 The following abbreviations used in this document: 210 AC Attachment Circuit. 212 CE Customer Edge equipment. 214 CoS Class of Service. 216 CW Control Word. 218 d-CW DetNet Control Word. 220 DetNet Deterministic Networking. 222 DF DetNet Flow. 224 L2VPN Layer 2 Virtual Private Network. 226 LSR Label Switching Router. 228 MPLS Multiprotocol Label Switching. 230 MPLS-TP Multiprotocol Label Switching - Transport Profile. 232 MS-PW Multi-Segment PseudoWire (MS-PW). 234 NSP Native Service Processing. 236 OAM Operations, Administration, and Maintenance. 238 PE Provider Edge. 240 PREF Packet Replication and Elimination Function. 242 PSN Packet Switched Network. 244 PW PseudoWire. 246 QoS Quality of Service. 248 TSN Time-Sensitive Network. 250 3. Requirements language 252 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 253 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 254 document are to be interpreted as described in [RFC2119]. 256 4. DetNet data plane overview 258 This document describes how to use IP and/or MPLS to support a data 259 plane method of flow identification and packet formwarding over 260 layer-3. Two different cases are covered: (i) the inter-connect 261 scenario, in which IEEE802.1 TSN is routed over a layer-3 network 262 (i.e., to enlarge the layer-2 domain), and (ii) native connectivity 263 between DetNet-aware end systems. 265 Figure 1 illustrates how DetNet can provide services for IEEE 266 802.1TSN end systems over a DetNet enabled network. The edge nodes 267 insert and remove required DetNet data plane encapsulation. The 'X' 268 in the edge and relay nodes represents a potential DetNet flow packet 269 replication and elimination point. This conceptually parallels L2VPN 270 services, and could leverage existing related solutions as discussed 271 below. 273 TSN |<---------- End to End DetNet Service ------>| TSN 274 Service | Transit Transit | Service 275 TSN (AC) | |<-Tunnel->| |<-Tnl->| | (AC) TSN 276 End | V V 1 V V 2 V V | End 277 System | +--------+ +--------+ +--------+ | System 278 +---+ | | E1 |==========| R1 |=======| E2 | | +---+ 279 | |--|----|._X_....|..DetNet..|.._ _...|..DF3..|...._X_.|---|---| | 280 |CE1| | | \ | Flow 1 | X | | / | | |CE2| 281 | | | \_.|...DF2....|._/ \_..|..DF4..|._/ | | | 282 +---+ | |==========| |=======| | +---+ 283 ^ +--------+ +--------+ +--------+ ^ 284 | Edge Node Relay Node Edge Node | 285 | | 286 |<----- Emulated Time Sensitive Networking (TSN) Service ---->| 288 Figure 1: IEEE 802.1TSN over DetNet 290 Figure 2 illustrates how end to end MPLS-based DetNet service can be 291 provided. In this case, the end systems are able to send and receive 292 DetNet flows. For example, an end system sends data encapsulated in 293 MPLS. Like earlier the 'X' in the end systems, edge and relay nodes 294 represents potential DetNet flow packet replication and elimination 295 points. Here the relay nodes may change the underlying transport, 296 for example tunneling IP over MPLS, or simply interconnect network 297 segments. 299 DetNet DetNet 300 Service Transit Transit Service 301 DetNet | |<-Tnl->| |<-Tnl->| | DetNet 302 End | V 1 V V 2 V | End 303 System | +--------+ +--------+ +--------+ | System 304 +---+ | | R1 |=======| R2 |=======| R3 | | +---+ 305 | X...DFa...|._X_....|..DF1..|.__ ___.|..DF3..|...._X_.|.DFa..|.X | 306 |CE1|========| \ | | X | | / |======|CE2| 307 | | | | \_.|..DF2..|._/ \__.|..DF4..|._/ | | | | 308 +---+ | |=======| |=======| | +---+ 309 ^ +--------+ +--------+ +--------+ ^ 310 | Relay Node Relay Node Relay Node | 311 | | 312 |<--------------- End to End DetNet Service -------------->| 314 Figure 2: MPLS-Based Native DetNet 316 Figure 3 illustrates how end to end IP-based DetNet service can be 317 provided. In this case, the end systems are able to send and receive 318 DetNet flows. [Editor's note: TBD] 319 NOTE: This figures is TBD 321 DetNet DetNet 322 Service Transit Transit Service 323 DetNet | |<-Tnl->| |<-Tnl->| | DetNet 324 End | V 1 V V 2 V | End 325 System | +--------+ +--------+ +--------+ | System 326 +---+ | | R1 |=======| R2 |=======| R3 | | +---+ 327 | X...DFa...| | | | | | .|.X | 328 | H1|========| | | | | |======| H2| 329 | | | | | | | | | | | | 330 +---+ | |=======| |=======| | +---+ 331 ^ +--------+ +--------+ +--------+ ^ 332 | Relay Node Relay Node Relay Node | 333 | | 334 |<--------------- End to End DetNet Service -------------->| 336 Figure 3: IP-Based Native DetNet 338 4.1. DetNet data plane encapsulation requirements 340 Two major groups of scenarios can be distinguished which require flow 341 identification during transport: 343 1. DetNet function related scenarios: 345 * Congestion protection and latency control: usage of allocated 346 resources (queuing, policing, shaping). 348 * Explicit routes: select/apply the flow specific path. 350 * Service protection: recognize DetNet compound and member flows 351 for replication an elimination. 353 Comment #12 I am not sure whether the correct architectural 354 construct is flow or flow group. Flow suggests that sharing/ 355 aggregation is not allowed but whether this is allowed or not 356 is an application specific issue. 358 Discussion: Agree that a flow group would be a better 359 characterization. 361 Comment #13 I think that there needs to be some clarification as 362 to whether FG is is understood by the DN system exclusively or 363 whether there is an expectation that it is understood by the 364 underlay. 366 Discussion: Agree that more detail is needed here. DetNet aware 367 nodes need to understand flow groups. Underlay needs to be 368 aware of flow groups at the resource allocation level. 370 2. OAM function related scenarios: 372 * troubleshooting (e.g., identify misbehaving flows, etc.) 374 * recognize flow(s) for analytics (e.g., increase counters, 375 etc.) 377 * correlate events with flows (e.g., volume above threshold, 378 etc.) 380 * etc. 382 Each DetNet node (edge, relay and transit) use an internal/ 383 implementation specific local-ID of the DetNet-(compound)-flow in 384 order to accomplish its role during transport. Recognizing the 385 DetNet flow is more relaxed for edge and relay nodes, as they are 386 fully aware of both the DetNet service and transport layers. The 387 primary DetNet role of intermediate transport nodes is limited to 388 ensuring congestion protection and latency control for the above 389 listed DetNet functions. 391 The DetNet data plane allows for the aggregation of DetNet flows, 392 e.g., via MPLS hierarchical LSPs, to improved scaling. When DetNet 393 flows are aggregated, transit nodes may have limited ability to 394 provide service on per-flow DetNet identifiers. Therefore, 395 identifying each individual DetNet flow on a transit node may not be 396 achieved in some network scenarios, but DetNet service can still be 397 assured in these scenarios through resource allocation and control. 399 Comment #14 You could introduce the concept of a flow group 400 identified into the packet. You may also include a flow id at a 401 lower layer. 403 Discussion: Agree on the identification properties. Adding a 404 specific id into actual on-wire formats is not necessarily needed. 406 On each DetNet node dealing with DetNet flows, an internal local-ID 407 is assumed to determine what local operation a packet goes through. 408 Therefore, local-IDs has to be unique on each edge and relay nodes. 409 Local-ID is unambiguously bound to the DetNet flow. 411 4.2. Packet replication and elimination considerations 413 DetNet service layer introduces packet replication and elimination 414 functionality (PREF) for use in DetNet edge and relay node and end 415 system packet processing. PREF MAY be enabled in a DetNet node and 416 the required processing is only applied to packets with a positive 417 flow identification at the DetNet service layer. PREF utilizes a 418 sequence number carried within a DetNet flow packets. 420 At a DetNet node level the output of the PREF elimination function is 421 always a single packet. The output of the PREF replication function 422 at a DetNet node level is always one or more packets (i.e., 1:M 423 replication). The replicated packets MUST share the same d-CW i.e., 424 the sequence number is the same for each member flow of the compound 425 flow. The location and mechanism on the packet processing pipeline 426 used for replication is implementation specific. 428 The complex part of the DetNet PREF processing is tracking the 429 history of received packets for multiple DetNet member flows. These 430 ingress DetNet member flows (to a node) MUST have the same local-ID 431 if they belong to the same DetNet (compound) flow and share the same 432 sequence number counter and the history information. The location of 433 the packet elimination on the packet processing pipeline is 434 implementation specific. 436 4.3. Packet reordering considerations 438 DetNet service layer introduces also packet reordering functionality 439 for use in DetNet edge and relay node and end system packet 440 processing. The reordering functionality MAY be enabled in a DetNet 441 node. The reordeing functionality relies on a presence of sequence 442 numbers in a DetNet (compound) flows. The reordering processing is 443 only applied to packets with a positive flow identification at the 444 DetNet service layer. 446 5. DetNet encapsulation 448 5.1. End-system specific considerations 450 Data-flows requiring DetNet service are generated and terminated on 451 end-systems. Encapsulation depends on application and its 452 preferences. In a DetNet (or even a TSN) domain the DN (TSN) 453 functions use at most two flow parameters, namely Flow-ID and 454 Seq.Number. However, an application may exchange further flow 455 related parameters (e.g., time-stamp), which are not considered by DN 456 functions. 458 Two types of end-systems are distinguished: 460 o L3 (IP) end-system: application over L3 462 o L2 (Ethernet) end-system: application directly over L2 464 In case of Ethernet end-systems the application data is encapsulated 465 directly in L2. From the DN domain perspective no upper layer 466 protocols are visible. The Data-flow uses only Ethernet tag(s) and 467 further flow specific parameters (if needed) are hidden inside the 468 PDU. 470 The IP end-system scenario is different. Data-flows are encapsulated 471 directly in L3 (i.e., IP) and the application may use further upper 472 layer protocols (e.g., RTP). Many valid combinations exist, and it 473 may be application specific how the IP header fields are used. Also, 474 usage of further upper layer protocols depends on application 475 requirements (e.g., time-stamp). Some examples for encoding of Flow- 476 ID or Seq.Number attributes: IP address, IPv6-Flow-label, L4 ports, 477 RTP-header, etc. 479 As a general rule, DetNet domains MUST be capable to forward any 480 Data-flows and the DetNet domain MUST NOT intend to mandate end- 481 system encapsulation format. 483 Furthermore, no application-level-proxy function is envisioned inside 484 the DetNet domain, so end-systems peer with end-systems using the 485 same application encapsulation format (see figure below): 487 o L3 end-systems peer with L3 end-systems and 489 o L2 end-systems peer with L2 end-systems 490 +-----+ 491 | X | +-----+ 492 +-----+ | X | 493 | Eth | ________ +-----+ 494 +-----+ _____ / \ | Eth | 495 \ / \__/ \___ +-----+ 496 \ / \ / 497 0======== tunnel-1 ========0_ 498 | \ 499 \ | 500 0========= tunnel-2 =========0 501 / \ __/ \ 502 +-----+ \__ DetNet domain / \ 503 | X | \ __ / +-----+ 504 +-----+ \_______/ \_____/ | X | 505 | IP | +-----+ 506 +-----+ | IP | 507 +-----+ 509 Figure 4: End-systems and the DetNet domain 511 5.2. DetNet domain specific considerations 513 From connection type perspective three scenarios are distinguished: 515 1. Directly attached: end-system is directly connected to an edge 516 node 518 2. Indirectly attached: end-system is behind a (L2-TSN / L3-DetNet) 519 sub-net 521 3. DN integrated: end-system is part of the DetNet domain 523 L3 end-systems may use any of these connection types, however L2 end- 524 systems may use only the first two (directly or indirectly attached). 525 DetNet domain MUST allow communication between any end-systems of the 526 same type (L2-L2, L3-L3), independent of their connection type and 527 DetNet capability. However directly attached and indirectly attached 528 end-systems have no knowledge about the DetNet domain and its 529 encapsulation format at all. See the figure below for L3 end-system 530 scenarios. 532 ____+----+ 533 +----+ _____ / | ES3| 534 | ES1|____ / \__/ +----+___ 535 +----+ \ / \ 536 + | 537 ____ \ _/ 538 +----+ __/ \ +__ DetNet domain / 539 | ES2|____/ L2/L3 |___/ \ __ __/ 540 +----+ \_______/ \_______/ \___/ 542 Figure 5: Connection types of L3 end-systems 544 5.2.1. DetNet Bridging Service 546 The simplest DetNet service is to provide bridging (i.e., tunneling 547 for L2), where the connected hosts are in the same broadcast (BC) 548 domain. Forwarding over the DetNet domain is based on L2 (MAC) 549 addresses (i.e. dst-MAC), so L2 headers MUST be kept. For both IP 550 and MPLS PSN a DetNet specific tunnel encapsulation MUST be 551 introduced. 553 +------+ 554 | X | 555 +------+ +------+ 556 | X | | IP | 557 +------+ +------+ 558 End+system | L2 | | L2 | 559 +-----+======+-+======+--+======+-+======++ 560 DetNet tunnel | IP | | MPLS | 561 +------+ +------+ 562 | L2 | | L2 | 563 +------+ +------+ 565 Examples 567 +------+ +------+ 568 | X | | X | 569 +------+ +------+ +------+ +------+ 570 | X | | X | | IP | | IP | 571 +------+ +------+ +------+ +------+ 572 | L2 | | L2 | | L2 | | L2 | 573 ++======+-+======+--+======+-+======++ 574 | IP | | MPLS | | IP | | MPLS | 575 +------+ +------+ +------+ +------+ 576 | L2 | | L2 | | L2 | | L2 | 577 +------+ +------+ +------+ +------+ 579 Figure 6: Encapsulation format for DetNet Bridging 581 As shown on the figure both L2 and L3 end-systems can be served by 582 such a DetNet Bridging service. 584 5.2.2. DetNet Routing Service 586 DetNet Routing service provides routing, therefore available only for 587 L3 hosts that are in different BC domains. Forwarding over the 588 DetNet domain is based on L3 (IP) addresses (i.e. dst-IP). 590 5.2.2.1. MPLS PSN 592 In case of an MPLS PSN at the ingress/egress (i.e., PE nodes of 593 DetNet domain) the IP packets are encapsulated in MPLS. The data- 594 flow IP header MUST be preserved as-is. 596 +------+ +------+ 597 | X | | X | 598 +------+ +------+ 599 End+system | IP | | IP | 600 -----+------+-------+======+--- --+======+-- 601 DetNet | MPLS | | MPLS | 602 +------+ +------+ 603 | L2 | | L2 | 604 +------+ +------+ 606 Figure 7: Encapsulation format for DetNet Routing in MPLS PSN for L3 607 end-systems 609 5.2.2.2. IP PSN 611 In case of an IP PSN the same tunneling concept can be used as for an 612 MPLS PSN, but the tunnel is constructed by a new IP header (and 613 possible upper layer fields). The data-flow IP header MUST be 614 preserved as-is. 616 +------+ +------+ 617 | X | | X | 618 +======+ +------+ 619 End-system | IP | | IP | 620 -----+------+-------+======+--- --+======+-- 621 DetNet | IP | | IP | 622 +------+ +------+ 623 | L2 | | L2 | 624 +------+ +------+ 626 Figure 8: Encapsulation format for DetNet Routing in IP PSN for L3 627 end-systems 629 DetNet IP header contains the IP addresses of the ingress/egress PE 630 nodes of DetNet domain. The End-system IP header contains the IP 631 addresses of the end-systems. 633 Note: In case of IP PSN one may consider avoiding the additional IP 634 encapsulation, however there are many issues with such an approach. 635 First, the DetNet nodes MUST be able to extract from the IP header 636 (and maybe upper layers) the attributes required by DetNet functions 637 (i.e. Flow-ID, Seq.Number). The challenge is that encoding of those 638 attributes may be application specific, so DetNet nodes MUST be 639 prepared to handle all application specific formats. Second, adding 640 further fields (e.g., explicit path information) to an existing IP 641 header may be impossible (e.g., due to security/encryption). 643 Furthermore, DetNet domain IP-header format may collide with IP- 644 header format used by the source of a flow. Implementing such an 645 approach requires that source encapsulation is in-line with DetNet 646 domain encapsulation format, however we do not intend to mandate end- 647 systems' encapsulation format (see former text: As a general rule, 648 DetNet domains MUST be capable to forward any Data-flows and the 649 DetNet domain MUST NOT intend to mandate end-system encapsulation 650 format). 652 Another approach with IP PSN can be based on MPLS over IP [RFC4023] 653 and/or MPLS over UDP/IP [RFC7510]. In this case the encapsulations 654 over the PSNs were the same i.e., basically the DetNet MPLS-based 655 data plane encapsulation as described in Section 6.2 for both IP and 656 MPLS PSNs. 658 [Editor's node: this approach was actually proposed earlier in 659 draft-dt-detnet-dp-sol-00 in a PseudoWire context for IP PSN] 661 5.2.2.3. Simplified IP Service 663 In this case there is no "tunneling" below the DetNet Service, but 664 the DetNet Service flows are mapped to each link / sub net using its 665 technology specific methods. The DetNet IP header containes the IP 666 address destination DetNet end system. The data-flow IP header MUST 667 be preserved as-is. 669 This solution provides end to end DetNet service consisting of 670 congestion protection and latency control and the rouse allocation 671 (queuing, policing, shaping) done using the underlying link / sub net 672 specific mechanisms. Compared to previously described DetNet routing 673 services, the service protections (packet replication and packet 674 emilination functions) and not provided end to end, but per 675 underlying layer-2 link / sub net. 677 +------+ +------+ 678 | X | | X | 679 +======+ +------+ 680 End-system | IP | | IP | 681 -----+------+-------+======+--- --+======+-- 682 DetNet |L2/SbN| |L2/SbN| 683 +------+ +------+ 685 Figure 9: Encapsulation of DetNet Routing in simplified IP service L3 686 end-systems 688 Note: the DetNet Service Flow MUST be mapped to the link / sub net 689 specific resources using an underlying system specific means. This 690 implies each DetNet aware node on path MUST look into the transported 691 DetNet Service Flow packet and utilize e.g., a five tuple to find out 692 the required mapping in a node. As noted earlier, the Service 693 Protection is done within each link / sub net independently using the 694 domain specific mechanisms (due the lack of a unified end to end 695 sequencing information that would be available for intermediate 696 nodes). If end to end service protection is desired that can be 697 implemented, for example, by the DetNet end systems using Layer-4 698 transport protocols or application protocols. However, these are out 699 of scope of this document. 701 [Editor's note: the service protection to be clarified further.] 703 5.3. DetNet Inter-Working Function (DN-IWF) 705 5.3.1. Networks with multiple technology segments 707 There are network scenarios, where the DetNet domain contains 708 multiple technology segments (IP, MPLS) and all those segments are 709 under the same administrative control (see Figure 10). Furthermore, 710 DetNet nodes may be interconnected via TSN segments. 712 An important aspect of DetNet network design is placement of DetNet 713 functions across the domain. Designs based on segment-by-segment 714 optimization can provide only suboptimal solutions. In order to 715 achieve global optimum Inter-Working Functions (DN-IWF) can be placed 716 at segment border nodes, which stich together DetNet flows across 717 connected segments. 719 DN-IWF may ensure that flow attributes are correlated across segment 720 borders. For example, there are two DetNet functions which require 721 Seq.Numbers: (1) Elimination: removes duplications from flows and (2) 722 IOD: ensures in-order-delivery of packet in a flow. Stitching flows 723 together and correlating attributes means for example that 724 replication of packets can happen in one segment and elimination of 725 duplicates in a different one. 727 ______ 728 _____ / \__ 729 ____ / \__/ \___ ______ 730 +----+ __/ +======+ +==+ \ +----+ 731 |src |__/ Seg1 ) | | \ Seg3 \____| dst| 732 +----+ \_______+ \ Segment-2 | \+_____/ +----+ 733 \======+__ _+===/ 734 \ __ __/ 735 \_______/ \___/ 737 +------------+ 738 +------------------E----+ | | 739 +----+ | | +----R---+ | +----+ 740 |src |-------R +---+ | E----------+ dst| 741 +----+ | | E--------+ +----+ 742 +--------------R | 743 +-----------------+ 745 Figure 10: Optimal replication and elimination placement across 746 technology segments example 748 5.3.2. DN-IWF related considerations 750 The ultimate goal of DN-IWF is to (1) match and (2) translate segment 751 specific flow attributes. The DN-IWF ensures that segment specific 752 attributes comprise per domain unique attributes for the whole DetNet 753 domain. This characteristic can ensure that DetNet functions can be 754 based on per domain attributes and not per segment attributes. 756 The two DetNet specific attributes have the following 757 characteristics: 759 o Flow-ID: it is same in all packets of a flow 761 o Seq.Number: it is different packet-by-packet 763 For the Flow-ID the DN-IWF can implement a static mapping. The 764 situation is more complicated for Seq.Number as it is different 765 packet-by-packet, so it may need more sophisticated translation 766 unless its format is exactly the same in the two technology segments. 767 In this later case the DN-IWF can simple copy the Seq.Number field 768 between the tunneling encapsulation of the two technology segments. 770 In case of three technology segments (IP, MPLS and TSN) three DN-IWF 771 functions can be specified. In the rest of this section the focus is 772 on the (1) IP - MPLS network scenario. Note: the use-cases are out- 773 of-scope for (2) TSN - IP, (3) TSN - MPLS. Note2: incompatible 774 format of Seq.Number with TSN. 776 Simplest implementation of DN-IWF is provided if the flow attributes 777 have the same format. Such a common denominator of the tunnel 778 encapsulation format is the pseudowire encapsulation over both IP and 779 MPLS. 781 Placeholder 783 Figure 11: FIGURE Placeholder PW over X 785 6. MPLS-based DetNet data plane solution 787 6.1. DetNet specific packet fields 789 The DetNet data plane encapsulation MUST include two DetNet specific 790 information elements in each packet of a DetNet flow: (1) a flow 791 identification and (2) a sequence number. 793 The DetNet data plane encapsulation may consists further elements 794 used for overlay tunneling, to distinguish between DetNet member 795 flows of the same DetNet compound flow or to support OAM functions. 797 6.2. Data plane encapsulation 799 Figure 12 illustrates a DetNet data plane MPLS encapsulation. The 800 MPLS-based encapsulation of the DetNet flows is a good fit for the 801 Layer-2 interconnect deployment cases (see Figure 1). Furthermore, 802 end to end DetNet service i.e., native DetNet deployment (see 803 Figure 2) is also possible if DetNet end systems are capable of 804 initiating and termination MPLS encapsulated packets. Transport of 805 IP encapsulated DetNet flows, see Section 7, over MPLS-based DetNet 806 data plane is also possible. Interworking between PW- and IPv6-based 807 encapsulations is discussed further in Section 8.6. 809 The MPLS-based DetNet data plane encapsulation consists of: 811 o DetNet control word (d-CW) containing sequencing information for 812 packet replication and duplicate elimination purposes. There MUST 813 a separate sequence number space for each DetNet flow. 815 o DetNet Label that identifies a DetNet flow within a DetNet Edge or 816 a Relay node. The DetNet label MUST be at the bottom of the label 817 stack. 819 o An optional DetNet service lable (S-Label) that represents DetNet 820 Service LSP used between DetNet Egde and/or Relay nodes. One 821 possible use of an S-Label is to identify DetNet member flows used 822 to provide protection to a DetNet compound flow, perhaps even when 823 both LSPs appear on the same link for some reason. 825 One or more MPLS transport LSP label(s) (T-label) which may be a hop- 826 by-hop label used between LSR and MUST appear higher in the label 827 stack than S-labels. A top of stack T-label may be PHPed before 828 arriving at a DetNet node. In general T-labels should be considered 829 to be part of the underlying transport network rather the actual 830 DetNet data plane encapsulation. 832 DetNet MPLS-based encapsulation 834 +---------------------------------+ 835 | | 836 | DetNet Flow | 837 | Payload Packet | 838 | | 839 +---------------------------------+ <--\ 840 | DetNet Control Word | | 841 +---------------------------------+ +--> DetNet data plane 842 | S-Label | | MPLS encapsulation 843 +---------------------------------+ <--/ 844 | T-Label(s) | 845 +---------------------------------+ 846 | Data-Link | 847 +---------------------------------+ 848 | Physical | 849 +---------------------------------+ 851 Figure 12: Encapsulation of a DetNet flow in an MPLS(-TP) PSN 853 6.3. DetNet control word 855 A DetNet control word (d-CW) conforms to the Generic PW MPLS Control 856 Word (PWMCW) defined in [RFC4385] and is illustrated in Figure 13. 857 The upper nibble of the d-CW MUST be set to zero (0). The effective 858 sequence number bit length is between 0 and 28 bits, and configured 859 either by a control plane or manually for each DetNet flow. The 860 sequence number is aligned to the right (least significant bits) and 861 unused bits MUST be set to zero (0). Each DetNet flow MUST have its 862 own sequence number counter. The sequence number is incremented by 863 one for each new packet. 865 The d-CW MUST always be present in a packet. In a case the sequence 866 number is not used (e.g., for DetNet-t-flows) the control plane or 867 the manual configuration has to define zero (0) bit length seqeunce 868 number and the value of the sequence number MUST be set to zero (0). 870 0 1 2 3 871 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 872 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 873 |0 0 0 0| Sequence Number | 874 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 876 Figure 13: DetNet Control Word 878 6.4. Flow identification 880 DetNet flow identification at a DetNet service layer is realized by 881 an S-label. It maps a Detnet flow to a specific d-CW in a DetNet 882 node. The S-label used for flow identification MUST be bottom label 883 of the label stack for a DetNet-s- or DetNet-st-flow and MUST precede 884 the d-CW. 886 An S-label for a single DetNet flow does not need to be unique DetNet 887 domain wide. As long as two or more different DetNet flows do not 888 errorneously map to a same d-CW in a DetNet node the labels may vary. 890 6.5. Service layer considerations 892 [Editor's note: quite a bit of unfinished and old text in the 893 following sections.] 895 The edge and relay node internal procedures of the PREF are 896 implementation specific. The order of a packet elimination or 897 replication is out of scope in this specification. However, care 898 should be taken that the replication function does not actually 899 loopback packets as "replicas". Looped back packets include 900 artificial delay when the node that originally initiated the packet 901 receives it again. Also, looped back packets may make the network 902 condition to look healthier than it actually is (in some cases link 903 failures are not reflected properly because looped back packets make 904 the situation appear better than it actually is). 906 Comment #29: SB> There needs to be some text about preventing a node 907 ever receiving its own replicated packets. Indeed that would 908 suggest that the flow id should be changed and replication should 909 only take place on configured flow IDs. I have a feeling that 910 this would all be a lot safer if replication only happened at 911 ingress and we managed the diversity of the paths. 913 Discussion: Agree on hardening the loopback text considerations. 915 6.5.1. Edge node processing 917 TBD. 919 [Editor's note: Since we are not defining the inner workings and 920 implementation of the DetNet Egde node - rather only what goes in and 921 what comes out, and of course the on-wire details, then the figures 922 shown in the coming section would not need to detail the inner 923 architecture of a DetNet Node.] 925 +---------------------------------------+ 926 | DetNet Edge Node | 927 +---------------------------------------+ 928 | | | | 929 | | | | 930 Client AC | +---o <-------> o o<----------> 931 | Elim. | | | | 932 <---------->o <-------| | +-------------+ 933 | | | | | 934 | +---o <-------> o | 935 | | | o<----------> 936 | | +-> o | 937 +-------------+ | +-------------+ 938 | | | | | 939 Client AC | | Repl. | | | 940 <---------->o o <-----X-> o o<----------> 941 | | Elim. | | 942 +-------------+ +-------------+ 943 | | | | 944 Client AC | | | | 945 <---------->o o <-------> o o<----------> 946 | | | | 947 +---------------------------------------+ 949 Figure 14: DetNet Edge Node processing 951 An edge node participates to the packet replication and duplication 952 elimination. Required processing is done within an extended 953 forwarder function. In the case the native service processing (NSP) 954 is IEEE 802.1CB [IEEE8021CB] capable, the packet replication and 955 duplicate elimination MAY entirely be done in the NSP and bypassing 956 the DetNet flow encapsulation and logic entirely, and thus is able to 957 operate over unmodified implementation and deployment. The NSP 958 approach works only between edge nodes and cannot make use of relay 959 nodes (see Section 6.5.2). 961 Comment #31 SB> This would be a fine way to operate the PW system - 962 edge to edge. 964 Discussion: When it comes to use of NSPs, agree. Also for "island 965 interconnect" this is a fine. However, when there is a need to do 966 PREF in a middle, plain edge to edge is not enough. 968 The DetNet-aware extended forwarder selects the egress DetNet member 969 flow based on the DetNet forwarding rules. In both "normal AC" and 970 "Packet AC" cases there may be no DetNet encapsulation header 971 available yet as it is the case with relay nodes (see Section 6.5.2). 972 It is the responsibility of the extended forwarder within the edge 973 node to push the DetNet specific encapsulation (if not already 974 present) to the packet before forwarding it to the appropriate egress 975 DetNet member flow instance(s). 977 Comment #32 SB> I am not convinced of the wisdom of having a mid- 978 point node convert a flow into a DN flow, which is what you are 979 implying here. This seems like an ingress function. 981 Discussion: OK. The text here has issues and seems to mix relay and 982 edge. 984 The extended forwarder MAY copy the sequencing information from the 985 native DetNet packet into the DetNet sequence number field and vice 986 versa. If there is no existing sequencing information available in 987 the native packet or the forwarder chose not to copy it from the 988 native packet, then the extended forwarder MUST maintain a sequence 989 number counter for each DetNet flow (indexed by the DetNet flow 990 identification). 992 6.5.2. Relay node processing 994 A DetNet Relay node participates to the packet replication and 995 duplication elimination. This processing is done within an extended 996 forwarder function. Whether an ingress DetNet member flow receives 997 DetNet specific processing depends on how the forwarding is 998 programmed. For some DetNet member flows the relay node can act as a 999 normal relay node and for some apply the DetNet specific processing 1000 (i.e., PREF). 1002 Comment #34 SB> Again relay node is not a normal term, so am not 1003 sure what it does in the absence of a PREF function. 1005 Discussion: Relay node was a DetNet aware S-PE originally, which is 1006 not explicitly stated here anymore, thus slightly confusing text 1007 here. The text here needs to clarify the roles of PREF and 1008 switching functions. A DetNet relay is described in the 1009 architecture document. However, there is definitely room for 1010 termonilogy and text improvements. 1012 It is also possible to treat the relay node as a transit node, see 1013 Section 8.3. Again, this is entirely up to how the forwarding has 1014 been programmed. 1016 The DetNet-aware forwarder selects the egress DetNet member flow 1017 segment based on the flow identification. The mapping of ingress 1018 DetNet member flow segment to egress DetNet member flow segment may 1019 be statically or dynamically configured. Additionally the DetNet- 1020 aware forwarder does duplicate frame elimination based on the flow 1021 identification and the sequence number combination. The packet 1022 replication is also done within the DetNet-aware forwarder. During 1023 elimination and the replication process the sequence number of the 1024 DetNet member flow MUST be preserved and copied to the egress DetNet 1025 member flow. 1027 +---------------------------------------+ 1028 | DetNet Relay Node | 1029 Ingress +---------------------------------------+ 1030 | | | | Egress 1031 | o---------> o--+ Elim. | 1032 ----------->o | | +--------->o-----------> 1033 | | +-----> o--+ | 1034 Ingress +-------------+ | +-------------+ 1035 | | | | | Egress 1036 | | | | | 1037 ----------->o o --+ +-> o o-----------> 1038 | | | | | 1039 Ingress +-------------+ | +-------------+ 1040 | | | | | Egress 1041 | | Repl. | | | 1042 ----------->o o ------+-> o o-----------> 1043 | | | | 1044 Ingress +-------------+ +-------------+ 1045 | | | | Egress 1046 | | | | 1047 ----------->o o --------> o o-----------> 1048 | | | | 1049 +---------------------------------------+ 1051 Figure 15: DetNet Relay Node processing 1053 Comment #35 SB> Somewhere in the dp document there needs to be a 1054 note of the requirement for interfaces to do fast exchange of 1055 counter state, and a note to those planning the network and 1056 designing the control plane that they need to provide support for 1057 this. 1059 Discussion: We kinf of agree but also think the above exchange or 1060 synchronization of counter states is not in our scope to solve. 1062 6.5.3. End system processing 1064 TBD. 1066 6.6. Transport node considerations 1068 6.6.1. Congestion protection 1070 TBD. 1072 6.6.2. Explicit routes 1074 TBD. 1076 7. Simplified IP based DetNet data plane solution 1078 [Editor's note: describe the 6 tuple way of doing DetNet service 1079 flows. Also stress that PREF is per network segment as described in 1080 Section 5.3.1] 1082 Section 5.2.2.3 illustrated the case for DetNet simplified IP data 1083 plane solution. 1085 8. Other DetNet data plane considerations 1087 8.1. Class of Service 1089 Class and quality of service, i.e., CoS and QoS, are terms that are 1090 often used interchangeably and confused. In the context of DetNet, 1091 CoS is used to refer to mechanisms that provide traffic forwarding 1092 treatment based on aggregate group basis and QoS is used to refer to 1093 mechanisms that provide traffic forwarding treatment based on a 1094 specific DetNet flow basis. Examples of existing network level CoS 1095 mechanisms include DiffServ which is enabled by IP header 1096 differentiated services code point (DSCP) field [RFC2474] and MPLS 1097 label traffic class field [RFC5462], and at Layer-2, by IEEE 802.1p 1098 priority code point (PCP). 1100 CoS for DetNet flows carried in PWs and MPLS is provided using the 1101 existing MPLS Differentiated Services (DiffServ) architecture 1102 [RFC3270]. Both E-LSP and L-LSP MPLS DiffServ modes MAY be used to 1103 support DetNet flows. The Traffic Class field (formerly the EXP 1104 field) of an MPLS label follows the definition of [RFC5462] and 1105 [RFC3270]. The Uniform, Pipe, and Short Pipe DiffServ tunneling and 1106 TTL processing models are described in [RFC3270] and [RFC3443] and 1107 MAY be used for MPLS LSPs supporting DetNet flows. MPLS ECN MAY also 1108 be used as defined in ECN [RFC5129] and updated by [RFC5462]. 1110 CoS for DetNet flows carried in IPv6 is provided using the standard 1111 differentiated services code point (DSCP) field [RFC2474] and related 1112 mechanisms. The 2-bit explicit congestion notification (ECN) 1113 [RFC3168] field MAY also be used. 1115 One additional consideration for DetNet nodes which support CoS 1116 services is that they MUST ensure that the CoS service classes do not 1117 impact the congestion protection and latency control mechanisms used 1118 to provide DetNet QoS. This requirement is similar to requirement 1119 for MPLS LSRs to that CoS LSPs do not impact the resources allocated 1120 to TE LSPs via [RFC3473]. 1122 8.2. Quality of Service 1124 Quality of Service (QoS) mechanisms for flow specific traffic 1125 treatment typically includes a guarantee/agreement for the service, 1126 and allocation of resources to support the service. Example QoS 1127 mechanisms include discrete resource allocation, admission control, 1128 flow identification and isolation, and sometimes path control, 1129 traffic protection, shaping, policing and remarking. Example 1130 protocols that support QoS control include Resource ReSerVation 1131 Protocol (RSVP) [RFC2205] (RSVP) and RSVP-TE [RFC3209] and [RFC3473]. 1132 The existing MPLS mechanisms defined to support CoS [RFC3270] can 1133 also be used to reserve resources for specific traffic classes. 1135 In addition to explicit routes, and packet replication and 1136 elimination, described in Section 6 above, DetNet provides zero 1137 congestion loss and bounded latency and jitter. As described in 1138 [I-D.ietf-detnet-architecture], there are different mechanisms that 1139 maybe used separately or in combination to deliver a zero congestion 1140 loss service. These mechanisms are provided by the either the MPLS 1141 or IP layers, and may be combined with the mechanisms defined by the 1142 underlying network layer such as 802.1TSN. 1144 A baseline set of QoS capabilities for DetNet flows carried in PWs 1145 and MPLS can provided by MPLS with Traffic Engineering (MPLS-TE) 1146 [RFC3209] and [RFC3473]. TE LSPs can also support explicit routes 1147 (path pinning). Current service definitions for packet TE LSPs can 1148 be found in "Specification of the Controlled Load Quality of 1149 Service", [RFC2211], "Specification of Guaranteed Quality of 1150 Service", [RFC2212], and "Ethernet Traffic Parameters", [RFC6003]. 1151 Additional service definitions are expected in future documents to 1152 support the full range of DetNet services. In all cases, the 1153 existing label-based marking mechanisms defined for TE-LSPs and even 1154 E-LSPs are use to support the identification of flows requiring 1155 DetNet QoS. 1157 QoS for DetNet service flows carried in IP MUST be provided locally 1158 by the DetNet-aware hosts and routers supporting DetNet flows. Such 1159 support will leverage the underlying network layer such as 802.1TSN. 1160 The traffic control mechanisms used to deliver QoS for IP 1161 encapsulated DetNet flows are expected to be defined in a future 1162 document. From an encapsulation perspective, and as defined in 1163 Section 7, the combination of the "6 tuple" i.e., the typical 5 tuple 1164 enhanced with the DSCP code, uniquely identifies a DetNet service 1165 flow. 1167 Packets that are marked with a DetNet Class of Service value, but 1168 that have not been the subject of a completed reservation, can 1169 disrupt the QoS offered to properly reserved DetNet flows by using 1170 resources allocated to the reserved flows. Therefore, the network 1171 nodes of a DetNet network MUST: 1173 o Defend the DetNet QoS by discarding or remarking (to a non-DetNet 1174 CoS) packets received that are not the subject of a completed 1175 reservation. 1177 o Not use a DetNet reserved resource, e.g. a queue or shaper 1178 reserved for DetNet flows, for any packet that does not carry a 1179 DetNet Class of Service marker. 1181 8.3. Cross-DetNet flow resource aggregation 1183 The ability to aggregate individual flows, and their associated 1184 resource control, into a larger aggregate is an important technique 1185 for improving scaling of control in the data, management and control 1186 planes. This document identifies the traffic identification related 1187 aspects of aggregation of DetNet flows. The resource control and 1188 management aspects of aggregation (including the queuing/shaping/ 1189 policing implications) will be covered in other documents. The data 1190 plane implications of aggregation are independent for PW/MPLS and IP 1191 encapsulated DetNet flows. 1193 DetNet flows transported via MPLS can leverage MPLS-TE's existing 1194 support for hierarchical LSPs (H-LSPs), see [RFC4206]. H-LSPs are 1195 typically used to aggregate control and resources, they may also be 1196 used to provide OAM or protection for the aggregated LSPs. Arbitrary 1197 levels of aggregation naturally falls out of the definition for 1198 hierarchy and the MPLS label stack [RFC3032]. DetNet nodes which 1199 support aggregation (LSP hierarchy) map one or more LSPs (labels) 1200 into and from an H-LSP. Both carried LSPs and H-LSPs may or may not 1201 use the TC field, i.e., L-LSPs or E-LSPs. Such nodes will need to 1202 ensure that traffic from aggregated LSPs are placed (shaped/policed/ 1203 enqueued) onto the H-LSPs in a fashion that ensures the required 1204 DetNet service is preserved. 1206 DetNet flows transported via IP have more limited aggregation 1207 options, due to the available traffic flow identification fields of 1208 the IP solution. One available approach is to manage the resources 1209 associated with a DSCP identified traffic class and to map (remark) 1210 individually controlled DetNet flows onto that traffic class. This 1211 approach also requires that nodes support aggregation ensure that 1212 traffic from aggregated LSPs are placed (shaped/policed/enqueued) in 1213 a fashion that ensures the required DetNet service is preserved. 1215 Comment #38 SB> I am sure we can do better than this with SR, or the 1216 use of routing techniques that map certain addresses to certain 1217 paths. 1219 Discussion: -- 1221 In both the MPLS and IP cases, additional details of the traffic 1222 control capabilities needed at a DetNet-aware node may be covered in 1223 the new service descriptions mentioned above or in separate future 1224 documents. Management and control plane mechanisms will also need to 1225 ensure that the service required on the aggregate flow (H-LSP or 1226 DSCP) are provided, which may include the discarding or remarking 1227 mentioned in the previous sections. 1229 8.4. Bidirectional traffic 1231 Some DetNet applications generate bidirectional traffic. Using MPLS 1232 definitions [RFC5654] there are associated bidirectional flows, and 1233 co-routed bidirectional flows. MPLS defines a point-to-point 1234 associated bidirectional LSP as consisting of two unidirectional 1235 point-to-point LSPs, one from A to B and the other from B to A, which 1236 are regarded as providing a single logical bidirectional transport 1237 path. This would be analogous of standard IP routing, or PWs running 1238 over two reciprocal unidirection LSPs. MPLS defines a point-to-point 1239 co-routed bidirectional LSP as an associated bidirectional LSP which 1240 satisfies the additional constraint that its two unidirectional 1241 component LSPs follow the same path (in terms of both nodes and 1242 links) in both directions. An important property of co-routed 1243 bidirectional LSPs is that their unidirectional component LSPs share 1244 fate. In both types of bidirectional LSPs, resource allocations may 1245 differ in each direction. The concepts of associated bidirectional 1246 flows and co-routed bidirectional flows can be applied to DetNet 1247 flows as well whether IPv6 or MPLS is used. 1249 While the IPv6 and MPLS data planes must support bidirectional DetNet 1250 flows, there are no special bidirectional features with respect to 1251 the data plane other than need for the two directions take the same 1252 paths. Fate sharing and associated vs co-routed bidirectional flows 1253 can be managed at the control level. Note, that there is no stated 1254 requirement for bidirectional DetNet flows to be supported using the 1255 same IPv6 Flow Labels or MPLS Labels in each direction. Control 1256 mechanisms will need to support such bidirectional flows for both 1257 IPv6 and MPLS, but such mechanisms are out of scope of this document. 1258 An example control plane solution for MPLS can be found in [RFC7551]. 1260 8.5. Layer 2 addressing and QoS Considerations 1262 The Time-Sensitive Networking (TSN) Task Group of the IEEE 802.1 1263 Working Group have defined (and are defining) a number of amendments 1264 to IEEE 802.1Q [IEEE8021Q] that provide zero congestion loss and 1265 bounded latency in bridged networks. IEEE 802.1CB [IEEE8021CB] 1266 defines packet replication and elimination functions that should 1267 prove both compatible with and useful to, DetNet networks. 1269 As is the case for DetNet, a Layer 2 network node such as a bridge 1270 may need to identify the specific DetNet flow to which a packet 1271 belongs in order to provide the TSN/DetNet QoS for that packet. It 1272 also will likely need a CoS marking, such as the priority field of an 1273 IEEE Std 802.1Q VLAN tag, to give the packet proper service. 1275 Although the flow identification methods described in IEEE 802.1CB 1276 [IEEE8021CB] are flexible, and in fact, include IP 5-tuple 1277 identification methods, the baseline TSN standards assume that every 1278 Ethernet frame belonging to a TSN stream (i.e. DetNet flow) carries 1279 a multicast destination MAC address that is unique to that flow 1280 within the bridged network over which it is carried. Furthermore, 1281 IEEE 802.1CB [IEEE8021CB] describes three methods by which a packet 1282 sequence number can be encoded in an Ethernet frame. 1284 Ensuring that the proper Ethernet VLAN tag priority and destination 1285 MAC address are used on a DetNet/TSN packet may require further 1286 clarification of the customary L2/L3 transformations carried out by 1287 routers and edge label switches. Edge nodes may also have to move 1288 sequence number fields among Layer 2, PW, and IPv6 encapsulations. 1290 8.6. Interworking between MPLS- and IPv6-based encapsulations 1292 [Editor's note: add considerations for interworking between MPLS- 1293 based and native IPv6-based DetNet encapsuations.] 1295 8.7. IPv4 considerations 1297 [Editor's note: The fact is that there are and will be deployments 1298 using IPv4. Neglecting it entirely is not feasible.] 1300 9. Time synchronization 1302 Comment #39 SB> This section should point the reader to RFC8169 1303 (residence time in MPLS n/w. We need to consider if we need to 1304 introduce the same concept in IP. 1306 Discussion: Agree. For IP we could reference to PTPv2 or v3 over 1307 UDP/IP, since it measures residence time among other things. 1309 [Editor's note: describe a bit of issues and deployment 1310 considerations related to time-synchronization within DetNet. Refer 1311 to DT discussion and the slides that summarize different approaches 1312 and rough synchronization performance numbers. Finally, scope time- 1313 synchronization solution outside data plane.] 1315 When DetNet is used, there is an underlying assumption that the 1316 applicaton(s) require clock synchronization such as the Precision 1317 Time Protocol (PTP) [IEEE1588]. The relay nodes may or may not 1318 utilize clock synchronization in order to provide zero congestion 1319 loss and controlled latency delivery. In either case, there are a 1320 few possible approaches of how synchronization protocol packets are 1321 forwarded and handled by the network: 1323 o PTP packets can be sent either as DetNet flows or as high-priority 1324 best effort packets. Using DetNet for PTP packets requires 1325 careful consideration to prevent unwanted interactions between 1326 clock-synchronized network nodes and the packets that synchronize 1327 the clocks. 1329 o PTP packets are sent as a normal DetNet flow through network nodes 1330 that are not time-synchronized: in this approach PTP traffic is 1331 forwarded as a DetNet flow, and as such it is forwarded in a way 1332 that allows a low delay variation. However, since intermediate 1333 nodes do not take part in the synchronization protocol, this 1334 approach provides a relatively low degree of accuracy. 1336 o PTP with on-path support: in this approach PTP packets are sent as 1337 ordinary or as DetNet flows, and intermediate nodes take part in 1338 the protocol as Transparent Clocks or Boundary Clocks [IEEE1588]. 1339 The on-path PTP support by intermediate nodes provides a higher 1340 degree of accuracy than the previous approach. The actual 1341 accuracy depends on whether all intermediate nodes are PTP- 1342 capable, or only a subset of them. 1344 o Time-as-a-service: in this approach accurate time is provided as- 1345 a-service to the DetNet source and destination, as well as the 1346 intermediate nodes. Since traffic between the source and 1347 destination is sent over a provider network, if the provider 1348 supports time-as-a-service, then accurate time can be provided to 1349 both the source and the destination of DetNet traffic. This 1350 approach can potentially provide the highest degree of accuracy. 1352 It is expected that the latter approach will be the most common one, 1353 as it provides the highest degree of accuracy, and creates a layer 1354 separation between the DetNet data and the synchronization service. 1356 It should be noted that in all four approaches it is not recommended 1357 to use replication and elimination for synchronization packets; the 1358 replication/elimination approach may in some cases reduce the 1359 synchronization accuracy, since the observed path delay will be 1360 bivalent. 1362 Comment #40 SB> I am not sure why we sould not use PREP. We should 1363 explain to the reader. 1365 Discussion: Agree that a this can be opened a bit more in detail. 1366 The issue is explained briefly in the last sentence but it could 1367 be more clear. 1369 10. Management and control considerations 1371 [Editor's note: This section needs to be different for MPLS and IPv6 1372 solutions. Most solutions are technology dependant,] 1374 While management plane and control planes are traditionally 1375 considered separately, from the Data Plane perspective there is no 1376 practical difference based on the origin of flow provisioning 1377 information. This document therefore does not distinguish between 1378 information provided by a control plane protocol, e.g., RSVP-TE 1379 [RFC3209] and [RFC3473], or by a network management mechanisms, e.g., 1380 RestConf [RFC8040] and YANG [RFC7950]. 1382 [Editor's note: This section is a work in progress. discuss here 1383 what kind of enhancements are needed for DetNet and specifically for 1384 PREF and DetNet zero congest loss and latency control. Need to cover 1385 both traffic control (queuing) and connection control (control 1386 plane).] 1388 10.1. MPLS-based data plane 1390 10.1.1. S-Label assignment and distribution 1392 [Editor's note: Outdated and MPLS specific.. and needs more work.] 1394 The DetNet S-Label distribution follows the same mechanisms specified 1395 for XYZ . The details of the control plane protocol solution required 1396 for the label distribution and the management of the label number 1397 space are out of scope of this document. 1399 10.1.2. Explicit routes 1401 [Editor's note: Outdated.. and needs more work.] 1403 [TBD: based on MPLS TE and possibly IPv6 SR] 1405 10.2. IPv6-based data plane 1407 10.2.1. Flow Label assignment and distribution 1409 [Editor's note: Outdated and IPv6 Specific.. and needs more work.] 1411 The IPv6 Flow Label distribution and the label number space are out 1412 of scope of this document. However, it should be noted that the 1413 combination of the IPv6 source address and the IPv6 Flow Label is 1414 assumed to be unique within the DetNet-enabled network. Therefore, 1415 as long as each node is able to assign unique Flow Labels for the 1416 source address(es) it is using the DetNet-enabled network wide flow 1417 identification uniqueness is guaranteed. 1419 10.2.2. Explicit routes 1421 [Editor's note: Outdated.. and needs more work.] 1423 [TBD: What we have there for IPv6 and explicit routes] 1425 10.3. Packet replication and elimination 1427 [Editor's note: Outdated and at the functional level technology 1428 independent.. but needs more work.] 1430 The control plane protocol solution required for managing the PREF 1431 processing is outside the scope of this document. 1433 10.4. Congestion protection and latency control 1435 [TBD] 1437 10.5. Flow aggregation control 1439 [TBD] 1441 11. Security considerations 1443 The security considerations of DetNet in general are discussed in 1444 [I-D.ietf-detnet-architecture] and [I-D.sdt-detnet-security]. Other 1445 security considerations will be added in a future version of this 1446 draft. 1448 12. IANA considerations 1450 TBD. 1452 13. Acknowledgements 1454 The author(s) ACK and NACK. 1456 The following people were part of the DetNet Data Plane Solution 1457 Design Team: 1459 Jouni Korhonen 1461 Janos Farkas 1463 Norman Finn 1465 Balazs Varga 1467 Loa Andersson 1469 Tal Mizrahi 1471 David Mozes 1473 Yuanlong Jiang 1475 Carlos J. Bernardos 1477 The DetNet chairs serving during the DetNet Data Plane Solution 1478 Design Team: 1480 Lou Berger 1481 Pat Thaler 1483 14. References 1485 14.1. Normative references 1487 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1488 Requirement Levels", BCP 14, RFC 2119, 1489 DOI 10.17487/RFC2119, March 1997, 1490 . 1492 [RFC2211] Wroclawski, J., "Specification of the Controlled-Load 1493 Network Element Service", RFC 2211, DOI 10.17487/RFC2211, 1494 September 1997, . 1496 [RFC2212] Shenker, S., Partridge, C., and R. Guerin, "Specification 1497 of Guaranteed Quality of Service", RFC 2212, 1498 DOI 10.17487/RFC2212, September 1997, 1499 . 1501 [RFC2474] Nichols, K., Blake, S., Baker, F., and D. Black, 1502 "Definition of the Differentiated Services Field (DS 1503 Field) in the IPv4 and IPv6 Headers", RFC 2474, 1504 DOI 10.17487/RFC2474, December 1998, 1505 . 1507 [RFC3032] Rosen, E., Tappan, D., Fedorkow, G., Rekhter, Y., 1508 Farinacci, D., Li, T., and A. Conta, "MPLS Label Stack 1509 Encoding", RFC 3032, DOI 10.17487/RFC3032, January 2001, 1510 . 1512 [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition 1513 of Explicit Congestion Notification (ECN) to IP", 1514 RFC 3168, DOI 10.17487/RFC3168, September 2001, 1515 . 1517 [RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V., 1518 and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP 1519 Tunnels", RFC 3209, DOI 10.17487/RFC3209, December 2001, 1520 . 1522 [RFC3270] Le Faucheur, F., Wu, L., Davie, B., Davari, S., Vaananen, 1523 P., Krishnan, R., Cheval, P., and J. Heinanen, "Multi- 1524 Protocol Label Switching (MPLS) Support of Differentiated 1525 Services", RFC 3270, DOI 10.17487/RFC3270, May 2002, 1526 . 1528 [RFC3443] Agarwal, P. and B. Akyol, "Time To Live (TTL) Processing 1529 in Multi-Protocol Label Switching (MPLS) Networks", 1530 RFC 3443, DOI 10.17487/RFC3443, January 2003, 1531 . 1533 [RFC3473] Berger, L., Ed., "Generalized Multi-Protocol Label 1534 Switching (GMPLS) Signaling Resource ReserVation Protocol- 1535 Traffic Engineering (RSVP-TE) Extensions", RFC 3473, 1536 DOI 10.17487/RFC3473, January 2003, 1537 . 1539 [RFC4023] Worster, T., Rekhter, Y., and E. Rosen, Ed., 1540 "Encapsulating MPLS in IP or Generic Routing Encapsulation 1541 (GRE)", RFC 4023, DOI 10.17487/RFC4023, March 2005, 1542 . 1544 [RFC4206] Kompella, K. and Y. Rekhter, "Label Switched Paths (LSP) 1545 Hierarchy with Generalized Multi-Protocol Label Switching 1546 (GMPLS) Traffic Engineering (TE)", RFC 4206, 1547 DOI 10.17487/RFC4206, October 2005, 1548 . 1550 [RFC4385] Bryant, S., Swallow, G., Martini, L., and D. McPherson, 1551 "Pseudowire Emulation Edge-to-Edge (PWE3) Control Word for 1552 Use over an MPLS PSN", RFC 4385, DOI 10.17487/RFC4385, 1553 February 2006, . 1555 [RFC5129] Davie, B., Briscoe, B., and J. Tay, "Explicit Congestion 1556 Marking in MPLS", RFC 5129, DOI 10.17487/RFC5129, January 1557 2008, . 1559 [RFC5462] Andersson, L. and R. Asati, "Multiprotocol Label Switching 1560 (MPLS) Label Stack Entry: "EXP" Field Renamed to "Traffic 1561 Class" Field", RFC 5462, DOI 10.17487/RFC5462, February 1562 2009, . 1564 [RFC6003] Papadimitriou, D., "Ethernet Traffic Parameters", 1565 RFC 6003, DOI 10.17487/RFC6003, October 2010, 1566 . 1568 [RFC6073] Martini, L., Metz, C., Nadeau, T., Bocci, M., and M. 1569 Aissaoui, "Segmented Pseudowire", RFC 6073, 1570 DOI 10.17487/RFC6073, January 2011, 1571 . 1573 [RFC7510] Xu, X., Sheth, N., Yong, L., Callon, R., and D. Black, 1574 "Encapsulating MPLS in UDP", RFC 7510, 1575 DOI 10.17487/RFC7510, April 2015, 1576 . 1578 [RFC8200] Deering, S. and R. Hinden, "Internet Protocol, Version 6 1579 (IPv6) Specification", STD 86, RFC 8200, 1580 DOI 10.17487/RFC8200, July 2017, 1581 . 1583 14.2. Informative references 1585 [I-D.ietf-6man-segment-routing-header] 1586 Previdi, S., Filsfils, C., Raza, K., Dukes, D., Leddy, J., 1587 Field, B., daniel.voyer@bell.ca, d., 1588 daniel.bernier@bell.ca, d., Matsushima, S., Leung, I., 1589 Linkova, J., Aries, E., Kosugi, T., Vyncke, E., Lebrun, 1590 D., Steinberg, D., and R. Raszuk, "IPv6 Segment Routing 1591 Header (SRH)", draft-ietf-6man-segment-routing-header-09 1592 (work in progress), March 2018. 1594 [I-D.ietf-detnet-architecture] 1595 Finn, N., Thubert, P., Varga, B., and J. Farkas, 1596 "Deterministic Networking Architecture", draft-ietf- 1597 detnet-architecture-04 (work in progress), October 2017. 1599 [I-D.ietf-detnet-dp-alt] 1600 Korhonen, J., Farkas, J., Mirsky, G., Thubert, P., 1601 Zhuangyan, Z., and L. Berger, "DetNet Data Plane Protocol 1602 and Solution Alternatives", draft-ietf-detnet-dp-alt-00 1603 (work in progress), October 2016. 1605 [I-D.sdt-detnet-security] 1606 Mizrahi, T., Grossman, E., Hacker, A., Das, S., 1607 "Deterministic Networking (DetNet) Security 1608 Considerations, draft-sdt-detnet-security, work in 1609 progress", 2017. 1611 [IEEE1588] 1612 IEEE, "IEEE 1588 Standard for a Precision Clock 1613 Synchronization Protocol for Networked Measurement and 1614 Control Systems Version 2", 2008. 1616 [IEEE8021CB] 1617 Finn, N., "Draft Standard for Local and metropolitan area 1618 networks - Seamless Redundancy", IEEE P802.1CB 1619 /D2.1 P802.1CB, December 2015, 1620 . 1623 [IEEE8021Q] 1624 IEEE 802.1, "Standard for Local and metropolitan area 1625 networks--Bridges and Bridged Networks (IEEE Std 802.1Q- 1626 2014)", 2014, . 1628 [RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S. 1629 Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1 1630 Functional Specification", RFC 2205, DOI 10.17487/RFC2205, 1631 September 1997, . 1633 [RFC3985] Bryant, S., Ed. and P. Pate, Ed., "Pseudo Wire Emulation 1634 Edge-to-Edge (PWE3) Architecture", RFC 3985, 1635 DOI 10.17487/RFC3985, March 2005, 1636 . 1638 [RFC5654] Niven-Jenkins, B., Ed., Brungard, D., Ed., Betts, M., Ed., 1639 Sprecher, N., and S. Ueno, "Requirements of an MPLS 1640 Transport Profile", RFC 5654, DOI 10.17487/RFC5654, 1641 September 2009, . 1643 [RFC7551] Zhang, F., Ed., Jing, R., and R. Gandhi, Ed., "RSVP-TE 1644 Extensions for Associated Bidirectional Label Switched 1645 Paths (LSPs)", RFC 7551, DOI 10.17487/RFC7551, May 2015, 1646 . 1648 [RFC7950] Bjorklund, M., Ed., "The YANG 1.1 Data Modeling Language", 1649 RFC 7950, DOI 10.17487/RFC7950, August 2016, 1650 . 1652 [RFC8040] Bierman, A., Bjorklund, M., and K. Watsen, "RESTCONF 1653 Protocol", RFC 8040, DOI 10.17487/RFC8040, January 2017, 1654 . 1656 Appendix A. Example of DetNet data plane operation 1658 [Editor's note: Add a simplified example of DetNet data plane and how 1659 labels etc work in the case of MPLS-based PSN and utilizing PREF. 1660 The figure is subject to change depending on the further DT decisions 1661 on the label handling..] 1663 Appendix B. Example of pinned paths using IPv6 1665 TBD. 1667 Authors' Addresses 1669 Jouni Korhonen (editor) 1670 Nordic Semiconductor 1672 Email: jouni.nospam@gmail.com 1674 Loa Andersson 1675 Huawei 1677 Email: loa@pi.nu 1679 Yuanlong Jiang 1680 Huawei 1682 Email: jiangyuanlong@huawei.com 1684 Norman Finn 1685 Huawei 1686 3101 Rio Way 1687 Spring Valley, CA 91977 1688 USA 1690 Email: norman.finn@mail01.huawei.com 1692 Balazs Varga 1693 Ericsson 1694 Konyves Kalman krt. 11/B 1695 Budapest 1097 1696 Hungary 1698 Email: balazs.a.varga@ericsson.com 1699 Janos Farkas 1700 Ericsson 1701 Konyves Kalman krt. 11/B 1702 Budapest 1097 1703 Hungary 1705 Email: janos.farkas@ericsson.com 1707 Carlos J. Bernardos 1708 Universidad Carlos III de Madrid 1709 Av. Universidad, 30 1710 Leganes, Madrid 28911 1711 Spain 1713 Phone: +34 91624 6236 1714 Email: cjbc@it.uc3m.es 1715 URI: http://www.it.uc3m.es/cjbc/ 1717 Tal Mizrahi 1718 Marvell 1719 6 Hamada st. 1720 Yokneam 1721 Israel 1723 Email: talmi@marvell.com 1725 Lou Berger 1726 LabN Consulting, L.L.C. 1728 Email: lberger@labn.net