idnits 2.17.1 draft-dt-detnet-dp-sol-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 886 has weird spacing: '..."Packet o <-...' == Line 891 has weird spacing: '...Service o<--...' == Line 896 has weird spacing: '...Service o<--...' == Line 901 has weird spacing: '...Service o<--...' == Line 999 has weird spacing: '...Service o -...' == (3 more instances...) -- The document date (September 7, 2017) is 2422 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'RFC8214' is mentioned on line 157, but not defined == Missing Reference: 'Network' is mentioned on line 377, but not defined == Missing Reference: 'TBD' is mentioned on line 1367, but not defined == Outdated reference: A later version (-26) exists of draft-ietf-6man-segment-routing-header-07 == Outdated reference: A later version (-13) exists of draft-ietf-detnet-architecture-03 Summary: 0 errors (**), 0 flaws (~~), 12 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 DetNet J. Korhonen, Ed. 3 Internet-Draft Nordic 4 Intended status: Standards Track L. Andersson 5 Expires: March 11, 2018 Y. Jiang 6 N. Finn 7 Huawei 8 B. Varga 9 J. Farkas 10 Ericsson 11 CJ. Bernardos 12 UC3M 13 T. Mizrahi 14 Marvell 15 L. Berger 16 LabN 17 September 7, 2017 19 DetNet Data Plane Encapsulation 20 draft-dt-detnet-dp-sol-02 22 Abstract 24 This document specifies Deterministic Networking data plane 25 encapsulation solutions. The described data plane solutions can be 26 applied over either IP or MPLS Packet Switched Networks. 28 Comment #1: SB> An overarching comment is that the early part of the 29 document is really fundamental architecture and perhaps belongs in 30 the arch draft, leaving this draft to be more specific about 31 solutions. Indeed if we cannot find a single solution that maps to 32 both IP and MPLS underlays I wonder if we should publish two 33 specialist RFCs? 35 Discussion: One document at the beginning, split into two if/when 36 needed. Would be post adoption in any case. 38 Comment #2: SB> Whilst I think we should look for a common solution 39 to IP and MPLS I do not think that this is where the DT ended up. 41 Discussion: Agree. 43 Status of This Memo 45 This Internet-Draft is submitted in full conformance with the 46 provisions of BCP 78 and BCP 79. 48 Internet-Drafts are working documents of the Internet Engineering 49 Task Force (IETF). Note that other groups may also distribute 50 working documents as Internet-Drafts. The list of current Internet- 51 Drafts is at https://datatracker.ietf.org/drafts/current/. 53 Internet-Drafts are draft documents valid for a maximum of six months 54 and may be updated, replaced, or obsoleted by other documents at any 55 time. It is inappropriate to use Internet-Drafts as reference 56 material or to cite them other than as "work in progress." 58 This Internet-Draft will expire on March 11, 2018. 60 Copyright Notice 62 Copyright (c) 2017 IETF Trust and the persons identified as the 63 document authors. All rights reserved. 65 This document is subject to BCP 78 and the IETF Trust's Legal 66 Provisions Relating to IETF Documents 67 (https://trustee.ietf.org/license-info) in effect on the date of 68 publication of this document. Please review these documents 69 carefully, as they describe your rights and restrictions with respect 70 to this document. Code Components extracted from this document must 71 include Simplified BSD License text as described in Section 4.e of 72 the Trust Legal Provisions and are provided without warranty as 73 described in the Simplified BSD License. 75 Table of Contents 77 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 78 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 79 2.1. Terms used in this document . . . . . . . . . . . . . . . 5 80 2.2. Abbreviations . . . . . . . . . . . . . . . . . . . . . . 7 81 3. Requirements language . . . . . . . . . . . . . . . . . . . . 8 82 4. DetNet data plane overview . . . . . . . . . . . . . . . . . 8 83 4.1. DetNet data plane encapsulation requirements . . . . . . 10 84 5. DetNet data plane solution . . . . . . . . . . . . . . . . . 12 85 5.1. DetNet specific packet fields . . . . . . . . . . . . . . 12 86 5.2. DetNet encapsulation . . . . . . . . . . . . . . . . . . 12 87 5.2.1. PseudoWire-based data plane encapsulation . . . . . . 13 88 5.2.2. Native IPv6-based data plane encapsulation . . . . . 15 89 5.3. DetNet flow identification for duplicate detection . . . 17 90 5.3.1. PseudoWire encapsulation . . . . . . . . . . . . . . 17 91 5.3.2. Native IPv6 encapsulation . . . . . . . . . . . . . . 18 92 6. PREF specific considerations . . . . . . . . . . . . . . . . 18 93 6.1. PseudoWire-based data plane . . . . . . . . . . . . . . . 18 94 6.1.1. Forwarder clarifications . . . . . . . . . . . . . . 18 95 6.1.2. Edge node processing clarifications . . . . . . . . . 19 96 6.1.3. Relay node processing clarifications . . . . . . . . 21 97 6.2. Native IPv6-based data plane . . . . . . . . . . . . . . 23 98 7. Other DetNet data plane considerations . . . . . . . . . . . 23 99 7.1. Class of Service . . . . . . . . . . . . . . . . . . . . 23 100 7.2. Quality of Service . . . . . . . . . . . . . . . . . . . 24 101 7.3. Cross-DetNet flow resource aggregation . . . . . . . . . 25 102 7.4. Bidirectional traffic . . . . . . . . . . . . . . . . . . 26 103 7.5. Layer 2 addressing and QoS Considerations . . . . . . . . 27 104 7.6. Interworking between PW- and IPv6-based encapsulations . 27 105 8. Time synchronization . . . . . . . . . . . . . . . . . . . . 27 106 9. Management and control considerations . . . . . . . . . . . . 29 107 9.1. PW Label and IPv6 Flow Label assignment and distribution 29 108 9.2. Packet replication and elimination . . . . . . . . . . . 30 109 9.3. Explicit paths . . . . . . . . . . . . . . . . . . . . . 30 110 9.4. Congestion protection and latency control . . . . . . . . 30 111 9.5. Flow aggregation control . . . . . . . . . . . . . . . . 30 112 10. Security considerations . . . . . . . . . . . . . . . . . . . 30 113 11. IANA considerations . . . . . . . . . . . . . . . . . . . . . 30 114 12. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 30 115 13. References . . . . . . . . . . . . . . . . . . . . . . . . . 31 116 13.1. Normative references . . . . . . . . . . . . . . . . . . 31 117 13.2. Informative references . . . . . . . . . . . . . . . . . 33 118 Appendix A. Example of DetNet data plane operation . . . . . . . 34 119 Appendix B. Example of pinned paths using IPv6 . . . . . . . . . 35 120 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 35 122 1. Introduction 124 Deterministic Networking (DetNet) is a service that can be offered by 125 a network to DetNet flows. DetNet provides these flows extremely low 126 packet loss rates and assured maximum end-to-end delivery latency. 127 General background and concepts of DetNet can be found in 128 [I-D.ietf-detnet-architecture]. 130 This document specifies the DetNet data plane. It defines how DetNet 131 traffic is encapsulated at the network layer, and how DetNet-aware 132 nodes can identity DetNet flows. Two data plane definitions are 133 given. 135 o PW-based: One solution is based on PseudoWires (PW) [RFC3985] and 136 [RFC5036] and makes use of multi-segment pseudowires (MS-PW) 137 [RFC6073] to map DetNet Relay and Edge Nodes 138 [I-D.ietf-detnet-architecture] [I-D.ietf-detnet-dp-alt] to PW 139 architecture. The PW-based data plane can be run over an MPLS 140 [RFC4448] [RFC6658] Packet Switched Network (PSN). 142 Comment #3: SB> This is really an MPLS one. The world of IP PWs is 143 a bit scruffy with some work in PWE3 and some in L2TPext which 144 really went their own ways. There is for example no MS-PW IP 145 design. The MS-PW approach needs to be examined more closely by 146 the WG and thus at this stage be marked as provisional. 148 Discussion: Agree. "can be" -> "is". 150 Comment #3.1 LB> EVPN-based encapsulation is also a potential 151 candidate. In general DetNet should look more closely to the 152 delevopment around EVPN. 154 Discussion Agree. EVPN could be a potential solution and the on- 155 wire encapsuations are likely to be the same as PW-based 156 encapsulation would be. EVPN even recommends using Control Word 157 [RFC8214] (in the absence of entropy labels). 159 o Native-IP: The other solution is based on IP header fields, namely 160 on the IPv6 Flow Label and a new DetNet Control Word extension 161 header option. It is targeted for native IPv6 networks. 163 Comment #4: SB> The IP solution has not been properly examined by 164 the WG and needs to be marked as provisional. 166 Discussion: IP vs. MPLS is a deployment choice. 168 It is worth noting that while PWs are designed to work over IP PSNs 169 this document describes a native-IP solution that operates without 170 PWs. The primary reason for this is the benefit gained by enabling 171 the use of a normal application stack, where transport protocols such 172 as TCP or UDP are directly encapsulated in IP. 174 Comment #5: SB> We clearly need to look at the implications of 175 running this with a new IP header extension. Firstly we need input 176 from 6man, but we also need to understand what happens in middle 177 boxes, other components of the host stack etc. 179 Discussion: A WG can develop their own extensions and then get 180 approval from 6man. Sometimes that ends up redoing extensions in 181 6man but not always. 183 This document specifies the encapsulation for DetNet flows, including 184 a DetNet Control Word (CW). Furthermore, it describes how DetNet 185 flows are identified, how DetNet Relay and Edge nodes work, and how 186 the Packet Replication and Elimination function (PREF) is implemented 187 with these two data plane solutions. This document does not define 188 the associated control plane functions, or Operations, 189 Administration, and Maintenance (OAM). It also does not specify 190 traffic handling capabilities required to deliver congestion 191 protection and latency control to DetNet flows as this is defined to 192 be provided by the underlying MPLS or IP network. 194 Comment #6: SB> OK, although I think that this may be a mistake. 195 There may well be some coupling needed between the Detnet DP and 196 the substrate/transport/underlay needed to make this work. If this 197 is a genuine technical layering we should talk about it. If this 198 is an artificial constraint imposed by the IESG we need to talk to 199 them. 201 Discussion: The only interaction needed is that the flow 202 identification is possible. That needs to be available for lower 203 layers. 205 Comment #6.1: LA> Even though this document does not specify any OAM 206 functions, we will need to verify that the GACh (Generalized 207 Associate Channel) works correctly in a network that has 208 replication and elimination. 210 Discussion: -- 212 2. Terminology 214 2.1. Terms used in this document 216 This document uses the terminology established in the DetNet 217 architecture [I-D.ietf-detnet-architecture] and the DetNet Data Plane 218 Solution Alternatives [I-D.ietf-detnet-dp-alt]. 220 The following terms are also used in this document: 222 DA-T-PE MPLS based DetNet edge node: a DetNet-aware PseudoWire 223 Terminating Provider Edge (T-PE). 225 DA-S-PE MPLS based DetNet relay node: a DetNet-aware PseudoWire 226 Switching Provider Edge (S-PE). 228 Comment #7 SB> We need to look at whether the S-PE concept is the 229 best fit, or whether we should use introduce a Detnet relay to do 230 this. An S-PE just swaps the PW label, but Detnet needs it to do 231 more and a better model might be a new construct. However we 232 could also discard the relay concept and run 1+n end to end, in 233 which case the S-PEs would retain heir original function. 235 Discussion: Disagree of the dropping comment. However, the issues 236 are most likely terminology related. The relay concept is part of 237 the DetNet architecture A DA-S-PE was intended to be a DetNet 238 relay, which may do more than just swapping labels (PREF 239 functionality). Current text is quite biased to MS-PW, which was 240 the starting point for the DetNet relay in a MPLS PW network. 242 T-Label A label used to identify the LSP used to transport a 243 DetNet flow across an MPLS PSN, e.g., a hop-by-hop 244 label used between label switching routers (LSR). 246 S-Label A DetNet node to DetNet node "service" label that is 247 used between DA-*-PE devices. 249 PW Label A PseudoWire label that is used to identify DetNet flow 250 related PW Instances within a PE node. 252 Flow Label IPv6 header field that is used to identify a DetNet 253 flow (together with the source IP address field). 255 Comment #8 SB> If this is the IPv6 Flow label I think caution is 256 needed as I don't think the handling of this is well defined or 257 consistently implemented in networking equipment. 259 Discussion: DetNet specifies the use and discusses possible 260 interaction with RFC6347 in this I-D. 262 local-ID An edge and relay node internal construct that uniquely 263 identifies a DetNet flow. It may be used to select 264 proper forwarding and/or DetNet specific service 265 function. 267 Comment #9 SB> Is this really an internal construct, or is it an on 268 the wire construct? If it is needed end to end, I don't think it 269 is correct to describe it as an internal construct. When you say 270 "select" presumably you mean by potentially any DN aware node on 271 the path? 273 Discussion: It is an internal construct, so yes. 275 PREF A Packet Replication and Elimination Function (PREF) 276 does the replication and elimination processing of 277 DetNet flow packets in edge or relay nodes. The 278 replication function is essentially the existing 1+1 279 protection mechanism. The elimination function reuses 280 and extends the existing duplicate detection mechanism 281 to operate over multiple (separate) DetNet member flows 282 of a DetNet compound flow. 284 Comment #10 SB> I wonder if 1+1 is the right way to go, or whether 285 1+n is better. A bunch of new techniques have emerged over the 286 years and we really ought to look at creating paths with MRT. 288 With 1+2 on main + the two MRT paths you have a two failure 289 resiliency as far as it is possible to construct such paths in the 290 underlying topology. 292 Discussion: As observed above, actually 1+n would be closer to what 293 is needed. 1+1 was meant to be more an example showing there is 294 existing work that can be leveraged. 296 2.2. Abbreviations 298 The following abbreviations used in this document: 300 AC Attachment Circuit. 302 CE Customer Edge equipment. 304 CoS Class of Service. 306 CW Control Word. 308 d-CW DetNet Control Word. 310 DetNet Deterministic Networking. 312 DF DetNet Flow. 314 L2VPN Layer 2 Virtual Private Network. 316 LSR Label Switching Router. 318 MPLS Multiprotocol Label Switching. 320 MPLS-TP Multiprotocol Label Switching - Transport Profile. 322 MS-PW Multi-Segment PseudoWire (MS-PW). 324 NSP Native Service Processing. 326 OAM Operations, Administration, and Maintenance. 328 PE Provider Edge. 330 PREF Packet Replication and Elimination Function. 332 PSN Packet Switched Network. 334 PW PseudoWire. 336 QoS Quality of Service. 338 TSN Time-Sensitive Network. 340 3. Requirements language 342 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 343 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 344 document are to be interpreted as described in [RFC2119]. 346 4. DetNet data plane overview 348 Comment #11 I am not sure whether this is a DP overview, or an 349 architecture overview and hence whether this needs to be here or 350 in the architecture draft. 352 Discussion: Overview is more of an editorial matter and its final 353 location can be discussed later on. Currently it is "no harm" to 354 have it here but there are no binding reasons to keep the text in 355 either. 357 This document describes how to use IP and/or MPLS to support a data 358 plane method of flow identification and packet formwarding over 359 layer-3. Two different cases are covered: (i) the inter-connect 360 scenario, in which IEEE802.1 TSN is routed over a layer-3 network 361 (i.e., to enlarge the layer-2 domain), and (ii) native connectivity 362 between DetNet-aware end systems. Figure 1 illustrates an exemplary 363 scenario. 365 TSN Edge Transit Relay DetNet 366 End System Node Node Node End System 368 +---------+ +.........+ +---------+ 369 | Appl. |<---:Svc Proxy:-- End to End Service ---------->| Appl. | 370 +---------+ +---------+ +---------+ +---------+ 371 | TSN | |TSN| |Svc|<-- DetNet flow ---: Service :-->| Service | 372 +---------+ +---+ +---+ +---------+ +---------+ +---------+ 373 |Transport| |Trp| |Trp| |Transport| |Trp| |Trp| |Transport| 374 +-------.-+ +-.-+ +-.-+ +--.----.-+ +-.-+ +-.-+ +---.-----+ 375 : Link : / ,-----. \ : Link : / ,-----. \ 376 +........+ +-[ Sub ]-+ +........+ +-[ Sub ]-+ 377 [Network] [Network] 378 `-----' `-----' 380 Figure 1: A simple DetNet enabled network architecture 382 Figure 2 illustrates how DetNet can provide services for IEEE 383 802.1TSN end systems over a DetNet enabled network. The edge nodes 384 insert and remove required DetNet data plane encapsulation. The 'X' 385 in the edge and relay nodes represents a potential DetNet flow packet 386 replication and elimination point. This conceptually parallels L2VPN 387 services, and could leverage existing related solutions as discussed 388 below. 390 TSN |<---------- End to End DetNet Service ------>| TSN 391 Service | Transit Transit | Service 392 TSN (AC) | |<-Tunnel->| |<-Tnl->| | (AC) TSN 393 End | V V 1 V V 2 V V | End 394 System | +--------+ +--------+ +--------+ | System 395 +---+ | | E1 |==========| R1 |=======| E2 | | +---+ 396 | |--|----|._X_....|..DetNet..|.._ _...|..DF3..|...._X_.|---|---| | 397 |CE1| | | \ | Flow 1 | X | | / | | |CE2| 398 | | | \_.|...DF2....|._/ \_..|..DF4..|._/ | | | 399 +---+ | |==========| |=======| | +---+ 400 ^ +--------+ +--------+ +--------+ ^ 401 | Edge Node Relay Node Edge Node | 402 | | 403 |<----- Emulated Time Sensitive Networking (TSN) Service ---->| 405 Figure 2: IEEE 802.1TSN over DetNet 407 Figure 3 illustrates how end to end PW-based DetNet service can be 408 provided. In this case, the end systems are able to send and receive 409 DetNet flows. For example, an end system sends data encapsulated in 410 PseudoWire (PW) and in MPLS. Like earlier the 'X' in the end 411 systems, edge and relay nodes represents potential DetNet flow packet 412 replication and elimination points. Here the relay nodes may change 413 the underlying transport, for example tunneling IP over MPLS, or 414 simply interconnect network segments. 416 DetNet DetNet 417 Service Transit Transit Service 418 DetNet | |<-Tnl->| |<-Tnl->| | DetNet 419 End | V 1 V V 2 V | End 420 System | +--------+ +--------+ +--------+ | System 421 +---+ | | R1 |=======| R2 |=======| R3 | | +---+ 422 | X...DFa...|._X_....|..DF1..|.__ ___.|..DF3..|...._X_.|.DFa..|.X | 423 |CE1|========| \ | | X | | / |======|CE2| 424 | | | | \_.|..DF2..|._/ \__.|..DF4..|._/ | | | | 425 +---+ | |=======| |=======| | +---+ 426 ^ +--------+ +--------+ +--------+ ^ 427 | Relay Node Relay Node Relay Node | 428 | | 429 |<--------------- End to End DetNet Service -------------->| 431 Figure 3: PW-Based Native DetNet 433 Figure 4 illustrates how end to end IP-based DetNet service can be 434 provided. In this case, the end systems are able to send and receive 435 DetNet flows. [Editor's note: TBD] 437 NOTE: This figures is TBD 439 DetNet DetNet 440 Service Transit Transit Service 441 DetNet | |<-Tnl->| |<-Tnl->| | DetNet 442 End | V 1 V V 2 V | End 443 System | +--------+ +--------+ +--------+ | System 444 +---+ | | R1 |=======| R2 |=======| R3 | | +---+ 445 | X...DFa...| | | | | | .|.X | 446 | H1|========| | | | | |======| H2| 447 | | | | | | | | | | | | 448 +---+ | |=======| |=======| | +---+ 449 ^ +--------+ +--------+ +--------+ ^ 450 | Relay Node Relay Node Relay Node | 451 | | 452 |<--------------- End to End DetNet Service -------------->| 454 Figure 4: IP-Based Native DetNet 456 4.1. DetNet data plane encapsulation requirements 458 Two major groups of scenarios can be distinguished which require flow 459 identification during transport: 461 1. DetNet function related scenarios: 463 * Congestion protection and latency control: usage of allocated 464 resources (queuing, policing, shaping). 466 * Explicit routes: select/apply the flow specific path. 468 * Service protection: recognize DetNet compound and member flows 469 for replication an elimination. 471 Comment #12 I am not sure whether the correct architectural 472 construct is flow or flow group. Flow suggests that sharing/ 473 aggregation is not allowed but whether this is allowed or not 474 is an application specific issue. 476 Discussion: Agree that a flow group would be a better 477 characterization. 479 Comment #13 I think that there needs to be some clarification as 480 to whether FG is is understood by the DN system exclusively or 481 whether there is an expectation that it is understood by the 482 underlay. 484 Discussion: Agree that more detail is needed here. DetNet aware 485 nodes need to understand flow groups. Underlay needs to be 486 aware of flow groups at the resource allocation level. 488 2. OAM function related scenarios: 490 * troubleshooting (e.g., identify misbehaving flows, etc.) 492 * recognize flow(s) for analytics (e.g., increase counters, 493 etc.) 495 * correlate events with flows (e.g., volume above threshold, 496 etc.) 498 * etc. 500 Each node (edge, relay and transit) use a local-ID of the DetNet- 501 (compound)-flow in order to accomplish its role during transport. 502 Recognizing the DetNet flow is more relaxed for edge and relay nodes, 503 as they are fully aware of both the DetNet service and transport 504 layers. The primary DetNet role of intermediate transport nodes is 505 limited to ensuring congestion protection and latency control for the 506 above listed DetNet functions. 508 The DetNet data plane allows for the aggregation of DetNet flows, 509 e.g., via MPLS hierarchical LSPs, to improved scaling. When DetNet 510 flows are aggregated, transit nodes may have limited ability to 511 provide service on per-flow DetNet identifiers. Therefore, 512 identifying each individual DetNet flow on a transit node may not be 513 achieved in some network scenarios, but DetNet service can still be 514 assured in these scenarios through resource allocation and control. 516 Comment #14 You could introduce the concept of a flow group 517 identified into the packet. You may also include a flow id at a 518 lower layer. 520 Discussion: Agree on the identification properties. Adding a 521 specific id into actual on-wire formats is not necessarily needed. 523 On each node dealing with DetNet flows, a local-ID is assumed to 524 determine what local operation a packet goes through. Therefore, 525 local-IDs MUST be unique on each edge and relay nodes. Local-ID MUST 526 be unambiguously bound to the DetNet flow. 528 Comment #15 I am confused as to what you mean by local ID. Is this 529 an internal construct which packet parameters map to, in which 530 case why is it being called up? IETF practise is to specify on 531 the wire behaviour but not internal behaviour of equipments. 533 Discussion: Local-id is an internal construct, which was intended to 534 clarify the discussion in the I-D. Obviously it did not work as 535 intended. 537 5. DetNet data plane solution 539 5.1. DetNet specific packet fields 541 The DetNet data plane encapsulation should include two DetNet 542 specific information element in each packet of a DetNet flow: (1) 543 flow identification and (2) sequence number. 545 Comment #16 should, SHOULD, must or MUST? 547 Discussion: SHOULD or MUST is ok. MUST is probably more 548 appropriate. 550 The DetNet data plane encapsulation may consists further elements 551 used for overlay tunneling, to distinguish between DetNet member 552 flows of the same DetNet compound flow or to support OAM functions. 554 5.2. DetNet encapsulation 556 This document specifies two encapsulations for the DetNet data plane: 557 (1) PseudoWire (PW) for MPLS PSN and (2) native IPv6 based 558 encapsulation for IP PSN. 560 5.2.1. PseudoWire-based data plane encapsulation 562 Figure 5 illustrates a DetNet PW encapsulation over an MPLS PSN. The 563 PW-based encapsulation of the DetNet flows fits perfectly for the 564 Layer-2 interconnect deployment cases (see Figure 2). Furthermore, 565 end to end DetNet service i.e., native DetNet deployment (see 566 Figure 3) is also possible if DetNet-aware end systems are capable of 567 initiating and termination MPLS encapsulated PWs. It is also 568 possible use the same encapsulation format with a Packet PW over MPLS 569 [RFC6658]. Transport of IP encapsulated DetNet flows, see 570 Section 5.2.2, over DetNet PWs is also possible. Interworking 571 between PW- and IPv6-based encapsulations is discussed further in 572 Section 7.6. 574 The PW-based DetNet data plane encapsulation consists of: 576 o DetNet control word (d-CW) containing sequencing information for 577 packet replication and duplicate elimination purposes. There is a 578 separate sequence number space for each DetNet flow. 580 o PseudoWire Label (PW Label) that is a standard PW label 581 identifying a DetNet flow and a PW Instance within a (DA-)T-PE or 582 (DA-)S-PE device. 584 o An optional S-Label that represents DetNet Service LSP used 585 between (DA-)T-PE or (DA-)S-PE nodes. One possible use of an 586 S-Label is to identify the different DetNet member flows used to 587 provide protection to a DetNet composite flow, perhaps even when 588 both LSPs appear on the same link for some reason. 590 Comment #17 This needs some discussion by the WG. 592 Discussion: Agree, specifically if the I-D becomes WG document. 594 o MPLS transport LSP label(s) (T-label) which may be a hop-by-hop 595 label used between LSRs. 597 Comment #18 Ordinarily this will of course be PHPed before arrival 598 at an x-PE. 600 Discussion: In most cases yes - depends on the network 601 configuration. PHP is not mandatory and TP does not even have 602 PHP. 604 RFC3985 Encapsulation DetNet PW Encapsulation 606 +---------------------+ 607 | Payload | +---------------------------------+ 608 /=====================\ | | 609 H Payload Convergence H--. | DetNet Flow | 610 H---------------------H | | Payload Packet | 611 H Timing H +-\ | | 612 H---------------------H | \ /=================================\ 613 H Sequencing H--' \-->H DetNet Control Word H 614 \=====================/ \=================================/ 615 | PW Demultiplexer |--------->| PW Label | 616 +---------------------+ +---------------------------------+ 617 | PSN Convergence | .--->| Optional MPLS S-Label | 618 +---------------------+ | +---------------------------------+ 619 | PSN |-----+--->| MPLS T-Label(s) | 620 +---------------------+ +---------------------------------+ 621 | Data-Link | 622 +---------------------+ 623 | Physical | 624 +---------------------+ 626 Figure 5: Encapsulation of a DetNet flow in a PW with MPLS(-TP) PSN 628 The DetNet control word (d-CW) is identical to the control word 629 defined for Ethernet over MPLS networks in [RFC4448]. The DetNet 630 control word is illustrated in Figure 6. 632 0 1 2 3 633 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 634 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 635 |0 0 0 0| reserved - set to 0 | 16 bit Sequence Number | 636 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 638 Figure 6: DetNet Control Word 640 Comment #19 We need to think about whether "identical is the correct 641 term. The Ethernet S/N skips zero (it uses zero to mean no S/N in 642 use). is that the behaviour that we want? 644 Discussion: Good point. Identical is a wrong statement. The format 645 is the same the behaviour of SN is slightly different as 0 is 646 assumed to be a valid SN. 648 5.2.2. Native IPv6-based data plane encapsulation 650 Comment #20 SB> This part of the design needs to be marked as 651 provisional until it has a more thorough WG review. 653 Discussion: Ok, however, this is still an individual I-D. 655 Figure 7 illustrates a DetNet native IPv6 encapsulation. The native 656 IPv6 encapsulation is meant for end to end Detnet service use cases, 657 where the end stations are DetNet-aware (see Figure 4). Technically 658 it is possible to use the IPv6 encapsulation to tunnel any traffic 659 over a DetNet enabled network, which would make native IPv6 660 encapsulation also a valid data plane choice for an interconnect use 661 case (see Figure 2). 663 The native IPv6-based DetNet data plane encapsulation consists of: 665 o IPv6 header as the transport protocol. 667 o IPv6 header Flow Label that is used to help to identify a DetNet 668 flow (i.e., roughly an equivalent to the PW Label). A Flow Label 669 together with the IPv6 source address uniquely identifies a DetNet 670 flow. 672 Comment #21 SB> Have we validated that it is unconditionally safe to 673 make this assumption about the use of the FL? 675 Discussion: RFC6437 does not restrict such use and DetNet DP 676 solution can always define their own use of flow label. It should 677 be noted that a DetNet aware node will always contain new code and 678 is not a load balancer. 680 o DetNet Control Word IPv6 Destination Option containing sequencing 681 information for packet replication and duplicate elimination 682 function (PREF) purposes. The DetNet Destination Option is 683 equivalent to the DetNet Control Word. 685 A DetNet-aware end station (a host) or an intermediate node 686 initiating an IPv6 packet is responsible for setting the Flow Label, 687 adding the required DetNet Destination Option, and possibly adding a 688 routing header such as the segment routing option (for pre-defined 689 paths [I-D.ietf-6man-segment-routing-header]). The payload of the 690 native IPv6 encapsulation is any payload protocol that can be 691 identified using the Next Header field either in the IPv6 packet 692 header or in the last IPv6 extension header. 694 Comment #22 SB> We will probably need to agree an option ordering, 695 and need to note that the 6man IPv6 solution already operates on 696 the edge of the ability of h/w to see that far into the packet. 698 Discussion: RFC8200 describes extension header ordering - there is 699 not much to agree there. Agree on the hardware lookup challenges. 700 However, the issues of SR header are not this I-D to fix. 702 Comment #23 SB> I am not sure the above is needed since it is by 703 definition correct. 705 Discussion: (next header) agree. 707 A DetNet-aware end station (a host) or an intermediate node receiving 708 an IPv6 packet destined to it and containing a DetNet Destination 709 Option does the appropriate processing of the packet. This may 710 involve packet duplication and elimination (PREF processing), 711 terminating a tunnel or delivering the packet to the upper layers/ 712 Applications. 714 +---------------------------------+ 715 | | 716 | DetNet Flow | 717 | Payload | 718 | | 719 /---------------------------------\ 720 H DetNet Control Word DstOpt Hdr H 721 \---------------------------------/ 722 | IPv6 header | 723 | (with set Flow label) | 724 +---------------------------------+ 726 Figure 7: Encapsulation of a native IPv6 DetNet flow 728 A DetNet flow must carry sequencing information for packet 729 replication and elimination function (PREF) purposes. This document 730 specifies a new IPv6 Destination Option: the DetNet Destination 731 Option for that purpose. The format of the option is illustrated in 732 Figure 8. 734 Comment #24 SB> Can an SR node look at a DO? 736 Discussion: Yes. 738 0 1 2 3 739 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 740 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 741 | TBD1 | 4 | Reserved | 742 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 743 | 16 bit Sequence Number | 744 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 746 Figure 8: DetNet Destination Option 748 The Option Type for the DetNet Destination Option is set to TBD1. 749 [To be removed from the final version of the document: The Option 750 Type MUST have the two most significant bits set to 10b] 752 5.3. DetNet flow identification for duplicate detection 754 Duplicate elimination depends on flow identification. Mapping 755 between packet fields and Local-ID may impact the implementation of 756 duplicate elimination. 758 Comment #25 SB> So I wonder if the right place to put the FI is in 759 the IPv6 FI, or in the IPv6 address itself? 761 Discussion: Each flow having different address is challenging if we 762 want to terminate multiple flows into the same node with one 763 address or originate multiple flows from a node with one address 764 (note, we are aware of the one /64 per node discussion but cannot 765 assume it here, at least not yet). 767 5.3.1. PseudoWire encapsulation 769 RFC3985 Section 5.2.1. describes PW sequencing provides a duplicate 770 detection service among other things. This specification clarifies 771 this definition as follows: 773 DetNet flows that need to undergo PREF processing MUST have the 774 same PW Label when they arrive at the DA-*-PE node. 776 From the label stack processing point of view receiving the same 777 label from multiple sources is analogous to Fast Reroute backup 778 tunnel behavior [RFC4090]. The PW Label for a DetNet flow can be 779 different on each PW segment. 781 Comment #26 SB> I am not sure of the utility of this reference. In 782 FRR you should not receive packets concurrently on two paths. It 783 seems fine to state the the requirement that a single label is 784 used for both paths. 786 Discussion: OK with the same label comment. OK to remove the FRR 787 reference here. 789 5.3.2. Native IPv6 encapsulation 791 The DetNet flow identification is based on the IPv6 Flow Label and 792 the source address combination. The two fields uniquelly identify 793 the end to end native IPv6 encapsulated DetNet flow. Obviously, the 794 identification fails if any intermediate node modifies either the 795 source address or the Flow Label. 797 Comment #27 SB> See earlier. If there are enough IPv6 addresses to 798 address video fragments, why not DN flows? Then this problem goes 799 away. 801 Discussion: See the earlier comment #25 discussion. If nodes get 802 their addressies via DHCPv6 basically ruins this mechanism. Also 803 the assumption for this to work is that the node has a full /64 to 804 use, which is not always the case. Otherwise the idea is just 805 fine. 807 6. PREF specific considerations 809 This section applies equally to DetNet flows transported via IPv6 and 810 MPLS. While flow identification and some header related processing 811 will differ between the two, the considerations covered in this 812 section are common to both. 814 6.1. PseudoWire-based data plane 816 6.1.1. Forwarder clarifications 818 The DetNet specific new functionality in an edge or relay node 819 processing is the packet replication and duplication elimination 820 function (PREF). This function is a part of the DetNet-aware 821 "extended" forwarder. The PREF processing is triggered by the 822 received packet of a DetNet flow. 824 Comment #28 SB> I am not sure what you mean by triggered here. 825 Hopefully we are not thinking of dataplane triggered 826 configuration? 828 Discussion: "Initiated" is probably more appropriate wording. 830 Basically the forwarding entry has to be extended with a "PREF 831 enabled" boolean configuration switch that is associated with the 832 normal forwarding actions (e.g., in case of MPLS a swap, push, pop, 833 ..). The output of the PREF elimination function is always a single 834 packet. The output of the PREF replication function is always one or 835 more packets (i.e., 1:M replication). The replicated packets MUST 836 share the same DetNet control word sequence number. 838 The complex part of the DetNet PREF processing is tracking the 839 history of received packets for multiple DetNet member flows. These 840 ingress DetNet member flows (to a node) MUST have the same local-ID 841 if they belong to the same DetNet-(compound)-flow and share the same 842 sequence number counter and the history information. 844 The edge and relay node internal procedures of the PREF are 845 implementation specific. The order of a packet elimination or 846 replication is out of scope in this specification. However, care 847 should be taken that the replication function does not actually 848 loopback packets as "replicas". Looped back packets include 849 artificial delay when the node that originally initiated the packet 850 receives it again. Also, looped back packets may make the network 851 condition to look healthier than it actually is (in some cases link 852 failures are not reflected properly because looped back packets make 853 the situation appear better than it actually is). 855 Comment #29: SB> There needs to be some text about preventing a node 856 ever receiving its own replicated packets. Indeed that would 857 suggest that the flow id should be changed and replication should 858 only take place on configured flow IDs. I have a feeling that 859 this would all be a lot safer if replication only happened at 860 ingress and we managed the diversity of the paths. 862 Discussion: Agree on hardening the loopback text considerations. 864 6.1.2. Edge node processing clarifications 866 The DetNet data plane solution overloads the edge node with DetNet 867 Edge Node functions. Edge nodes are also aware of DetNet flows and 868 may need to operate upon those. Figure 9 illustrates the overall 869 edge device functions. The figure shows both physical attachment 870 circuit (AC) (e.g., Ethernet [RFC4448]) connecting to the edge node, 871 and a packet service connecting to the edge node via an embedded 872 router function (similarly as described e.g., in [RFC6658]). Whether 873 traffic flow from a client AC and PSN tunnel receives DetNet specific 874 treatment is up to a local configuration and policy. 876 Comment #30: SB> Shouldn't the behaviour simply be a property of a 877 given PW? 879 Discussion: Agree in principle. 881 +---------------------------------------+ 882 | DetNet Edge Device | 883 +---------------------------------------+ Egress/ 884 | | Forwarder | | Ingress 885 | | | Single | member Inst. 886 Client PSN | "Packet o <-X-----> o Service o<----------> 887 tunnels | NSP" | | Repl. | Instance | 888 <---------->o | | Elim. +-------------+ Duplicate 889 | | : | | Egress 890 | | . | Single | member Inst. 891 | | +-> o Service o<----------> 892 | | | | Instance | 893 +-------------+ | +-------------+ Egress/ 894 | | | | | Ingress 895 Client AC | NSP | Repl. | | Single | member Inst. 896 <---------->o o <-----X-> o Service o<----------> 897 | | Elim. | Instance | 898 +-------------+ +-------------+ Egress/ 899 | | | | Ingress 900 Client AC | NSP | | Single | member Inst. 901 <---------->o o <-------> o Service o<----------> 902 | | | Instance | 903 +---------------------------------------+ 905 Figure 9: DetNet Edge Node processing 907 An edge node participates to the packet replication and duplication 908 elimination. Required processing is done within an extended 909 forwarder function. In the case the native service processing (NSP) 910 is IEEE 802.1CB [IEEE8021CB] capable, the packet replication and 911 duplicate elimination MAY entirely be done in the NSP and bypassing 912 the DetNet flow encapsulation and logic entirely, and thus is able to 913 operate over unmodified implementation and deployment. The NSP 914 approach works only between edge nodes and cannot make use of relay 915 nodes (see Section 6.1.3). 917 Comment #31 SB> This would be a fine way to operate the PW system - 918 edge to edge. 920 Discussion: When it comes to use of NSPs, agree. Also for "island 921 interconnect" this is a fine. However, when there is a need to do 922 PREF in a middle, plain edge to edge is not enough. 924 The DetNet-aware extended forwarder selects the egress DetNet member 925 flow based on the DetNet forwarding rules. In both "normal AC" and 926 "Packet AC" cases there may be no DetNet encapsulation header 927 available yet as it is the case with relay nodes (see Section 6.1.3). 929 It is the responsibility of the extended forwarder within the edge 930 node to push the DetNet specific encapsulation (if not already 931 present) to the packet before forwarding it to the appropriate egress 932 DetNet member flow instance(s). 934 Comment #32 SB> I am not convinced of the wisdom of having a mid- 935 point node convert a flow into a DN flow, which is what you are 936 implying here. This seems like an ingress function. 938 Discussion: OK. The text here has issues and seems to mix relay and 939 edge. 941 The extended forwarder MAY copy the sequencing information from the 942 native DetNet packet into the DetNet sequence number field and vice 943 versa. If there is no existing sequencing information available in 944 the native packet or the forwarder chose not to copy it from the 945 native packet, then the extended forwarder MUST maintain a sequence 946 number counter for each DetNet flow (indexed by the DetNet flow 947 identification). 949 6.1.3. Relay node processing clarifications 951 The DetNet data plane solution overloads a relay node with DetNet 952 Relay node functions. Relay node is aware of DetNet flows and may 953 operate upon those. Figure 10 illustrates the overall DetNet relay 954 device functions. 956 Comment #33 SB> I don't think that a relay node in not a normal 957 construct so I am not sure "overload" is the right term here. 959 Discussion: Agree. There is a terminology issue here. 961 A DetNet Relay node participates to the packet replication and 962 duplication elimination. This processing is done within an extended 963 forwarder function. Whether an ingress DetNet member flow receives 964 DetNet specific processing depends on how the forwarding is 965 programmed. For some DetNet member flows the relay node can act as a 966 normal relay node and for some apply the DetNet specific processing 967 (i.e., PREF). 969 Comment #34 SB> Again relay node is not a normal term, so am not 970 sure what it does in the absence of a PREF function. 972 Discussion: Relay node was a DetNet aware S-PE originally, which is 973 not explicitly stated here anymore, thus slightly confusing text 974 here. The text here needs to clarify the roles of PREF and 975 switching functions. A DetNet relay is described in the 976 architecture document. However, there is definitely room for 977 termonilogy and text improvements. 979 It is also possible to treat the relay node as a transit node, see 980 Section 7.3. Again, this is entirely up to how the forwarding has 981 been programmed. 983 The DetNet-aware forwarder selects the egress DetNet member flow 984 segment based on the flow identification. The mapping of ingress 985 DetNet member flow segment to egress DetNet member flow segment may 986 be statically or dynamically configured. Additionally the DetNet- 987 aware forwarder does duplicate frame elimination based on the flow 988 identification and the sequence number combination. The packet 989 replication is also done within the DetNet-aware forwarder. During 990 elimination and the replication process the sequence number of the 991 DetNet member flow MUST be preserved and copied to the egress DetNet 992 member flow. 994 +---------------------------------------+ 995 | DetNet Relay Device | 996 Ingress +---------------------------------------+ 997 member | | Forwarder | | Egress 998 instance | Single | | Single | member Inst. 999 ----------->o Service o --X-----> o Service o-----------> 1000 | Instance | | Elim. | Instance | 1001 Ingress +-------------+ | +-------------+ Duplicate 1002 member | | | | | Egress 1003 instance | Single | | | Single | member Inst. 1004 ----------->o Service o --+ +-> o Service o-----------> 1005 | Instance | | | Instance | 1006 Ingress +-------------+ | +-------------+ 1007 member | | | | | Egress 1008 instance | Single | Repl. | | Single | member Inst. 1009 ----------->o Service o ------X-> o Service o-----------> 1010 | Instance | | Instance | 1011 Ingress +-------------+ +-------------+ 1012 member | | | | Egress 1013 instance | Single | | Single | member Inst. 1014 ----------->o Service o --------> o Service o-----------> 1015 | Instance | | Instance | 1016 +---------------------------------------+ 1018 Figure 10: DetNet Relay Node processing 1020 Comment #35 SB> Somewhere in the dp document there needs to be a 1021 note of the requirement for interfaces to do fast exchange of 1022 counter state, and a note to those planning the network and 1023 designing the control plane that they need to provide support for 1024 this. 1026 Discussion: We kinf of agree but also think the above exchange or 1027 synchronization of counter states is not in our scope to solve. 1029 6.2. Native IPv6-based data plane 1031 [Editor's note: this section is TBD.] 1033 7. Other DetNet data plane considerations 1035 7.1. Class of Service 1037 Class and quality of service, i.e., CoS and QoS, are terms that are 1038 often used interchangeably and confused. In the context of DetNet, 1039 CoS is used to refer to mechanisms that provide traffic forwarding 1040 treatment based on aggregate group basis and QoS is used to refer to 1041 mechanisms that provide traffic forwarding treatment based on a 1042 specific DetNet flow basis. Examples of existing network level CoS 1043 mechanisms include DiffServ which is enabled by IP header 1044 differentiated services code point (DSCP) field [RFC2474] and MPLS 1045 label traffic class field [RFC5462], and at Layer-2, by IEEE 802.1p 1046 priority code point (PCP). 1048 CoS for DetNet flows carried in PWs and MPLS is provided using the 1049 existing MPLS Differentiated Services (DiffServ) architecture 1050 [RFC3270]. Both E-LSP and L-LSP MPLS DiffServ modes MAY be used to 1051 support DetNet flows. The Traffic Class field (formerly the EXP 1052 field) of an MPLS label follows the definition of [RFC5462] and 1053 [RFC3270]. The Uniform, Pipe, and Short Pipe DiffServ tunneling and 1054 TTL processing models are described in [RFC3270] and [RFC3443] and 1055 MAY be used for MPLS LSPs supporting DetNet flows. MPLS ECN MAY also 1056 be used as defined in ECN [RFC5129] and updated by [RFC5462]. 1058 CoS for DetNet flows carried in IPv6 is provided using the standard 1059 differentiated services code point (DSCP) field [RFC2474] and related 1060 mechanisms. The 2-bit explicit congestion notification (ECN) 1061 [RFC3168] field MAY also be used. 1063 One additional consideration for DetNet nodes which support CoS 1064 services is that they MUST ensure that the CoS service classes do not 1065 impact the congestion protection and latency control mechanisms used 1066 to provide DetNet QoS. This requirement is similar to requirement 1067 for MPLS LSRs to that CoS LSPs do not impact the resources allocated 1068 to TE LSPs via [RFC3473]. 1070 7.2. Quality of Service 1072 Quality of Service (QoS) mechanisms for flow specific traffic 1073 treatment typically includes a guarantee/agreement for the service, 1074 and allocation of resources to support the service. Example QoS 1075 mechanisms include discrete resource allocation, admission control, 1076 flow identification and isolation, and sometimes path control, 1077 traffic protection, shaping, policing and remarking. Example 1078 protocols that support QoS control include Resource ReSerVation 1079 Protocol (RSVP) [RFC2205] (RSVP) and RSVP-TE [RFC3209] and [RFC3473]. 1080 The existing MPLS mechanisms defined to support CoS [RFC3270] can 1081 also be used to reserve resources for specific traffic classes. 1083 In addition to path pinning and packet replication and elimination, 1084 described in Section 5 above, DetNet provides zero congestion loss 1085 and bounded latency and jitter. 1087 Comment #36 SB> I just searched from the beginning of the document 1088 and this was the the first reference I found to pinning. 1090 Discussion: Terminology isssue. Should use, for example, explicit 1091 paths which is used in the architecture I-D. 1093 As described in [I-D.ietf-detnet-architecture], there are different 1094 mechanisms that maybe used separately or in combination to deliver a 1095 zero congestion loss service. These mechanisms are provided by the 1096 either the MPLS or IP layers, and may be combined with the mechanisms 1097 defined by the underlying network layer such as 802.1TSN. 1099 A baseline set of QoS capabilities for DetNet flows carried in PWs 1100 and MPLS can provided by MPLS with Traffic Engineering (MPLS-TE) 1101 [RFC3209] and [RFC3473]. TE LSPs can also support explicit routes 1102 (path pinning). Current service definitions for packet TE LSPs can 1103 be found in "Specification of the Controlled Load Quality of 1104 Service", [RFC2211], "Specification of Guaranteed Quality of 1105 Service", [RFC2212], and "Ethernet Traffic Parameters", [RFC6003]. 1106 Additional service definitions are expected in future documents to 1107 support the full range of DetNet services. In all cases, the 1108 existing label-based marking mechanisms defined for TE-LSPs and even 1109 E-LSPs are use to support the identification of flows requiring 1110 DetNet QoS. 1112 QoS for DetNet flows carried in IPv6 MUST be provided locally by the 1113 DetNet-aware hosts and routers supporting DetNet flows. Such support 1114 will leverage the underlying network layer such as 802.1TSN. The 1115 traffic control mechanisms used to deliver QoS for IP encapsulated 1116 DetNet flows are expected to be defined in a future document. From 1117 an encapsulation perspective, and as defined in Section 5.2.2, the 1118 combination of the Flow Label together with the IP source address 1119 uniquely identifies a DetNet flow. 1121 Packets that are marked with a DetNet Class of Service value, but 1122 that have not been the subject of a completed reservation, can 1123 disrupt the QoS offered to properly reserved DetNet flows by using 1124 resources allocated to the reserved flows. Therefore, the network 1125 nodes of a DetNet network SHOULD: 1127 Comment #37 SB> Why not MUST? 1129 Discussion: OK with MUST. 1131 o Defend the DetNet QoS by discarding or remarking (to a non-DetNet 1132 CoS) packets received that are not the subject of a completed 1133 reservation. 1135 o Not use a DetNet reserved resource, e.g. a queue or shaper 1136 reserved for DetNet flows, for any packet that does not carry a 1137 DetNet Class of Service marker. 1139 7.3. Cross-DetNet flow resource aggregation 1141 The ability to aggregate individual flows, and their associated 1142 resource control, into a larger aggregate is an important technique 1143 for improving scaling of control in the data, management and control 1144 planes. This document identifies the traffic identification related 1145 aspects of aggregation of DetNet flows. The resource control and 1146 management aspects of aggregation (including the queuing/shaping/ 1147 policing implications) will be covered in other documents. The data 1148 plane implications of aggregation are independent for PW/MPLS and IP 1149 encapsulated DetNet flows. 1151 DetNet flows transported via MPLS can leverage MPLS-TE's existing 1152 support for hierarchical LSPs (H-LSPs), see [RFC4206]. H-LSPs are 1153 typically used to aggregate control and resources, they may also be 1154 used to provide OAM or protection for the aggregated LSPs. Arbitrary 1155 levels of aggregation naturally falls out of the definition for 1156 hierarchy and the MPLS label stack [RFC3032]. DetNet nodes which 1157 support aggregation (LSP hierarchy) map one or more LSPs (labels) 1158 into and from an H-LSP. Both carried LSPs and H-LSPs may or may not 1159 use the TC field, i.e., L-LSPs or E-LSPs. Such nodes will need to 1160 ensure that traffic from aggregated LSPs are placed (shaped/policed/ 1161 enqueued) onto the H-LSPs in a fashion that ensures the required 1162 DetNet service is preserved. 1164 DetNet flows transported via IP have more limited aggregation 1165 options, due to the available traffic flow identification fields of 1166 the IP solution. One available approach is to manage the resources 1167 associated with a DSCP identified traffic class and to map (remark) 1168 individually controlled DetNet flows onto that traffic class. This 1169 approach also requires that nodes support aggregation ensure that 1170 traffic from aggregated LSPs are placed (shaped/policed/enqueued) in 1171 a fashion that ensures the required DetNet service is preserved. 1173 Comment #38 SB> I am sure we can do better than this with SR, or the 1174 use of routing techniques that map certain addresses to certain 1175 paths. 1177 Discussion: -- 1179 In both the MPLS and IP cases, additional details of the traffic 1180 control capabilities needed at a DetNet-aware node may be covered in 1181 the new service descriptions mentioned above or in separate future 1182 documents. Management and control plane mechanisms will also need to 1183 ensure that the service required on the aggregate flow (H-LSP or 1184 DSCP) are provided, which may include the discarding or remarking 1185 mentioned in the previous sections. 1187 7.4. Bidirectional traffic 1189 Some DetNet applications generate bidirectional traffic. Using MPLS 1190 definitions [RFC5654] there are associated bidirectional flows, and 1191 co-routed bidirectional flows. MPLS defines a point-to-point 1192 associated bidirectional LSP as consisting of two unidirectional 1193 point-to-point LSPs, one from A to B and the other from B to A, which 1194 are regarded as providing a single logical bidirectional transport 1195 path. This would be analogous of standard IP routing, or PWs running 1196 over two reciprocal unidirection LSPs. MPLS defines a point-to-point 1197 co-routed bidirectional LSP as an associated bidirectional LSP which 1198 satisfies the additional constraint that its two unidirectional 1199 component LSPs follow the same path (in terms of both nodes and 1200 links) in both directions. An important property of co-routed 1201 bidirectional LSPs is that their unidirectional component LSPs share 1202 fate. In both types of bidirectional LSPs, resource allocations may 1203 differ in each direction. The concepts of associated bidirectional 1204 flows and co-routed bidirectional flows can be applied to DetNet 1205 flows as well whether IPv6 or MPLS is used. 1207 While the IPv6 and MPLS data planes must support bidirectional DetNet 1208 flows, there are no special bidirectional features with respect to 1209 the data plane other than need for the two directions take the same 1210 paths. Fate sharing and associated vs co-routed bidirectional flows 1211 can be managed at the control level. Note, that there is no stated 1212 requirement for bidirectional DetNet flows to be supported using the 1213 same IPv6 Flow Labels or MPLS Labels in each direction. Control 1214 mechanisms will need to support such bidirectional flows for both 1215 IPv6 and MPLS, but such mechanisms are out of scope of this document. 1216 An example control plane solution for MPLS can be found in [RFC7551]. 1218 7.5. Layer 2 addressing and QoS Considerations 1220 The Time-Sensitive Networking (TSN) Task Group of the IEEE 802.1 1221 Working Group have defined (and are defining) a number of amendments 1222 to IEEE 802.1Q [IEEE8021Q] that provide zero congestion loss and 1223 bounded latency in bridged networks. IEEE 802.1CB [IEEE8021CB] 1224 defines packet replication and elimination functions that should 1225 prove both compatible with and useful to, DetNet networks. 1227 As is the case for DetNet, a Layer 2 network node such as a bridge 1228 may need to identify the specific DetNet flow to which a packet 1229 belongs in order to provide the TSN/DetNet QoS for that packet. It 1230 also will likely need a CoS marking, such as the priority field of an 1231 IEEE Std 802.1Q VLAN tag, to give the packet proper service. 1233 Although the flow identification methods described in IEEE 802.1CB 1234 [IEEE8021CB] are flexible, and in fact, include IP 5-tuple 1235 identification methods, the baseline TSN standards assume that every 1236 Ethernet frame belonging to a TSN stream (i.e. DetNet flow) carries 1237 a multicast destination MAC address that is unique to that flow 1238 within the bridged network over which it is carried. Furthermore, 1239 IEEE 802.1CB [IEEE8021CB] describes three methods by which a packet 1240 sequence number can be encoded in an Ethernet frame. 1242 Ensuring that the proper Ethernet VLAN tag priority and destination 1243 MAC address are used on a DetNet/TSN packet may require further 1244 clarification of the customary L2/L3 transformations carried out by 1245 routers and edge label switches. Edge nodes may also have to move 1246 sequence number fields among Layer 2, PW, and IPv6 encapsulations. 1248 7.6. Interworking between PW- and IPv6-based encapsulations 1250 [Editor's note: add considerations for interworking between PW-based 1251 and native IPv6-based DetNet encapsuations.] 1253 8. Time synchronization 1255 Comment #39 SB> This section should point the reader to RFC8169 1256 (residence time in MPLS n/w. We need to consider if we need to 1257 introduce the same concept in IP. 1259 Discussion: agree. 1261 [Editor's note: describe a bit of issues and deployment 1262 considerations related to time-synchronization within DetNet. Refer 1263 to DT discussion and the slides that summarize different approaches 1264 and rough synchronization performance numbers. Finally, scope time- 1265 synchronization solution outside data plane.] 1267 When DetNet is used, there is an underlying assumption that the 1268 applicaton(s) require clock synchronization such as the Precision 1269 Time Protocol (PTP) [IEEE1588]. The relay nodes may or may not 1270 utilize clock synchronization in order to provide zero congestion 1271 loss and controlled latency delivery. In either case, there are a 1272 few possible approaches of how synchronization protocol packets are 1273 forwarded and handled by the network: 1275 o PTP packets can be sent either as DetNet flows or as high-priority 1276 best effort packets. Using DetNet for PTP packets requires 1277 careful consideration to prevent unwanted interactions between 1278 clock-synchronized network nodes and the packets that synchronize 1279 the clocks. 1281 o PTP packets are sent as a normal DetNet flow through network nodes 1282 that are not time-synchronized: in this approach PTP traffic is 1283 forwarded as a DetNet flow, and as such it is forwarded in a way 1284 that allows a low delay variation. However, since intermediate 1285 nodes do not take part in the synchronization protocol, this 1286 approach provides a relatively low degree of accuracy. 1288 o PTP with on-path support: in this approach PTP packets are sent as 1289 ordinary or as DetNet flows, and intermediate nodes take part in 1290 the protocol as Transparent Clocks or Boundary Clocks [IEEE1588]. 1291 The on-path PTP support by intermediate nodes provides a higher 1292 degree of accuracy than the previous approach. The actual 1293 accuracy depends on whether all intermediate nodes are PTP- 1294 capable, or only a subset of them. 1296 o Time-as-a-service: in this approach accurate time is provided as- 1297 a-service to the DetNet source and destination, as well as the 1298 intermediate nodes. Since traffic between the source and 1299 destination is sent over a provider network, if the provider 1300 supports time-as-a-service, then accurate time can be provided to 1301 both the source and the destination of DetNet traffic. This 1302 approach can potentially provide the highest degree of accuracy. 1304 It is expected that the latter approach will be the most common one, 1305 as it provides the highest degree of accuracy, and creates a layer 1306 separation between the DetNet data and the synchronization service. 1308 It should be noted that in all four approaches it is not recommended 1309 to use replication and elimination for synchronization packets; the 1310 replication/elimination approach may in some cases reduce the 1311 synchronization accuracy, since the observed path delay will be 1312 bivalent. 1314 Comment #40 SB> I am not sure why we sould not use PREP. We should 1315 explain to the reader. 1317 Discussion: Agree that a this can be opened a bit more in detail. 1318 The issue is explained briefly in the last sentence but it could 1319 be more clear. 1321 9. Management and control considerations 1323 While management plane and control planes are traditionally 1324 considered separately, from the Data Plane perspective there is no 1325 practical difference based on the origin of flow provisioning 1326 information. This document therefore does not distinguish between 1327 information provided by a control plane protocol, e.g., RSVP-TE 1328 [RFC3209] and [RFC3473], or by a network management mechanisms, e.g., 1329 RestConf [RFC8040] and YANG [RFC7950]. 1331 [Editor's note: This section is a work in progress. discuss here 1332 what kind of enhancements are needed for DetNet and specifically for 1333 PREF and DetNet zero congest loss and latency control. Need to cover 1334 both traffic control (queuing) and connection control (control 1335 plane).] 1337 9.1. PW Label and IPv6 Flow Label assignment and distribution 1339 The PW label distribution follows the same mechanisms specified for 1340 MS-PW [RFC6073]. The details of the control plane protocol solution 1341 required for the label distribution and the management of the label 1342 number space are out of scope of this document. 1344 The IPv6 Flow Label distribution and the label number space are out 1345 of scope of this document. However, it should be noted that the 1346 combination of the IPv6 source address and the IPv6 Flow Label is 1347 assumed to be unique within the DetNet-enabled network. Therefore, 1348 as long as each node is able to assign unique Flow Labels for the 1349 source address(es) it is using the DetNet-enabled network wide flow 1350 identification uniqueness is guaranteed. 1352 9.2. Packet replication and elimination 1354 The control plane protocol solution required for managing the PREF 1355 processing is outside the scope of this document. 1357 9.3. Explicit paths 1359 [TBD: based on MPLS TE and SR.] 1361 9.4. Congestion protection and latency control 1363 [TBD] 1365 9.5. Flow aggregation control 1367 [TBD] 1369 10. Security considerations 1371 The security considerations of DetNet in general are discussed in 1372 [I-D.ietf-detnet-architecture] and [I-D.sdt-detnet-security]. Other 1373 security considerations will be added in a future version of this 1374 draft. 1376 11. IANA considerations 1378 TBD. 1380 12. Acknowledgements 1382 The author(s) ACK and NACK. 1384 The following people were part of the DetNet Data Plane Solution 1385 Design Team: 1387 Jouni Korhonen 1389 Janos Farkas 1391 Norman Finn 1393 Balazs Varga 1395 Loa Andersson 1397 Tal Mizrahi 1399 David Mozes 1400 Yuanlong Jiang 1402 Carlos J. Bernardos 1404 The DetNet chairs serving during the DetNet Data Plane Solution 1405 Design Team: 1407 Lou Berger 1409 Pat Thaler 1411 13. References 1413 13.1. Normative references 1415 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1416 Requirement Levels", BCP 14, RFC 2119, 1417 DOI 10.17487/RFC2119, March 1997, 1418 . 1420 [RFC2211] Wroclawski, J., "Specification of the Controlled-Load 1421 Network Element Service", RFC 2211, DOI 10.17487/RFC2211, 1422 September 1997, . 1424 [RFC2212] Shenker, S., Partridge, C., and R. Guerin, "Specification 1425 of Guaranteed Quality of Service", RFC 2212, 1426 DOI 10.17487/RFC2212, September 1997, 1427 . 1429 [RFC2474] Nichols, K., Blake, S., Baker, F., and D. Black, 1430 "Definition of the Differentiated Services Field (DS 1431 Field) in the IPv4 and IPv6 Headers", RFC 2474, 1432 DOI 10.17487/RFC2474, December 1998, 1433 . 1435 [RFC3032] Rosen, E., Tappan, D., Fedorkow, G., Rekhter, Y., 1436 Farinacci, D., Li, T., and A. Conta, "MPLS Label Stack 1437 Encoding", RFC 3032, DOI 10.17487/RFC3032, January 2001, 1438 . 1440 [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition 1441 of Explicit Congestion Notification (ECN) to IP", 1442 RFC 3168, DOI 10.17487/RFC3168, September 2001, 1443 . 1445 [RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V., 1446 and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP 1447 Tunnels", RFC 3209, DOI 10.17487/RFC3209, December 2001, 1448 . 1450 [RFC3270] Le Faucheur, F., Wu, L., Davie, B., Davari, S., Vaananen, 1451 P., Krishnan, R., Cheval, P., and J. Heinanen, "Multi- 1452 Protocol Label Switching (MPLS) Support of Differentiated 1453 Services", RFC 3270, DOI 10.17487/RFC3270, May 2002, 1454 . 1456 [RFC3443] Agarwal, P. and B. Akyol, "Time To Live (TTL) Processing 1457 in Multi-Protocol Label Switching (MPLS) Networks", 1458 RFC 3443, DOI 10.17487/RFC3443, January 2003, 1459 . 1461 [RFC3473] Berger, L., Ed., "Generalized Multi-Protocol Label 1462 Switching (GMPLS) Signaling Resource ReserVation Protocol- 1463 Traffic Engineering (RSVP-TE) Extensions", RFC 3473, 1464 DOI 10.17487/RFC3473, January 2003, 1465 . 1467 [RFC4206] Kompella, K. and Y. Rekhter, "Label Switched Paths (LSP) 1468 Hierarchy with Generalized Multi-Protocol Label Switching 1469 (GMPLS) Traffic Engineering (TE)", RFC 4206, 1470 DOI 10.17487/RFC4206, October 2005, 1471 . 1473 [RFC4448] Martini, L., Ed., Rosen, E., El-Aawar, N., and G. Heron, 1474 "Encapsulation Methods for Transport of Ethernet over MPLS 1475 Networks", RFC 4448, DOI 10.17487/RFC4448, April 2006, 1476 . 1478 [RFC5129] Davie, B., Briscoe, B., and J. Tay, "Explicit Congestion 1479 Marking in MPLS", RFC 5129, DOI 10.17487/RFC5129, January 1480 2008, . 1482 [RFC5462] Andersson, L. and R. Asati, "Multiprotocol Label Switching 1483 (MPLS) Label Stack Entry: "EXP" Field Renamed to "Traffic 1484 Class" Field", RFC 5462, DOI 10.17487/RFC5462, February 1485 2009, . 1487 [RFC6003] Papadimitriou, D., "Ethernet Traffic Parameters", 1488 RFC 6003, DOI 10.17487/RFC6003, October 2010, 1489 . 1491 [RFC6073] Martini, L., Metz, C., Nadeau, T., Bocci, M., and M. 1492 Aissaoui, "Segmented Pseudowire", RFC 6073, 1493 DOI 10.17487/RFC6073, January 2011, 1494 . 1496 [RFC6658] Bryant, S., Ed., Martini, L., Swallow, G., and A. Malis, 1497 "Packet Pseudowire Encapsulation over an MPLS PSN", 1498 RFC 6658, DOI 10.17487/RFC6658, July 2012, 1499 . 1501 13.2. Informative references 1503 [I-D.ietf-6man-segment-routing-header] 1504 Previdi, S., Filsfils, C., Raza, K., Leddy, J., Field, B., 1505 daniel.voyer@bell.ca, d., daniel.bernier@bell.ca, d., 1506 Matsushima, S., Leung, I., Linkova, J., Aries, E., Kosugi, 1507 T., Vyncke, E., Lebrun, D., Steinberg, D., and R. Raszuk, 1508 "IPv6 Segment Routing Header (SRH)", draft-ietf-6man- 1509 segment-routing-header-07 (work in progress), July 2017. 1511 [I-D.ietf-detnet-architecture] 1512 Finn, N., Thubert, P., Varga, B., and J. Farkas, 1513 "Deterministic Networking Architecture", draft-ietf- 1514 detnet-architecture-03 (work in progress), August 2017. 1516 [I-D.ietf-detnet-dp-alt] 1517 Korhonen, J., Farkas, J., Mirsky, G., Thubert, P., 1518 Zhuangyan, Z., and L. Berger, "DetNet Data Plane Protocol 1519 and Solution Alternatives", draft-ietf-detnet-dp-alt-00 1520 (work in progress), October 2016. 1522 [I-D.sdt-detnet-security] 1523 Mizrahi, T., Grossman, E., Hacker, A., Das, S., 1524 "Deterministic Networking (DetNet) Security 1525 Considerations, draft-sdt-detnet-security, work in 1526 progress", 2017. 1528 [IEEE1588] 1529 IEEE, "IEEE 1588 Standard for a Precision Clock 1530 Synchronization Protocol for Networked Measurement and 1531 Control Systems Version 2", 2008. 1533 [IEEE8021CB] 1534 Finn, N., "Draft Standard for Local and metropolitan area 1535 networks - Seamless Redundancy", IEEE P802.1CB 1536 /D2.1 P802.1CB, December 2015, 1537 . 1540 [IEEE8021Q] 1541 IEEE 802.1, "Standard for Local and metropolitan area 1542 networks--Bridges and Bridged Networks (IEEE Std 802.1Q- 1543 2014)", 2014, . 1545 [RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S. 1546 Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1 1547 Functional Specification", RFC 2205, DOI 10.17487/RFC2205, 1548 September 1997, . 1550 [RFC3985] Bryant, S., Ed. and P. Pate, Ed., "Pseudo Wire Emulation 1551 Edge-to-Edge (PWE3) Architecture", RFC 3985, 1552 DOI 10.17487/RFC3985, March 2005, 1553 . 1555 [RFC4090] Pan, P., Ed., Swallow, G., Ed., and A. Atlas, Ed., "Fast 1556 Reroute Extensions to RSVP-TE for LSP Tunnels", RFC 4090, 1557 DOI 10.17487/RFC4090, May 2005, 1558 . 1560 [RFC5036] Andersson, L., Ed., Minei, I., Ed., and B. Thomas, Ed., 1561 "LDP Specification", RFC 5036, DOI 10.17487/RFC5036, 1562 October 2007, . 1564 [RFC5654] Niven-Jenkins, B., Ed., Brungard, D., Ed., Betts, M., Ed., 1565 Sprecher, N., and S. Ueno, "Requirements of an MPLS 1566 Transport Profile", RFC 5654, DOI 10.17487/RFC5654, 1567 September 2009, . 1569 [RFC7551] Zhang, F., Ed., Jing, R., and R. Gandhi, Ed., "RSVP-TE 1570 Extensions for Associated Bidirectional Label Switched 1571 Paths (LSPs)", RFC 7551, DOI 10.17487/RFC7551, May 2015, 1572 . 1574 [RFC7950] Bjorklund, M., Ed., "The YANG 1.1 Data Modeling Language", 1575 RFC 7950, DOI 10.17487/RFC7950, August 2016, 1576 . 1578 [RFC8040] Bierman, A., Bjorklund, M., and K. Watsen, "RESTCONF 1579 Protocol", RFC 8040, DOI 10.17487/RFC8040, January 2017, 1580 . 1582 Appendix A. Example of DetNet data plane operation 1584 [Editor's note: Add a simplified example of DetNet data plane and how 1585 labels etc work in the case of MPLS-based PSN and utilizing PREF. 1586 The figure is subject to change depending on the further DT decisions 1587 on the label handling..] 1589 Appendix B. Example of pinned paths using IPv6 1591 TBD. 1593 Authors' Addresses 1595 Jouni Korhonen (editor) 1596 Nordic Semiconductor 1598 Email: jouni.nospam@gmail.com 1600 Loa Andersson 1601 Huawei 1603 Email: loa@pi.nu 1605 Yuanlong Jiang 1606 Huawei 1608 Email: jiangyuanlong@huawei.com 1610 Norman Finn 1611 Huawei 1612 3101 Rio Way 1613 Spring Valley, CA 91977 1614 USA 1616 Email: norman.finn@mail01.huawei.com 1618 Balazs Varga 1619 Ericsson 1620 Konyves Kalman krt. 11/B 1621 Budapest 1097 1622 Hungary 1624 Email: balazs.a.varga@ericsson.com 1625 Janos Farkas 1626 Ericsson 1627 Konyves Kalman krt. 11/B 1628 Budapest 1097 1629 Hungary 1631 Email: janos.farkas@ericsson.com 1633 Carlos J. Bernardos 1634 Universidad Carlos III de Madrid 1635 Av. Universidad, 30 1636 Leganes, Madrid 28911 1637 Spain 1639 Phone: +34 91624 6236 1640 Email: cjbc@it.uc3m.es 1641 URI: http://www.it.uc3m.es/cjbc/ 1643 Tal Mizrahi 1644 Marvell 1645 6 Hamada st. 1646 Yokneam 1647 Israel 1649 Email: talmi@marvell.com 1651 Lou Berger 1652 LabN Consulting, L.L.C. 1654 Email: lberger@labn.net