idnits 2.17.1 draft-ietf-detnet-data-plane-framework-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 205 has weird spacing: '...e stack v ...' -- The document date (October 27, 2019) is 1641 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-14) exists of draft-ietf-detnet-flow-information-model-05 == Outdated reference: A later version (-07) exists of draft-ietf-detnet-ip-01 == Outdated reference: A later version (-13) exists of draft-ietf-detnet-mpls-01 == Outdated reference: A later version (-07) exists of draft-ietf-detnet-mpls-over-tsn-00 == Outdated reference: A later version (-08) exists of draft-ietf-detnet-mpls-over-udp-ip-02 == Outdated reference: A later version (-16) exists of draft-ietf-detnet-security-05 == Outdated reference: A later version (-14) exists of draft-ietf-pce-pcep-extension-for-pce-controller-02 == Outdated reference: A later version (-28) exists of draft-ietf-spring-srv6-network-programming-05 -- Obsolete informational reference (is this intentional?): RFC 5575 (Obsoleted by RFC 8955) Summary: 0 errors (**), 0 flaws (~~), 10 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 DetNet B. Varga, Ed. 3 Internet-Draft J. Farkas 4 Intended status: Informational Ericsson 5 Expires: April 29, 2020 L. Berger 6 D. Fedyk 7 LabN Consulting, L.L.C. 8 A. Malis 9 Independent 10 S. Bryant 11 Futurewei Technologies 12 J. Korhonen 13 October 27, 2019 15 DetNet Data Plane Framework 16 draft-ietf-detnet-data-plane-framework-03 18 Abstract 20 This document provides an overall framework for the DetNet data 21 plane. It covers concepts and considerations that are generally 22 common to any Deterministic Networking data plane specification. 24 Status of This Memo 26 This Internet-Draft is submitted in full conformance with the 27 provisions of BCP 78 and BCP 79. 29 Internet-Drafts are working documents of the Internet Engineering 30 Task Force (IETF). Note that other groups may also distribute 31 working documents as Internet-Drafts. The list of current Internet- 32 Drafts is at https://datatracker.ietf.org/drafts/current/. 34 Internet-Drafts are draft documents valid for a maximum of six months 35 and may be updated, replaced, or obsoleted by other documents at any 36 time. It is inappropriate to use Internet-Drafts as reference 37 material or to cite them other than as "work in progress." 39 This Internet-Draft will expire on April 29, 2020. 41 Copyright Notice 43 Copyright (c) 2019 IETF Trust and the persons identified as the 44 document authors. All rights reserved. 46 This document is subject to BCP 78 and the IETF Trust's Legal 47 Provisions Relating to IETF Documents 48 (https://trustee.ietf.org/license-info) in effect on the date of 49 publication of this document. Please review these documents 50 carefully, as they describe your rights and restrictions with respect 51 to this document. Code Components extracted from this document must 52 include Simplified BSD License text as described in Section 4.e of 53 the Trust Legal Provisions and are provided without warranty as 54 described in the Simplified BSD License. 56 Table of Contents 58 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 59 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 60 2.1. Terms Used in This Document . . . . . . . . . . . . . . . 4 61 2.2. Abbreviations . . . . . . . . . . . . . . . . . . . . . . 4 62 3. DetNet Data Plane Overview . . . . . . . . . . . . . . . . . 5 63 3.1. Data Plane Characteristics . . . . . . . . . . . . . . . 6 64 3.1.1. Data Plane Technology . . . . . . . . . . . . . . . . 6 65 3.1.2. Data Plane Format . . . . . . . . . . . . . . . . . . 6 66 3.2. Encapsulation . . . . . . . . . . . . . . . . . . . . . . 6 67 3.3. DetNet Specific Metadata . . . . . . . . . . . . . . . . 7 68 3.4. DetNet IP Data Plane . . . . . . . . . . . . . . . . . . 8 69 3.5. DetNet MPLS Data Plane . . . . . . . . . . . . . . . . . 9 70 3.6. Further DetNet Data Plane Considerations . . . . . . . . 9 71 3.6.1. Per Flow Related Functions . . . . . . . . . . . . . 9 72 3.6.2. Service Protection . . . . . . . . . . . . . . . . . 11 73 3.6.3. Aggregation Considerations . . . . . . . . . . . . . 13 74 3.6.4. End-System-Specific Considerations . . . . . . . . . 14 75 3.6.5. Sub-Network Considerations . . . . . . . . . . . . . 15 76 4. Controller Plane (Management and Control) 77 Considerations . . . . . . . . . . . . . . . . . . . . . . . 16 78 4.1. DetNet Controller Plane Requirements . . . . . . . . . . 16 79 4.2. Generic Controller Plane Considerations . . . . . . . . . 17 80 4.2.1. Flow Aggregation Control . . . . . . . . . . . . . . 18 81 4.2.2. Explicit Routes . . . . . . . . . . . . . . . . . . . 19 82 4.2.3. Contention Loss and Jitter Reduction . . . . . . . . 19 83 4.2.4. Bidirectional Traffic . . . . . . . . . . . . . . . . 20 84 4.3. Packet Replication, Elimination, and Ordering (PREOF) . . 21 85 5. Security Considerations . . . . . . . . . . . . . . . . . . . 21 86 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 22 87 7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 22 88 8. References . . . . . . . . . . . . . . . . . . . . . . . . . 22 89 8.1. Normative References . . . . . . . . . . . . . . . . . . 22 90 8.2. Informative References . . . . . . . . . . . . . . . . . 22 91 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 25 93 1. Introduction 95 DetNet (Deterministic Networking) provides a capability to carry 96 specified unicast or multicast data flows for real-time applications 97 with extremely low packet loss rates and assured maximum end-to-end 98 delivery latency. A description of the general background and 99 concepts of DetNet can be found in [I-D.ietf-detnet-architecture]. 101 This document describes the concepts needed by any DetNet data plane 102 specification and provides considerations for any corresponding 103 implementation. It covers the building blocks that provide the 104 DetNet service, the DetNet service sub-layer and the DetNet 105 forwarding sub-layer functions as described in the DetNet 106 Architecture. 108 The DetNet Architecture models the DetNet related data plane 109 functions decomposed into two sub-layers: a service sub-layer and a 110 forwarding sub-layer. The service sub-layer is used to provide 111 DetNet service protection and reordering. The forwarding sub-layer 112 leverages Traffic Engineering mechanisms and provides congestion 113 protection (low loss, assured latency, and limited out-of-order 114 delivery). 116 As part of the service sub-layer functions, this document describes 117 typical DetNet node data plane operation. It describes the function 118 and operation of the Packet Replication (PRF) Packet Elimination 119 (PEF) and the Packet Ordering (POF) functions within the service sub- 120 layer. Furthermore, it also describes the forwarding sub-layer. 122 DetNet flows may be carried over network technologies that can 123 provide the DetNet required service characteristics. For example, 124 DetNet MPLS flows can be carried over IEEE 802.1 Time Sensitive 125 Network (TSN) [IEEE802.1TSNTG] sub-networks. However, IEEE 802.1 TSN 126 support is not required and some of the DetNet benefits can be gained 127 by running over a data link layer that has not been specifically 128 enhanced to support TSN. 130 Different application flows (e.g., Ethernet, IP, etc.), can be mapped 131 on top of DetNet. DetNet can optionally reuse header information 132 provided by, or shared with, applications. An example of shared 133 header fields can be found in [I-D.ietf-detnet-ip]. 135 This document also covers basic concepts related to the controller 136 plane and Operations, Administration, and Maintenance (OAM). Data 137 plane OAM specifics are out of scope for this docuement. 139 2. Terminology 141 2.1. Terms Used in This Document 143 This document uses the terminology established in the DetNet 144 architecture [I-D.ietf-detnet-architecture], and the reader is 145 assumed to be familiar with that document and its terminology. 147 2.2. Abbreviations 149 The following abbreviations are used in this document: 151 CW Control Word. 153 d-CW DetNet Control Word. 155 DetNet Deterministic Networking. 157 DN DetNet. 159 GRE Generic Routing Encapsulation. 161 IPSec IP Security. 163 L2 Layer 2. 165 LSR Label Switching Router. 167 MPLS Multiprotocol Label Switching. 169 MPLS-TE Multiprotocol Label Switching - Traffic Engineering. 171 OAM Operations, Administration, and Maintenance. 173 PEF Packet Elimination Function. 175 PRF Packet Replication Function. 177 PREOF Packet Replication, Elimination and Ordering Functions. 179 POF Packet Ordering Function. 181 PSN Packet Switched Network. 183 PW PseudoWire. 185 QoS Quality of Service. 187 S-Label DetNet "service" label. 189 TDM Time-Division Multiplexing. 191 TSN Time-Sensitive Network. 193 3. DetNet Data Plane Overview 195 This document describes how application flows, or app-flows, are 196 carried over DetNet networks. The DetNet Architecture, 197 [I-D.ietf-detnet-architecture], models the DetNet related data plane 198 functions as decomposed into two sub-layers: a service sub-layer and 199 a forwarding sub-layer. 201 Figure 1 reproduced from the [I-D.ietf-detnet-architecture],shows a 202 logical DetNet service with the two sub-layers. 204 | packets going | ^ packets coming ^ 205 v down the stack v | up the stack | 206 +-----------------------+ +-----------------------+ 207 | Source | | Destination | 208 +-----------------------+ +-----------------------+ 209 | Service sub-layer: | | Service sub-layer: | 210 | Packet sequencing | | Duplicate elimination | 211 | Flow replication | | Flow merging | 212 | Packet encoding | | Packet decoding | 213 +-----------------------+ +-----------------------+ 214 | Forwarding sub-layer: | | Forwarding sub-layer: | 215 | Resource allocation | | Resource allocation | 216 | Explicit routes | | Explicit routes | 217 +-----------------------+ +-----------------------+ 218 | Lower layers | | Lower layers | 219 +-----------------------+ +-----------------------+ 220 v ^ 221 \_________________________/ 223 Figure 1: DetNet data plane protocol stack 225 The DetNet forwarding sub-layer may be directly provided by the 226 DetNet service sub-layer, for example by IP tunnels or MPLS. 227 Alternatively, an overlay approach may be used in which the packet is 228 natively carried between key nodes within the DetNet network (say 229 between PREOF nodes) and a sub-layer is used to provide the 230 information needed to reach the next hop in the overlay. 232 The forwarding sub-layer provides the QoS related functions needed by 233 the DetNet flow. It may do this directly through the use of queuing 234 techniques and traffic engineering methods, or it may do this through 235 the assistance of its underlying connectivity. For example it may 236 call upon Ethernet TSN capabilities defined in IEEE 802.1 TSN 237 [IEEE802.1TSNTG]. 239 The service sub-layer provides additional support beyond the 240 connectivity function of the forwarding sub-layer. An example of 241 this is Packet Replication, Elimination, and Ordering functions see 242 Section 4.3. 244 The method of instantiating each of the layers is specific to the 245 particular DetNet data plane method, and more than one approach may 246 be applicable to a given bearer network type. 248 3.1. Data Plane Characteristics 250 There are two major characteristics to the data plane: the technology 251 and the encapsulation, as discussed below. 253 3.1.1. Data Plane Technology 255 The DetNet data plane is provided by the DetNet service and 256 forwarding sub layers. The DetNet service sub-layer generally 257 provides its functions for the DetNet application flows by using or 258 applying existing standardized headers and/or encapsulations. The 259 Detnet forwarding sub-layer may provide capabilities leveraging that 260 same header or encapsulation technology (e.g., DN IP or DN MPLS) or 261 it may be achieved by other technologies (e.g., Figure 2). DetNet is 262 currently defined for operation over packet switched (IP) networks or 263 label switched (MPLS) networks. 265 3.1.2. Data Plane Format 267 DetNet encodes specific flow attributes (flow identity and sequence 268 number) in packets. For example, in DetNet IP, zero encapsulation is 269 used and no sequence number is available, and in DetNet MPLS, DetNet 270 specific information may be added explicitly to the packets in the 271 format of S-label and d-CW. 273 3.2. Encapsulation 275 The encapsulation of a DetNet flow allows it to be sent over a data 276 plane technology other than its native type. DetNet uses header 277 information to perform traffic classification, i.e., identify DetNet 278 flows, and provide DetNet service and forwarding functions. As 279 mentioned above, DetNet may add headers, as is the case for DN MPLS, 280 or may use headers that are already present, as is the case in DN IP. 281 Figure 2 illustrates some relationships between the components. 283 +-----+ 284 | TSN | 285 +-------+ +-+-----+-+ 286 | DN IP | | DN MPLS | 287 +--+--+----+----+ +-+---+-----+-+ 288 | TSN | DN MPLS | | TSN | DN IP | 289 +-----+---------+ +-----+-------+ 291 Figure 2: DetNet Service Examples 293 The use of encapsulation is also required if additional information 294 (metadata) is needed by the DetNet data plane and there is either no 295 ability to include it in the client data packet, or the specification 296 of the client data plane does not permit the modification of the 297 packet to include additional data. An example of such metadata is 298 the inclusion of a sequence number required by the PREOF function. 300 Encapsulation may also be used to carry or aggregate flows for 301 equipment with limited DetNet capability. 303 3.3. DetNet Specific Metadata 305 The DetNet data plane can provide or carry metadata: 307 1. Flow-ID 309 2. Sequence Number 311 The DetNet data plane framework supports a Flow-ID (for 312 identification of the flow or aggregate flow) and/or a Sequence 313 Number (for PREOF) for each DetNet flow. The DetNet Service sub- 314 layer requires both; the DetNet forwarding sub-layer requires only 315 Flow-ID. Metadata can also be used for OAM indications and 316 instrumentation of DetNet data plane operation. 318 Metadata can be included implicit or explicit. Explicit means that a 319 dedicated header field is used to include metadata in a DetNet 320 packet. In case of implicit method a part of an already existing 321 header field is used to encode the metadata. 323 Explicit inclusion of metadata is possible through the use of IP 324 options or IP extension headers. New IP options are almost 325 impossible to get standardized or to deploy in an operational network 326 and will not be discussed further in this text. IPv6 extensions 327 headers are finding popularity in current IPv6 development work, 328 particularly in connection with Segment Routing of IPv6 (SRv6) and IP 329 OAM. The design of a new IPv6 extension header or the modification 330 of an existing one is a technique available in the tool box of the 331 DetNet IP data plane designer. 333 Explicit inclusion of metadata in an IP packet is also possible 334 through the inclusion of an MPLS label stack and the MPLS DetNet 335 Control Word using one of the methods for carrying MPLS over IP 336 [I-D.ietf-detnet-mpls-over-udp-ip]. This is described in more detail 337 in Section 3.6.5. 339 Implicit metadata in IP can be included through the use of the 340 network programming paradigm 341 [I-D.ietf-spring-srv6-network-programming] in which the suffix of an 342 IPv6 address is used to encode additional information for use by the 343 network of the receiving host. 345 Some MPLS examples of implicit metadata include the sequence number 346 for use by the PREOF function, or even all the essential information 347 being included into the DetNet over MPLS label stack (the DetNet 348 Control Word and the DetNet Service label). 350 3.4. DetNet IP Data Plane 352 An IP data plane may operate natively or through the use of an 353 encapsulation. Many types of IP encapsulation can satisfy DetNet 354 requirements and it is anticipated that more than one encapsulation 355 may be deployed, for example GRE, IPSec etc. 357 One method of operating an IP DetNet data plane without encapsulation 358 is to use "6-tuple" based flow identification, where "6-tuple" refers 359 to information carried in IP and higher layer protocol headers. 360 General background on the use of IP headers, and "6-tuples", to 361 identify flows and support Quality of Service (QoS) can be found in 362 [RFC3670]. [RFC7657] provides useful background on differentiated 363 services (DiffServ) and "tuple" based flow identification. DetNet 364 flow aggregation may be enabled via the use of wildcards, masks, 365 prefixes and ranges. The operation of this method is described in 366 detail in [I-D.ietf-detnet-ip]. 368 The DetNet forwarding plane may use explicit route capabilities and 369 traffic engineering capabilities to provide a forwarding sub-layer 370 that is responsible for providing resource allocation and explicit 371 routes. It is possible to include such information in a native IP 372 packet explicitly, or implicitly. 374 3.5. DetNet MPLS Data Plane 376 MPLS provides the ability to forward traffic over implicit and 377 explicit paths to the point in the network where the next DetNet 378 service sub-layer action needs to take place. It does this through 379 the use of a stack of one or more labels with various forwarding 380 semantics. 382 MPLS also provides the ability to identify a service instance that is 383 used to process the packet through the use of a label that maps the 384 packet to a service instance. 386 In cases where metadata is needed to process an MPLS encapsulated 387 packet at the service sub-layer, a shim layer called a control word 388 (CW) [RFC4385] can be used. Although such CWs are frequently 32 bits 389 long, there is no architectural constraint on its size of this 390 structure, only the requirement that it is fully understood by all 391 parties operating on it in the DetNet service sub-layer. The 392 operation of this method is described in detail in 393 [I-D.ietf-detnet-mpls]. 395 3.6. Further DetNet Data Plane Considerations 397 This section provides informative considerations related to providing 398 DetNet service to flows which are identified based on their header 399 information. 401 3.6.1. Per Flow Related Functions 403 At a high level, the following functions are provided on a per flow 404 basis. 406 3.6.1.1. Reservation and Allocation of resources 408 Reservation of resources can allocate resources to specific DetNet 409 flows. This can eliminate packet contention and packet loss for 410 DetNet traffic. This also can reduce jitter for DetNet traffic. 411 Resources allocated to a DetNet flow protect it from other traffic 412 flows. On the other hand, DetNet flows are assumed to behave with 413 respect to the reserved traffic profile. Misbehaving DetNet flows 414 must be detected and it have to be ensured that they do not 415 compromise QoS of other flows. The use of (queuing, policing, 416 shaping) policies can be used to ensure that the allocation of 417 resources reserved for DetNet is met. 419 3.6.1.2. Explicit routes 421 Use of a specific path for a flow. This allows control of the 422 network delay by steering the packet with the ability to influence 423 the physical path. Explicit routes complement reservation by 424 ensuring that a consistent path can be associated with its resources 425 for the duration of that path. Coupled with the traffic mechanism, 426 this limits misordering and bounds latency. Explicit route 427 computation can encompass a wide set of constraints and optimize the 428 path for a certain characteristic e.g. highest bandwidth or lowest 429 jitter. In these cases the "best" path for any set of 430 characteristics may not be a shortest path. The selection of path 431 can take into account multiple network metrics. Some of these 432 metrics are measured and distributed by the routing system as traffic 433 engineering metrics. 435 3.6.1.3. Service protection 437 Use of multiple packet streams using multiple paths, for example 1+1 438 or 1:1 linear protection. For DetNet this primarily relates to 439 packet replication and elimination capabilities. MPLS offers a 440 number of protection schemes. MPLS hitless protection can be used to 441 switch traffic to an already established path in order to restore 442 delivery rapidly after a failure. Path changes, even in the case of 443 failure recovery, can lead to the out of order delivery of data 444 requiring packet ordering functions either within the DetNet service 445 or at a high layer in the application traffic. Establishment of new 446 paths after a failure is out of scope for DetNet services. 448 3.6.1.4. Network Coding 450 Network Coding, not to be confused with network programming, 451 comprises several techniques where multiple data flows are encoded. 452 These resulting flows can then be sent on different paths. The 453 encoding operation can combine flows and error recovery information. 454 When the encoded flows are decoded and recombined the original flows 455 can be recovered. Note that Network coding uses an alternative to 456 packet by packet PREOF. Therefore, for certain network topologies 457 and traffic loads, Network Coding can be used to improve a network's 458 throughput, efficiency, latency, and scalability, as well as 459 resilience to partition, attacks, and eavesdropping, as compared to 460 traditional methods. DetNet could utilized Network coding as an 461 alternative to other protection means. Network coding is often 462 applied in wireless networks and is being explored for other network 463 types. 465 3.6.1.5. Load sharing 467 Use of packet-by-packet distribution of the same DetNet flow over 468 multiple paths is not recommended except for the cases listed above 469 where PREOF is utilized to improve protection of traffic and maintain 470 order. Packet by packet load sharing, e.g., via ECMP or UCMP, 471 impacts ordering and possibly jitter. 473 3.6.1.6. Troubleshooting 475 Detnet leverages many different forwarding sub-layers, each of which 476 supports various tools to troubleshoot connectivity, for example 477 identification of misbehaving flows. The DetNet Service layer can 478 leverage existing mechanisms to troubleshoot or monitor flows, such 479 as those in use by IP and MPLS networks. At the Application layer a 480 client of a DetNet service can use existing techniques to detect and 481 monitor delay and loss. 483 3.6.1.7. Flow recognition for analytics 485 Network analytics can be inherited from the technologies of the 486 Service and Forwarding sub-layers. At the DetNet service edge, 487 packet and bit counters (e.g. sent, received, dropped, and out-of- 488 sequence) can be maintained. 490 3.6.1.8. Correlation of events with flows 492 The provider of a DetNet service may provide other capabilities to 493 monitor flows, such as more detailed loss statistics and time 494 stamping of events. The details of these capabilities are currently 495 out of scope for this document. 497 3.6.2. Service Protection 499 Service protection allow DetNet services to increase reliability and 500 maintain a DetNet Service Assurance in the case of network congestion 501 or network failure. Detnet relies on the underlying technology 502 capabilities for various protection schemes. Protection schemes 503 enable partial or complete coverage of the network paths and active 504 protection with combinations of PRF, PEF, and POF. 506 3.6.2.1. Linear Service Protection 508 An example DetNet MPLS network fragment and packet flow is 509 illustrated in Figure 3. 511 1 1.1 1.1 1.2.1 1.2.1 1.2.2 512 CE1----EN1--------R1-------R2-------R3--------EN2-----CE2 513 \ 1.2.1 / / 514 \1.2 /-----+ / 515 +------R4------------------------+ 516 1.2.2 518 Figure 3: Example Packet Flow in DetNet protected Network 520 In Figure 3 the numbers are used to identify the instance of a 521 packet. Packet 1 is the original packet, and packets 1.1, and 1.2 522 are two first generation copies of packet 1. Packet 1.2.1 is a 523 second generation copy of packet 1.2 etc. Note that these numbers 524 never appear in the packet, and are not to be confused with sequence 525 numbers, labels or any other identifier that appears in the packet. 526 They simply indicate the generation number of the original packet so 527 that its passage through the network fragment can be identified to 528 the reader. 530 Customer Equipment CE1 sends a packet into the DetNet enabled 531 network. This is packet (1). Edge Node EN1 encapsulates the packet 532 as a DetNet Packet and sends it to Relay node R1 (packet 1.1). EN1 533 makes a copy of the packet (1.2), encapsulates it and sends this copy 534 to Relay node R4. 536 Note that along the path from EN1 to R1 there may be zero or more 537 nodes which, for clarity, are not shown. The same is true for any 538 other path between two DetNet entities shown in Figure 3 . 540 Relay node R4 has been configured to send one copy of the packet to 541 Relay Node R2 (packet 1.2.1) and one copy to Edge Node EN2 (packet 542 1.2.2). 544 R2 receives packet copy 1.2.1 before packet copy 1.1 arrives, and, 545 having been configured to perform packet elimination on this DetNet 546 flow, forwards packet 1.2.1 to Relay Node R3. Packet copy 1.1 is of 547 no further use and so is discarded by R2. 549 Edge Node EN2 receives packet copy 1.2.2 from R4 before it receives 550 packet copy 1.2.1 from R2 via relay Node R3. EN2 therefore strips 551 any DetNet encapsulation from packet copy 1.2.2 and forwards the 552 packet to CE2. When EN2 receives the later packet copy 1.2.1 this is 553 discarded. 555 The above is of course illustrative of many network scenarios that 556 can be configured. 558 This example also illustrates 1:1 protection scheme meaning there is 559 traffic over each segment of the end to end path. Local DetNet relay 560 nodes determine which packets are eliminated and which packets are 561 forwarded. A 1+1 scheme where only one path is used for traffic at a 562 time, could use the same topology. In this case there is no PRF 563 function and traffic is switched upon detection of failure. An OAM 564 scheme that monitors the paths detects the loss of path or traffic is 565 required to initiate the switch. A POF may still be used in this 566 case to prevent misordering of packets. In both cases the protection 567 paths are established and maintained for the duration of the DetNet 568 service. 570 3.6.2.2. Ring Service Protection 572 Ring protection may also be supported if the underlying technology 573 supports it. Many of the same concepts apply however rings are 574 normally 1+1 protection for data efficiency reasons. [RFC8227] is an 575 example of MPLS-TP data plane that supports Ring protection. 577 3.6.3. Aggregation Considerations 579 The DetNet data plane also allows for the aggregation of DetNet 580 flows, which can improve scalability by reducing the per-hop state. 581 How this is accomplished is data plane or control plane dependent. 582 When DetNet flows are aggregated, transit nodes provide service to 583 the aggregate and not on a per-DetNet flow basis. When aggregating 584 DetNet flows the flows should be compatible i.e. the same or very 585 similar QoS and CoS characteristics. In this case, nodes performing 586 aggregation will ensure that per-flow service requirements are 587 achieved. 589 If bandwidth reservations are used, the sum of the reservations 590 should be the sum of all the individual reservations; in other words, 591 the reservations should not add up to an over-subscription of 592 bandwidth reservation. If maximum delay bounds are used, the system 593 should ensure that the aggregate does not exceed the delay bounds of 594 the individual flows. 596 When an encapsulation is used the choice of reserving a maximum 597 resource level and then tracking the services in the aggregated 598 service or adjusting the aggregated resources as the services are 599 added is implementation and technology specific. 601 DetNet flows at edges must be able to handle rejection to an 602 aggregation group due to lack of resources as well as conditions 603 where requirements are not satisfied. 605 3.6.3.1. IP Aggregation 607 IP aggregation has both data plane and controller plane aspects. For 608 the data plane, flows may be aggregated for treatment based on shared 609 characteristics such as 6-tuple. Alternatively, an IP encapsulation 610 may be used to tunnel an aggregate number of DetNet Flows between 611 relay nodes. 613 3.6.3.2. MPLS Aggregation 615 MPLS aggregation also has data plane and controller plane aspects. 616 MPLS flows are often tunneled in a forwarding sub-layer, under the 617 reservation associated with that MPLS tunnel. 619 3.6.4. End-System-Specific Considerations 621 Data-flows requiring DetNet service are generated and terminated on 622 end-systems. Encapsulation depends on the application and its 623 preferences. For example, in a DetNet MPLS domain the sub-layer 624 functions use the d-CWs, S-Labels and F-Labels to provide DetNet 625 services. However, an application may exchange further flow related 626 parameters (e.g., time-stamp), which are not provided by DetNet 627 functions. 629 As a general rule, DetNet domains are capable of forwarding any 630 DetNet flows and the DetNet domain does not mandate the end-system or 631 edge node encapsulation format. Unless there is a proxy of some form 632 present, end-systems peer with similar end-systems using the same 633 application encapsulation format. For example, as shown in Figure 4, 634 IP applications peer with IP applications and Ethernet applications 635 peer with Ethernet applications. 637 +-----+ 638 | X | +-----+ 639 +-----+ | X | 640 | Eth | ________ +-----+ 641 +-----+ _____ / \ | Eth | 642 \ / \__/ \___ +-----+ 643 \ / \ / 644 0======== tunnel-1 ========0_ 645 | \ 646 \ | 647 0========= tunnel-2 =========0 648 / \ __/ \ 649 +-----+ \__ DetNet MPLS domain / \ 650 | X | \ __ / +-----+ 651 +-----+ \_______/ \_____/ | X | 652 | IP | +-----+ 653 +-----+ | IP | 654 +-----+ 656 Figure 4: End-Systems and The DetNet MPLS Domain 658 3.6.5. Sub-Network Considerations 660 Any of the DetNet service types may be transported by another DetNet 661 service. MPLS nodes may interconnected by different sub-network 662 technologies, which may include point-to-point links. Each of these 663 sub-network technologies need to provide appropriate service to 664 DetNet flows. In some cases, e.g., on dedicated point-to-point links 665 or TDM technologies, all that is required is for a DetNet node to 666 appropriately queue its output traffic. In other cases, DetNet nodes 667 will need to map DetNet flows to the flow semantics (i.e., 668 identifiers) and mechanisms used by an underlying sub-network 669 technology. Figure 5 shows several examples of header formats that 670 can be used to carry DetNet MPLS flows over different sub-network 671 technologies. L2 represent a generic layer-2 encapsulation that 672 might be used on a point-to-point link. TSN represents the 673 encapsulation used on an IEEE 802.1 TSN network, as described in 674 [I-D.ietf-detnet-mpls-over-tsn]. UDP/IP represents the encapsulation 675 used on a DetNet IP PSN, as described in 676 [I-D.ietf-detnet-mpls-over-udp-ip]. 678 +------+ +------+ +------+ 679 App-Flow | X | | X | | X | 680 +-----+======+--+======+--+======+-----+ 681 DetNet-MPLS | d-CW | | d-CW | | d-CW | 682 +------+ +------+ +------+ 683 |Labels| |Labels| |Labels| 684 +-----+======+--+======+--+======+-----+ 685 Sub-Network | L2 | | TSN | | UDP | 686 +------+ +------+ +------+ 687 | IP | 688 +------+ 689 | L2 | 690 +------+ 692 Figure 5: Example DetNet MPLS Sub-Network Formats 694 4. Controller Plane (Management and Control) Considerations 696 4.1. DetNet Controller Plane Requirements 698 While the definition of controller plane for DetNet is out of the 699 scope of this document, there are particular considerations and 700 requirements for such that result from the unique characteristics of 701 the DetNet architecture [I-D.ietf-detnet-architecture] and data plane 702 as defined herein. 704 The primary requirements of the DetNet controller plane are that it 705 must be able to: 707 o Instantiate DetNet flows in a DetNet domain (which may include 708 some or all of explicit path determination, link bandwidth 709 reservations, restricting flows to IEEE 802.1 TSN links, node 710 buffer and other resource reservations, specification of required 711 queuing disciplines along the path, ability to manage 712 bidirectional flows, etc.) as needed for a flow. 714 o In the case of MPLS, manage DetNet S-Label and F-Label allocation 715 and distribution, where the DetNet MPLS encapsulation is in use 716 see [I-D.ietf-detnet-mpls]. 718 o Support DetNet flow aggregation. 720 o Advertise static and dynamic node and link resources such as 721 capabilities and adjacencies to other network nodes (for dynamic 722 signaling approaches) or to network controllers (for centralized 723 approaches). 725 o Scale to handle the number of DetNet flows expected in a domain 726 (which may require per-flow signaling or provisioning). 728 o Provision flow identification information at each of the nodes 729 along the path. Flow identification may differ depending on the 730 location in the network and the DetNet functionality (e.g. transit 731 node vs. relay node). 733 These requirements, as stated earlier, could be satisfied using 734 distributed control protocol signaling (such as RSVP-TE), centralized 735 network management provisioning mechanisms (such as BGP, PCEP, YANG 736 [I-D.ietf-detnet-flow-information-model], etc.) or hybrid 737 combinations of the two, and could also make use of MPLS-based 738 segment routing. 740 In the abstract, the results of either distributed signaling or 741 centralized provisioning are equivalent from a DetNet data plane 742 perspective - flows are instantiated, explicit routes are determined, 743 resources are reserved, and packets are forwarded through the domain 744 using the DetNet data plane. 746 However, from a practical and implementation standpoint, they are not 747 equivalent at all. Some approaches are more scalable than others in 748 terms of signaling load on the network. Some can take advantage of 749 global tracking of resources in the DetNet domain for better overall 750 network resource optimization. Some are more resilient than others 751 if link, node, or management equipment failures occur. While a 752 detailed analysis of the control plane alternatives is out of the 753 scope of this document, the requirements from this document can be 754 used as the basis of a later analysis of the alternatives. 756 4.2. Generic Controller Plane Considerations 758 This section covers control plane considerations that are independent 759 of the data plane technology used for DetNet service delivery. 761 While management plane and control planes are traditionally 762 considered separately, from the Data Plane perspective there is no 763 practical difference based on the origin of flow provisioning 764 information, and the DetNet architecture 765 [I-D.ietf-detnet-architecture] refers to these collectively as the 766 'Controller Plane'. This document therefore does not distinguish 767 between information provided by distributed control plane protocols, 768 e.g., RSVP-TE [RFC3209] and [RFC3473], or by centralized network 769 management mechanisms, e.g., RestConf [RFC8040], YANG [RFC7950], and 770 the Path Computation Element Communication Protocol (PCEP) 771 [I-D.ietf-pce-pcep-extension-for-pce-controller] or any combination 772 thereof. Specific considerations and requirements for the DetNet 773 Controller Plane are discussed in Section 4.1. 775 Each respective data plane document also covers the control plane 776 considerations for that technology. For example [I-D.ietf-detnet-ip] 777 covers IP control plane normative considerations and 778 [I-D.ietf-detnet-mpls] covers MPLS control plane normative 779 considerations. 781 4.2.1. Flow Aggregation Control 783 Flow aggregation means that multiple App-flows are served by a single 784 new DetNet flow. There are many techniques to achieve aggregation, 785 for example in case of IP, it can be grouping of IP flows that share 786 6-tuple attributes or flow identifiers at the DetNet sub-layer. 787 Another example includes aggregation accomplished through the use of 788 hierarchical LSPs in MPLS and tunnels. 790 Control of aggregation involves a set of procedures listed here. 791 Aggregation may use some or all of these capabilities and the order 792 may vary: 794 o Traffic engineering resource collection and distribution: 796 Available resources are tracked through control plane or 797 management plane databases and distributed amongst controllers 798 or nodes that can manage resources. 800 o Path computation and resource allocation: 802 When DetNet services are provisioned or requested one or more 803 paths meeting the requirements are selected and the resources 804 verified and recorded. 806 o Resource assignment and data plane co-ordination: 808 The assignment of resources along the path depends on the 809 technology and it includes assignment of specific links and 810 coordination of the queuing and other traffic management 811 capabilities such as policing and shaping. 813 o Assigned Resource recording and updating: 815 Depending on the specific technology, the assigned resources 816 are updated and distributed in the databases, preventing over- 817 subscription. 819 4.2.2. Explicit Routes 821 Explicit routes are used to ensure that packets are routed through 822 the resources that have been reserved for them, and hence provide the 823 DetNet application with the required service. A requirement for the 824 DetNet Controller Plane will be the ability to assign a particular 825 identified DetNet IP flow to a path through the DetNet domain that 826 has been assigned the required nodal resources. This provides the 827 appropriate traffic treatment for the flow and also includes 828 particular links as a part of the path that are able to support the 829 DetNet flow. For example, by using IEEE 802.1 TSN links (as 830 discussed in [I-D.ietf-detnet-mpls-over-tsn] ) DetNet parameters can 831 be maintained. Further considerations and requirements for the 832 DetNet Controller Plane are discussed in Section 4.1. 834 Whether configuring, calculating and instantiating these routes is a 835 single-stage or multi-stage process, or in a centralized or 836 distributed manner, is out of scope of this document. 838 There are several approaches that could be used to provide explicit 839 routes and resource allocation in the DetNet forwarding sub-layer. 840 For example: 842 o The path could be explicitly set up by a controller which 843 calculates the path and explicitly configures each node along that 844 path with the appropriate forwarding and resource allocation 845 information. 847 o The path could use a distributed control plane such as RSVP 848 [RFC2205] or RSVP-TE [RFC3473] extended to support DetNet IP 849 flows. 851 o The path could be implemented using IPv6-based segment routing 852 when extended to support resource allocation. 854 See Section 4.1 for further discussion of these alternatives. In 855 addition, [RFC2386] contains useful background information on QoS- 856 based routing, and [RFC5575] discusses a specific mechanism used by 857 BGP for traffic flow specification and policy-based routing. 859 4.2.3. Contention Loss and Jitter Reduction 861 As discussed in Section 1, this document does not specify the 862 mechanisms needed to eliminate packet contention, packet loss or 863 reduce jitter for DetNet flows at the DetNet forwarding sub-layer. 864 The ability to manage node and link resources to be able to provide 865 these functions is a necessary part of the DetNet controller plane. 866 It is also necessary to be able to control the required queuing 867 mechanisms used to provide these functions along a flow's path 868 through the network. See [I-D.ietf-detnet-ip] and Section 4.1 for 869 further discussion of these requirements. 871 4.2.4. Bidirectional Traffic 873 DetNet applications typically generate bidirectional traffic. IP and 874 MPLS typically treat each direction separately and do not force 875 interdependence of each direction. MPLS has considered bidirectional 876 traffic requirements and the MPLS definitions from [RFC5654] are 877 useful to illustrate terms such as associated bidirectional flows and 878 co-routed bidirectional flows. MPLS defines a point-to-point 879 associated bidirectional LSP as consisting of two unidirectional 880 point-to-point LSPs, one from A to B and the other from B to A, which 881 are regarded as providing a single logical bidirectional forwarding 882 path. This is analogous to standard IP routing. MPLS defines a 883 point-to-point co-routed bidirectional LSP as an associated 884 bidirectional LSP which satisfies the additional constraint that its 885 two unidirectional component LSPs follow the same path (in terms of 886 both nodes and links) in both directions. An important property of 887 co-routed bidirectional LSPs is that their unidirectional component 888 LSPs share fate. In both types of bidirectional LSPs, resource 889 reservations may differ in each direction. The concepts of 890 associated bidirectional flows and co-routed bidirectional flows can 891 also be applied to DetNet IP flows. 893 While the DetNet IP data plane must support bidirectional DetNet 894 flows, there are no special bidirectional features with respect to 895 the data plane other than the need for the two directions of a co- 896 routed bidirectional flow to take the same path. That is to say that 897 bidirectional DetNet flows are solely represented at the management 898 and control plane levels, without specific support or knowledge 899 within the DetNet data plane. Fate sharing and associated or co- 900 routed bidirectional flows, can be managed at the control level. 902 DetNet's use of PREOF may increase the complexity of using co-routing 903 bidirectional flows, since if PREOF is used, then the replication 904 points in one direction would have to match the elimination points in 905 the other direction, and vice versa. In such cases the optimal 906 points for these functions in one direction may not match the optimal 907 points in the other, due to network and traffic constraints. 908 Furthermore, due to the per packet service protection nature, 909 bidirectional forwarding per packet may not be ensured. The first 910 packet of received member flows is selected by the elimination 911 function independently of which path it has taken through the 912 network. 914 Control and management mechanisms need to support bidirectional 915 flows, but the specification of such mechanisms are out of scope of 916 this document. An example control plane solution for MPLS can be 917 found in [RFC3473] , [RFC6387] and [RFC7551]. These requirements are 918 included in Section 4.1. 920 4.3. Packet Replication, Elimination, and Ordering (PREOF) 922 The controller plane protocol solution required for managing the 923 PREOF processing is outside the scope of this document. That said, 924 it should be noted that the ability to determine, for a particular 925 flow, optimal packet replication and elimination points in the DetNet 926 domain requires explicit support. There may be capabilities that can 927 be used, or extended, for example GMPLS end-to-end recovery [RFC4872] 928 and GMPLS segment recovery [RFC4873]. 930 5. Security Considerations 932 Security considerations for DetNet are described in detail in 933 [I-D.ietf-detnet-security]. General security considerations are 934 described in [I-D.ietf-detnet-architecture]. This section considers 935 general security considerations applicable to all data planes. 937 Security aspects which are unique to DetNet are those whose aim is to 938 provide the specific quality of service aspects of DetNet, which are 939 primarily to deliver data flows with extremely low packet loss rates 940 and bounded end-to-end delivery latency. 942 The primary considerations for the data plane is to maintain 943 integrity of data and delivery of the associated DetNet service 944 traversing the DetNet network. Application flows can be protected 945 through whatever means is provided by the underlying technology. For 946 example, encryption may be used, such as that provided by IPSec 947 [RFC4301] for IP flows and/or by an underlying sub-net using MACSec 948 [IEEE802.1AE-2018] for Ethernet (Layer-2) flows. 950 At the management and control level DetNet flows are identified on a 951 per-flow basis, which may provide controller plane attackers with 952 additional information about the data flows (when compared to 953 controller planes that do not include per-flow identification). This 954 is an inherent property of DetNet which has security implications 955 that should be considered when determining if DetNet is a suitable 956 technology for any given use case. 958 To provide uninterrupted availability of the DetNet service, 959 provisions can be made against DOS attacks and delay attacks. To 960 protect against DOS attacks, excess traffic due to malicious or 961 malfunctioning devices can be prevented or mitigated, for example 962 through the use of existing mechanism such as policing and shaping 963 applied at the input of a DetNet domain. To prevent DetNet packets 964 from being delayed by an entity external to a DetNet domain, DetNet 965 technology definition can allow for the mitigation of Man-In-The- 966 Middle attacks, for example through use of authentication and 967 authorization of devices within the DetNet domain. 969 In order to prevent or mitigate DetNet attacks on other networks via 970 flow escape, edge devices can for example use existing mechanism such 971 as policing and shaping applied at the output of a DetNet domain. 973 6. IANA Considerations 975 This document makes no IANA requests. 977 7. Acknowledgements 979 The authors wish to thank Pat Thaler, Norman Finn, Loa Anderson, 980 David Black, Rodney Cummings, Ethan Grossman, Tal Mizrahi, David 981 Mozes, Craig Gunther, George Swallow, Yuanlong Jiang and Carlos J. 982 Bernardos for their various contributions to this work. 984 8. References 986 8.1. Normative References 988 [RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V., 989 and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP 990 Tunnels", RFC 3209, DOI 10.17487/RFC3209, December 2001, 991 . 993 [RFC3473] Berger, L., Ed., "Generalized Multi-Protocol Label 994 Switching (GMPLS) Signaling Resource ReserVation Protocol- 995 Traffic Engineering (RSVP-TE) Extensions", RFC 3473, 996 DOI 10.17487/RFC3473, January 2003, 997 . 999 [RFC4385] Bryant, S., Swallow, G., Martini, L., and D. McPherson, 1000 "Pseudowire Emulation Edge-to-Edge (PWE3) Control Word for 1001 Use over an MPLS PSN", RFC 4385, DOI 10.17487/RFC4385, 1002 February 2006, . 1004 8.2. Informative References 1006 [I-D.ietf-detnet-architecture] 1007 Finn, N., Thubert, P., Varga, B., and J. Farkas, 1008 "Deterministic Networking Architecture", draft-ietf- 1009 detnet-architecture-13 (work in progress), May 2019. 1011 [I-D.ietf-detnet-flow-information-model] 1012 Farkas, J., Varga, B., Cummings, R., Jiang, Y., and D. 1013 Fedyk, "DetNet Flow Information Model", draft-ietf-detnet- 1014 flow-information-model-05 (work in progress), September 1015 2019. 1017 [I-D.ietf-detnet-ip] 1018 Varga, B., Farkas, J., Berger, L., Fedyk, D., Malis, A., 1019 Bryant, S., and J. Korhonen, "DetNet Data Plane: IP", 1020 draft-ietf-detnet-ip-01 (work in progress), July 2019. 1022 [I-D.ietf-detnet-mpls] 1023 Varga, B., Farkas, J., Berger, L., Fedyk, D., Malis, A., 1024 Bryant, S., and J. Korhonen, "DetNet Data Plane: MPLS", 1025 draft-ietf-detnet-mpls-01 (work in progress), July 2019. 1027 [I-D.ietf-detnet-mpls-over-tsn] 1028 Varga, B., Farkas, J., Malis, A., Bryant, S., and J. 1029 Korhonen, "DetNet Data Plane: MPLS over IEEE 802.1 Time 1030 Sensitive Networking (TSN)", draft-ietf-detnet-mpls-over- 1031 tsn-00 (work in progress), May 2019. 1033 [I-D.ietf-detnet-mpls-over-udp-ip] 1034 Varga, B., Farkas, J., Berger, L., Malis, A., Bryant, S., 1035 and J. Korhonen, "DetNet Data Plane: MPLS over UDP/IP", 1036 draft-ietf-detnet-mpls-over-udp-ip-02 (work in progress), 1037 October 2019. 1039 [I-D.ietf-detnet-security] 1040 Mizrahi, T., Grossman, E., Hacker, A., Das, S., Dowdell, 1041 J., Austad, H., Stanton, K., and N. Finn, "Deterministic 1042 Networking (DetNet) Security Considerations", draft-ietf- 1043 detnet-security-05 (work in progress), August 2019. 1045 [I-D.ietf-pce-pcep-extension-for-pce-controller] 1046 Zhao, Q., Li, Z., Negi, M., and C. Zhou, "PCEP Procedures 1047 and Protocol Extensions for Using PCE as a Central 1048 Controller (PCECC) of LSPs", draft-ietf-pce-pcep- 1049 extension-for-pce-controller-02 (work in progress), July 1050 2019. 1052 [I-D.ietf-spring-srv6-network-programming] 1053 Filsfils, C., Camarillo, P., Leddy, J., 1054 daniel.voyer@bell.ca, d., Matsushima, S., and Z. Li, "SRv6 1055 Network Programming", draft-ietf-spring-srv6-network- 1056 programming-05 (work in progress), October 2019. 1058 [IEEE802.1AE-2018] 1059 IEEE Standards Association, "IEEE Std 802.1AE-2018 MAC 1060 Security (MACsec)", 2018, 1061 . 1063 [IEEE802.1TSNTG] 1064 IEEE Standards Association, "IEEE 802.1 Time-Sensitive 1065 Networking Task Group", . 1067 [RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S. 1068 Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1 1069 Functional Specification", RFC 2205, DOI 10.17487/RFC2205, 1070 September 1997, . 1072 [RFC2386] Crawley, E., Nair, R., Rajagopalan, B., and H. Sandick, "A 1073 Framework for QoS-based Routing in the Internet", 1074 RFC 2386, DOI 10.17487/RFC2386, August 1998, 1075 . 1077 [RFC3670] Moore, B., Durham, D., Strassner, J., Westerinen, A., and 1078 W. Weiss, "Information Model for Describing Network Device 1079 QoS Datapath Mechanisms", RFC 3670, DOI 10.17487/RFC3670, 1080 January 2004, . 1082 [RFC4301] Kent, S. and K. Seo, "Security Architecture for the 1083 Internet Protocol", RFC 4301, DOI 10.17487/RFC4301, 1084 December 2005, . 1086 [RFC4872] Lang, J., Ed., Rekhter, Y., Ed., and D. Papadimitriou, 1087 Ed., "RSVP-TE Extensions in Support of End-to-End 1088 Generalized Multi-Protocol Label Switching (GMPLS) 1089 Recovery", RFC 4872, DOI 10.17487/RFC4872, May 2007, 1090 . 1092 [RFC4873] Berger, L., Bryskin, I., Papadimitriou, D., and A. Farrel, 1093 "GMPLS Segment Recovery", RFC 4873, DOI 10.17487/RFC4873, 1094 May 2007, . 1096 [RFC5575] Marques, P., Sheth, N., Raszuk, R., Greene, B., Mauch, J., 1097 and D. McPherson, "Dissemination of Flow Specification 1098 Rules", RFC 5575, DOI 10.17487/RFC5575, August 2009, 1099 . 1101 [RFC5654] Niven-Jenkins, B., Ed., Brungard, D., Ed., Betts, M., Ed., 1102 Sprecher, N., and S. Ueno, "Requirements of an MPLS 1103 Transport Profile", RFC 5654, DOI 10.17487/RFC5654, 1104 September 2009, . 1106 [RFC6387] Takacs, A., Berger, L., Caviglia, D., Fedyk, D., and J. 1107 Meuric, "GMPLS Asymmetric Bandwidth Bidirectional Label 1108 Switched Paths (LSPs)", RFC 6387, DOI 10.17487/RFC6387, 1109 September 2011, . 1111 [RFC7551] Zhang, F., Ed., Jing, R., and R. Gandhi, Ed., "RSVP-TE 1112 Extensions for Associated Bidirectional Label Switched 1113 Paths (LSPs)", RFC 7551, DOI 10.17487/RFC7551, May 2015, 1114 . 1116 [RFC7657] Black, D., Ed. and P. Jones, "Differentiated Services 1117 (Diffserv) and Real-Time Communication", RFC 7657, 1118 DOI 10.17487/RFC7657, November 2015, 1119 . 1121 [RFC7950] Bjorklund, M., Ed., "The YANG 1.1 Data Modeling Language", 1122 RFC 7950, DOI 10.17487/RFC7950, August 2016, 1123 . 1125 [RFC8040] Bierman, A., Bjorklund, M., and K. Watsen, "RESTCONF 1126 Protocol", RFC 8040, DOI 10.17487/RFC8040, January 2017, 1127 . 1129 [RFC8227] Cheng, W., Wang, L., Li, H., van Helvoort, H., and J. 1130 Dong, "MPLS-TP Shared-Ring Protection (MSRP) Mechanism for 1131 Ring Topology", RFC 8227, DOI 10.17487/RFC8227, August 1132 2017, . 1134 Authors' Addresses 1136 Balazs Varga (editor) 1137 Ericsson 1138 Magyar Tudosok krt. 11. 1139 Budapest 1117 1140 Hungary 1142 Email: balazs.a.varga@ericsson.com 1144 Janos Farkas 1145 Ericsson 1146 Magyar Tudosok krt. 11. 1147 Budapest 1117 1148 Hungary 1150 Email: janos.farkas@ericsson.com 1151 Lou Berger 1152 LabN Consulting, L.L.C. 1154 Email: lberger@labn.net 1156 Don Fedyk 1157 LabN Consulting, L.L.C. 1159 Email: dfedyk@labn.net 1161 Andrew G. Malis 1162 Independent 1164 Email: agmalis@gmail.com 1166 Stewart Bryant 1167 Futurewei Technologies 1169 Email: stewart.bryant@gmail.com 1171 Jouni Korhonen 1173 Email: jouni.nospam@gmail.com