idnits 2.17.1 draft-ietf-teas-enhanced-vpn-05.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There is 1 instance of too long lines in the document, the longest one being 1 character in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (February 18, 2020) is 1528 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'RFC2702' is mentioned on line 927, but not defined == Missing Reference: 'APL' is mentioned on line 1216, but not defined == Unused Reference: 'I-D.ietf-idr-bgp-ls-segment-routing-ext' is defined on line 1587, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-teas-yang-te' is defined on line 1607, but no explicit reference was found in the text == Unused Reference: 'I-D.www-bess-yang-vpn-service-pm' is defined on line 1617, but no explicit reference was found in the text == Unused Reference: 'RFC2992' is defined on line 1631, but no explicit reference was found in the text == Outdated reference: A later version (-24) exists of draft-ietf-teas-actn-vn-yang-07 == Outdated reference: A later version (-15) exists of draft-ietf-teas-te-service-mapping-yang-02 == Outdated reference: A later version (-18) exists of draft-ietf-idr-bgp-ls-segment-routing-ext-16 == Outdated reference: A later version (-18) exists of draft-ietf-opsawg-l3sm-l3nm-01 == Outdated reference: A later version (-13) exists of draft-ietf-opsawg-ntf-02 == Outdated reference: A later version (-12) exists of draft-ietf-teas-sf-aware-topo-model-04 == Outdated reference: A later version (-36) exists of draft-ietf-teas-yang-te-22 == Outdated reference: A later version (-06) exists of draft-www-bess-yang-vpn-service-pm-04 Summary: 1 error (**), 0 flaws (~~), 15 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 TEAS working group J. Dong 2 Internet-Draft Huawei 3 Intended status: Informational S. Bryant 4 Expires: July 2020 Futurewei 5 Z. Li 6 China Mobile 7 T. Miyasaka 8 KDDI Corporation 9 Y.Lee 10 Sung Kyun Kwan University 11 February 18, 2020 13 A Framework for Enhanced Virtual Private Networks (VPN+) Services 15 draft-ietf-teas-enhanced-vpn-05 17 Abstract 19 This document describes the framework for Enhanced Virtual Private 20 Network (VPN+) service. The purpose is to support the needs of new 21 applications, particularly applications that are associated with 5G 22 services, by utilizing an approach that is based on existing VPN and 23 TE technologies and adds features that specific services require 24 over and above traditional VPNs. 26 Typically, VPN+ will be used to form the underpinning of network 27 slicing, but could also be of use in its own right providing 28 enhanced connectivity services between customer sites. 30 It is envisaged that enhanced VPNs will be delivered using a 31 combination of existing, modified, and new networking technologies. 32 This document provides an overview of relevant technologies and 33 identifies some areas for potential new work. 35 It is not envisaged that quite large numbers of VPN+ services will be 36 deployed in a network and, in particular, it is not intended that 37 all VPNs supported by a network will use VPN+ related techniques. 39 Status of This Memo 41 This Internet-Draft is submitted in full conformance with the 42 provisions of BCP 78 and BCP 79. 44 Internet-Drafts are working documents of the Internet Engineering 45 Task Force (IETF). Note that other groups may also distribute 46 working documents as Internet-Drafts. The list of current Internet- 47 Drafts is at https://datatracker.ietf.org/drafts/current/. 49 Internet-Drafts are draft documents valid for a maximum of six 50 months and may be updated, replaced, or obsoleted by other documents 51 at any time. It is inappropriate to use Internet-Drafts as reference 52 material or to cite them other than as "work in progress." 54 This Internet-Draft will expire on August 18, 2020. 56 Copyright Notice 58 Copyright (c) 2020 IETF Trust and the persons identified as the 59 document authors. All rights reserved. 61 This document is subject to BCP 78 and the IETF Trust's Legal 62 Provisions Relating to IETF Documents 63 (https://trustee.ietf.org/license-info) in effect on the date of 64 publication of this document. Please review these documents 65 carefully, as they describe your rights and restrictions with 66 respect to this document. Code Components extracted from this 67 document must include Simplified BSD License text as described in 68 Section 4.e of the Trust Legal Provisions and are provided without 69 warranty as described in the Simplified BSD License. 71 Table of Contents 73 1. Introduction ................................................ 3 74 2. Terminologies ............................................... 6 75 3. Overview of the Requirements ............................... 7 76 3.1. Isolation between Enhanced VPN Services ................ 7 77 3.1.1. A Pragmatic Approach to Isolation ................. 8 78 3.2. Performance Guarantee .................................. 9 79 3.3. Integration ........................................... 11 80 3.3.1. Abstraction ...................................... 11 81 3.4. Dynamic Management .................................... 12 82 3.5. Customized Control .................................... 12 83 3.6. Applicability ......................................... 13 84 3.7. Inter-Domain and Inter-Layer Network .................. 13 85 4. Architecture of Enhanced VPN ............................... 13 86 4.1. Layered Architecture ................................. 15 87 4.2. Multi-Point to Multi-Point (MP2MP) Connectivity ....... 17 88 4.3. Application Specific Network Types .................... 18 89 4.4. Scaling Considerations ................................ 18 90 5. Candidate Technologies ..................................... 19 91 5.1. Layer-Two Data Plane .................................. 19 92 5.1.1. Flexible Ethernet ................................ 19 93 5.1.2. Dedicated Queues ................................. 20 94 5.1.3. Time Sensitive Networking ........................ 20 95 5.2. Layer-Three Data Plane ................................ 21 96 5.2.1. Deterministic Networking ......................... 21 97 5.2.2. MPLS Traffic Engineering (MPLS-TE) ............... 21 98 5.2.3. Segment Routing .................................. 21 99 5.3. Non-Packet Data Plane ................................. 22 100 5.4. Control Plane ......................................... 22 101 5.5. Management Plane ...................................... 23 102 5.6. Applicability of Service Data Models to Enhanced VPN .. 23 103 5.6.1. Enhanced VPN Delivery in the ACTN Architecture ... 24 104 5.6.2. Enhanced VPN Features with Service Data Models ... 25 105 5.6.3. 5G Transport Service Delivery via Coordinated Data 106 Modules ................................................. 27 107 6. Scalability Considerations ................................. 29 108 6.1. Maximum Stack Depth of SR ............................. 30 109 6.2. RSVP Scalability ...................................... 30 110 6.3. SDN Scaling ........................................... 30 111 7. OAM Considerations ......................................... 30 112 8. Telemetry Considerations ................................... 31 113 9. Enhanced Resiliency ........................................ 31 114 10. Operational Considerations ................................ 33 115 11. Security Considerations ................................... 33 116 12. IANA Considerations ....................................... 33 117 13. Contributors .............................................. 34 118 14. Acknowledgments ........................................... 34 119 15. References ................................................ 34 120 15.1. Normative References ................................. 34 121 15.2. Informative References ............................... 36 122 Authors' Addresses ............................................ 40 124 1. Introduction 126 Virtual private networks (VPNs) have served the industry well as a 127 means of providing different groups of users with logically isolated 128 connectivity over a common network. The common or base network that 129 is used to provide the VPNs is often referred to as the underlay, 130 and the VPN is often called an overlay. 132 Customers of a network operator may request a connectivity services 133 with advanced characteristics such as enhanced isolation from other 134 services so that changes in some other service (such as changes in 135 network load, or events such as congestion or outages) have no or 136 acceptable effect on the throughput or latency of the services 137 provided to the customer. These services are "enhanced VPNs" (known 138 as VPN+) in that they are similar to VPN services as they provide 139 the customer with required connectivity, but have enhanced 140 characteristics. 142 Driven largely by needs surfacing from 5G, the concept of network 143 slicing has gained traction [NGMN-NS-Concept] [TS23501] [TS28530] 144 [BBF-SD406]. According to [TS28530], a 5G end-to-end network slice 145 consists of three major types network segments: Radio Access Network 146 (RAN), Transport Network (TN) and Mobile Core Network (CN). The 147 transport network provides the required connectivity between 148 different entities in RAN and CN segments of an end-to-end network 149 slice, with specific performance commitment. 151 A transport network slice is a virtual (logical) network with a 152 particular network topology and a set of shared or dedicated network 153 resources, which are used to provide the network slice consumer with 154 the required connectivity, appropriate isolation and specific 155 Service Level Agreement (SLA) or Service Level Objective (SLO). 157 A transport network slice could span multiple technologies (such as 158 IP or Optical) and multiple administrative domains. Depending on the 159 consumer's requirement, a transport network slice could be isolated 160 from other, often concurrent transport network slices in terms of 161 data plane, control plane, and management plane resources. 163 In this document the term "network slice" refers to a transport 164 network slice, and is considered as one typical use case of enhanced 165 VPN. 167 Network slicing builds on the concept of resource management, 168 network virtualization, and abstraction to provide performance 169 assurance, flexibility, programmability and modularity. It may use 170 techniques such as Software Defined Networking (SDN) [RFC7149], 171 network abstraction [RFC7926] and Network Function Virtualization 172 (NFV) [RFC8172] [RFC8568] to create multiple logical (virtual) 173 networks, each tailored for a set of services or a particular tenant 174 or a group of tenants that share the same or similar set of 175 requirements, on top of a common network. How the network slices 176 are engineered can be deployment-specific. 178 VPN+ could be used to form the underpinning of transport network 179 slice, but could also be of use in general cases providing enhanced 180 connectivity services between customer sites. 182 The requirement of enhanced VPN services cannot be met by simple 183 overlay networks, as they require tighter coordination and 184 integration between the underlay and the overlay network. VPN+ is 185 built from a VPN overlay and a underlying Virtual Transport Network 186 (VTN) which has a customized network topology and a set of dedicated 187 or shared network resources. It may optionally include a set of 188 invoked service functions allocated from the underlay network. Thus 189 an enhanced VPN can achieve greater isolation with strict 190 performance guarantees. These new properties, which have general 191 applicability, may also be of interest as part of a network slicing 192 solution. It is not envisaged that VPN+ services will replace 193 traditional VPN services that can continue to be deployed using pre- 194 existing mechanisms. 196 This document specifies a framework for using 197 existing, modified, and potential new technologies as components to 198 provide a VPN+ service. Specifically we are concerned with: 200 o The design of the enhanced data plane. 202 o The necessary protocols in both the underlay and the overlay 203 of the enhanced VPN. 205 o The mechanisms to achieve integration between overlay and 206 underlay. 208 o The necessary Operation, Administration, and Management (OAM) 209 methods to instrument an enhanced VPN to make sure that the required 210 Service Level Agreement (SLA) is met, and to take any corrective 211 action to avoid SLA violation, such as switching to an alternate 212 path. 214 The required layered network structure to achieve this is shown in 215 Section 4.1. 217 Note that, in this document, the relationship of the four terms 218 "VPN", "Enhanced VPN" (or "VPN+"), "Virtual Transport Network (VTN)", 219 and "Network Slice" are described as below: 221 o An enhanced VPN (VPN+) can be considered as an evolution of 222 VPN service, but with additional service-specific commitments. Thus, 223 care must be taken with the term "VPN" to distinguish normal or 224 legacy VPNs from VPN+ services. 226 o A Virtual Transport Network (VTN) is a virtual underlay 227 network that connects customer edge points with the additional 228 capability of providing the isolation and performance 229 characteristics required by an enhanced VPN customer. 231 o An enhanced VPN (VPN+) is made by integrating an overlay VPN 232 and an VTN with a set of network resources allocated in the underlay 233 network. 235 o A network slice in transport network could be provided with an 236 enhanced VPN (VPN+). 238 2. Terminologies 240 The following terms are used in this document. Some of them are 241 newly defined, some others reference existing definitions: 243 ACTN: Abstraction and Control of TE Networks [RFC8453] 245 Detnet: Deterministic Networking [DETNET] 247 FlexE: Flexible Ethernet [FLEXE] 249 TSN: Time Sensitive Networking [TSN] 251 VN: Virtual Network [I-D.ietf-teas-actn-vn-yang] 253 VPN: Virtual Private Network. IPVPN is defined in [RFC2764], L2VPN 254 is defined in [RFC4664] 256 VPN+: Enhanced VPN service. An enhanced VPN service (VPN+) can be 257 considered as an evolution of VPN service, but with additional 258 service-specific commitments such as enhanced isolation and 259 performance guarantee. 261 VTP: Virtual Transport Path. A VTP is a virtual underlay path which 262 connects two customer edge points with the capability of providing 263 the isolation and performance characteristics required by an 264 enhanced VPN customer. A VTP usually has a customized path with a 265 set of reserved network resources along the path. 267 VTN: Virtual Transport Network. A VTN is a virtual underlay network 268 that connects customer edge points with the capability of providing 269 the isolation and performance characteristics required by an 270 enhanced VPN customer. A VTN usually has a customized topology and a 271 set of dedicated or shared network resources. 273 3. Overview of the Requirements 275 In this section we provide an overview of the requirements of an 276 enhanced VPN service. 278 3.1. Isolation between Enhanced VPN Services 280 One element of the SLA demanded for an enhanced VPN is a guarantee 281 that the service offered to the customer will not be perturbed by 282 any other traffic flows in the network. One way for a service 283 provider to guarantee the customer's SLA is by controlling the 284 degree of isolation from other services in the network. Isolation 285 is a feature that can be requested by customers. There are different 286 grades of how isolation may be enabled by a network operator and 287 that may result in different levels of service perceived by the 288 customer. These range from simple separation of service traffic on 289 delivery (ensuring that traffic is not delivered to the wrong 290 customer), all the way to complete separation within the underlay so 291 that the traffic from different services use distinct network 292 resources. 294 The terms hard and soft isolation are used to identify different 295 levels of isolation. A VPN has soft isolation if the traffic of one 296 VPN cannot be received by the customers of another VPN. Both IP and 297 MPLS VPNs are examples of VPNs with soft isolation: the network 298 delivers the traffic only to the required VPN endpoints. However, 299 with soft isolation, traffic from VPNs and regular non-VPN traffic 300 may congest the network resulting in packet loss and delay for other 301 VPNs operating normally. The ability for a VPN service or a group 302 of VPN services to be sheltered from this effect is called hard 303 isolation, and this property is required by some applications. Hard 304 isolation is needed so that applications with exacting requirements 305 can function correctly, despite other demands (perhaps a burst of 306 traffic in another VPN) competing for the underlying resources. In 307 practice isolation may be offered as a spectrum between soft and 308 hard, and in some cases soft and hard isolation may be used in a 309 hierarchical manner. An operator may offer its customers a choice 310 of different degrees of isolation ranging from soft isolation up to 311 hard isolation. 313 An example of the requirement for hard isolation is a network 314 supporting both emergency services and public broadband multi-media 315 services. During a major incident the VPNs supporting these 316 services would both be expected to experience high data volumes, and 317 it is important that both make progress in the transmission of their 318 data. In these circumstances the VPN services would require an 319 appropriate degree of isolation to be able to continue to operate 320 acceptably. On the other hand, VPNs servicing ordinary bulk data 321 may expect to contest for network resources and queue packets so 322 that traffic is delivered within SLAs, but with some potential 323 delays and interference. 325 In order to provide the required level of isolation, resources may 326 have to be reserved in the data plane of the underlay network and 327 dedicated to traffic from a specific VPN or a specific group of VPNs 328 to form different enhanced VPNs in the network. This may introduce 329 scalability concerns, thus some trade-off needs to be considered to 330 provide the required isolation between some enhanced VPNs while 331 still allowing reasonable sharing. 333 An optical layer can offer a high degree of isolation, at the cost 334 of allocating resources on a long term and end-to-end basis. On the 335 other hand, where adequate isolation can be achieved at the packet 336 layer, this permits the resources to be shared amongst a group of 337 services and only dedicated to a service on a temporary basis. 339 There are several new technologies that provide some assistance with 340 these data plane issues. Firstly there is the IEEE project on Time 341 Sensitive Networking [TSN] which introduces the concept of packet 342 scheduling of delay and loss sensitive packets. Then there is 343 [FLEXE] which provides the ability to multiplex multiple channels 344 over one or more Ethernet links in a way that provides hard 345 isolation. Finally there are advanced queueing approaches which 346 allow the construction of virtual sub-interfaces, each of which is 347 provided with dedicated resource in a shared physical interface. 348 These approaches are described in more detail later in this document. 350 Section 3.1.1 explores pragmatic approaches to isolation in packet 351 networks. 353 3.1.1. A Pragmatic Approach to Isolation 355 A key question is whether it is possible to achieve hard isolation 356 in packet networks that were never designed to support hard 357 isolation. On the contrary, they were designed to provide 358 statistical multiplexing, a significant economic advantage when 359 compared to a dedicated, or a Time Division Multiplexing (TDM) 360 network. However, there is no need to provide any harder isolation 361 than is required by the applications. An approximation to this 362 requirement is sufficient in most cases. Pseudowires [RFC3985] 363 emulate services that would have had hard isolation in their native 364 form. 366 This spectrum of isolation is shown in Figure 1: 368 O=================================================O 369 | \---------------v---------------/ 370 Statistical Pragmatic Absolute 371 Multiplexing Isolation Isolation 372 (Traditional VPNs) (Enhanced VPN) (Dedicated Network) 374 Figure 1 The Spectrum of Isolation 376 Figure 1 shows the spectrum of isolation that may be delivered by a 377 network. At one end of the figure, we have traditional statistical 378 multiplexing technologies that support VPNs. This is a service type 379 that has served the industry well and will continue to do so. At 380 the opposite end of the spectrum, we have the absolute isolation 381 provided by dedicated transport networks. The goal of enhanced VPNs 382 is "pragmatic isolation". This is isolation that is better than is 383 obtainable from pure statistical multiplexing, more cost effective 384 and flexible than a dedicated network, but which is a practical 385 solution that is good enough for the majority of applications. 386 Mechanisms for both soft isolation and hard isolation would be 387 needed to meet different levels of service requirement. 389 3.2. Performance Guarantee 391 There are several kinds of performance guarantee, including 392 guaranteed maximum packet loss, guaranteed maximum delay, and 393 guaranteed delay variation. Note that these guarantees apply to 394 conformance traffic, out-of-profile traffic will be handled 395 according to other requirements. 397 Guaranteed maximum packet loss is a common parameter, and is usually 398 addressed by setting packet priorities, queue size, and discard 399 policy. However this becomes more difficult when the requirement is 400 combined with latency requirements. The limiting case is zero 401 congestion loss, and that is the goal of the Deterministic 402 Networking work that the IETF [DETNET] and IEEE [TSN] are pursuing. 403 In modern optical networks, loss due to transmission errors already 404 approaches zero, but there are the possibilities of failure of the 405 interface or the fiber itself. This can only be addressed by some 406 form of signal duplication and transmission over diverse paths. 408 Guaranteed maximum latency is required in a number of applications 409 particularly real-time control applications and some types of 410 virtual reality applications. The work of the IETF Deterministic 411 Networking (DetNet) Working Group [DETNET] is relevant; however 412 additional methods of enhancing the underlay to better support the 413 delay guarantees may be needed, and these methods will need to be 414 integrated with the overall service provisioning mechanisms. 416 Guaranteed maximum delay variation is a service that may also be 417 needed. [RFC8578] calls up a number of cases where this is needed, 418 for example in electrical utilities. Time transfer is one example of 419 a service that needs this, although it is in the nature of time that 420 the service might be delivered by the underlay as a shared service 421 and not provided through different enhanced VPNs. Alternatively a 422 dedicated enhanced VPN may be used to provide this as a shared 423 service. 425 This suggests that a spectrum of service guarantee be considered 426 when deploying an enhanced VPN. As a guide to understanding the 427 design requirements we can consider four types: 429 o Best effort 431 o Assured bandwidth 433 o Guaranteed latency 435 o Enhanced delivery 437 Best effort service is the basic service that current VPNs provide. 439 An assured bandwidth service is one in which the bandwidth over some 440 period of time is assured. This can be achieved either simply based 441 on best effort with over-capacity provisioning, or it can be based 442 on TE-LSPs with bandwidth reservation. The instantaneous bandwidth 443 is however, not necessarily assured, depending on the technique used. 444 Providing assured bandwidth to VPNs, for example by using per-VPN 445 TE-LSPs, is not widely deployed at least partially due to 446 scalability concerns. 448 Guaranteed latency and enhanced delivery are not yet integrated with 449 VPNs. A guaranteed latency service has a latency upper bound 450 provided by the network. Assuring the upper bound is sometimes more 451 important than minimizing latency. 453 There are several new technologies that provide some assistance with 454 performance guarantee. Firstly there is the IEEE project on Time 455 Sensitive Networking [TSN] which introduces the concept of packet 456 scheduling of delay and loss sensitive packets. Then the DetNet work 457 is also of greater relevance in assuring upper bound of end-to-end 458 packet latency. Flex Ethernet [FLEXE] is also useful to provide 459 these guarantees. 461 An enhanced delivery service is one in which the underlay network 462 (at Layer 3) attempts to deliver the packet through multiple paths 463 in the hope of eliminating packet loss due to equipment or media 464 failures. 466 It is these last two characteristics (guaranteed upper bound to 467 latency and elimination of packet loss) that an enhanced VPN adds to 468 a VPN service. 470 3.3. Integration 472 The only way to achieve the enhanced characteristics provided by an 473 enhanced VPN (such as guaranteed or predicted performance) is by 474 integrating the overlay VPN with a particular set of network 475 resources in the underlay network which are allocated to meet the 476 service requirement. This needs be done in a flexible and scalable 477 way so that it can be widely deployed in operator networks to 478 support a reasonable number of enhanced VPN customers. 480 Taking mobile networks and in particular 5G into consideration, the 481 integration of network and the service functions is a likely 482 requirement. The work in IETF SFC working group [SFC] provides a 483 foundation for this integration. 485 3.3.1. Abstraction 487 Integration of the overlay VPN and the underlay network resources 488 does not need to be a tight mapping. As described in [RFC7926], 489 abstraction is the process of applying policy to a set of 490 information about a TE network to produce selective information that 491 represents the potential ability to connect across the network. The 492 process of abstraction presents the connectivity graph in a way that 493 is independent of the underlying network technologies, capabilities, 494 and topology so that the graph can be used to plan and deliver 495 network services in a uniform way. 497 Virtual networks can be built on top of an abstracted topology that 498 represents the connectivity capabilities of the underlay network as 499 described in the framework for Abstraction and Control of TE Networks 500 (ACTN) [RFC8453] as discussed further in Section 5.5. 502 3.4. Dynamic Management 504 Enhanced VPNs need to be created, modified, and removed from the 505 network according to service demand. An enhanced VPN that requires 506 hard isolation (section 3.1) must not be disrupted by the 507 instantiation or modification of another enhanced VPN. Determining 508 whether modification of an enhanced VPN can be disruptive to that 509 VPN, and in particular whether the traffic in flight will be 510 disrupted can be a difficult problem. 512 The data plane aspects of this problem are discussed further in 513 Sections 5.1, 5.2, and 5.3. 515 The control plane aspects of this problem are discussed further in 516 Section 5.4. 518 The management plane aspects of this problem are discussed further 519 in Section 5.5. 521 Dynamic changes both to the VPN and to the underlay transport 522 network need to be managed to avoid disruption to services that are 523 sensitive to the change of network performance. 525 In addition to non-disruptively managing the network as a result of 526 gross change such as the inclusion of a new VPN endpoint or a change 527 to a link, VPN traffic might need to be moved as a result of traffic 528 volume changes. 530 3.5. Customized Control 532 In some cases it is desirable that an enhanced VPN has a customized 533 control plane, so that the tenant of the enhanced VPN can have some 534 control of how the resources and functions allocated to this 535 enhanced VPN are used. For example, the tenant may be able to 536 specify the service paths in his own enhanced VPN. Depending on the 537 requirement, an enhanced VPN may have its own dedicated controller, 538 which may be provided with an interface to the control system 539 provided by the network operator. Note that such control is within 540 the scope of the tenant's enhanced VPN, any change beyond that would 541 require some intervention of the operator. 543 A description of the control plane aspects of this problem are 544 discussed further in Section 5.4. A description of the management 545 plane aspects of this feature can be found in Section 5.5. 547 3.6. Applicability 549 The technologies described in this document should be applicable to 550 a number of types of VPN services such as: 552 o Layer 2 point-to-point services such as pseudowires [RFC3985] 554 o Layer 2 VPNs [RFC4664] 556 o Ethernet VPNs [RFC7209] 558 o Layer 3 VPNs [RFC4364], [RFC2764] 560 Where such VPN types need enhanced isolation and delivery 561 characteristics, the technologies described in section 5 can be used 562 to provide an underlay with the required enhanced performance. 564 3.7. Inter-Domain and Inter-Layer Network 566 In some scenarios, an enhanced VPN services may span multiple 567 network domains. A domain is considered to be any collection of 568 network elements within a common realm of address space or path 569 computation responsibility [RFC5151]. In some domains the operator 570 may manage a multi-layered network, for example, a packet network 571 over an optical network. When enhanced VPNs are provisioned in such 572 network scenarios, the technologies used in different network planes 573 (data plane, control plane, and management plane) need to provide 574 mechanisms to support multi-domain and multi-layer coordination and 575 integration, so as to provide the required service characteristics 576 for different enhanced VPNs, and improve network efficiency and 577 operational simplicity. 579 4. Architecture of Enhanced VPN 581 A number of enhanced VPN services will typically be provided by a 582 common network infrastructure. Each enhanced VPN consists of both 583 the overlay and a corresponding VTN with a specific set of network 584 resources and functions allocated in the underlay to satisfy the 585 needs of the VPN tenant. The integration between overlay and 586 various underlay resources ensures the required isolation between 587 different enhanced VPNs, and achieves the guaranteed performance for 588 different services. 590 An enhanced VPN needs to be designed with consideration given to: 592 o An enhanced data plane 593 o A control plane to create enhanced VPNs, making use of the 594 data plane isolation and performance guarantee techniques 596 o A management plane for enhanced VPN service life-cycle 597 management. 599 These required characteristics are expanded below: 601 o Enhanced data plane 603 * Provides the required resource isolation capability, e.g. 604 bandwidth guarantee. 606 * Provides the required packet latency and jitter 607 characteristics. 609 * Provides the required packet loss characteristics. 611 * Provides the mechanism to associate a packet with the set 612 of resources allocated to the enhanced VPN which the packet belongs. 614 o Control plane 616 * Collect information about the underlying network topology 617 and resources available and export this to nodes in the network 618 and/or the centralized controller as required. 620 * Create the required virtual transport networks (VTNs) with 621 the resource and properties needed by the enhanced VPN services that 622 are assigned to them. 624 * Determine the risk of SLA violation and take appropriate 625 avoiding action. 627 * Determine the right balance of per-packet and per-node 628 state according to the needs of enhanced VPN service to scale to the 629 required size. 631 o Management plane 633 * Provides an interface between the enhanced VPN provider 634 (e.g., the Transport Network Manager) and the enhanced VPN clients 635 (e.g., the 3GPP Management System) such that some of the operation 636 requests can be met without interfering with the enhanced VPN of 637 other clients. 639 * Provides an interface between the enhanced VPN provider and 640 the enhanced VPN clients to expose transport network capability 641 information toward the enhanced VPN client. 643 * Provides the service life-cycle management and operation of 644 enhanced VPN (e.g. creation, modification, assurance/monitoring and 645 decommissioning). 647 o Operations, Administration, and Maintenance (OAM) 649 * Provides the OAM tools to verify the connectivity and 650 performance of the enhanced VPN. 652 * Provide the OAM tools to verify whether the underlay 653 network resources are correctly allocated and operated properly. 655 o Telemetry 657 * Provides the mechanism to collect data plane, control plane, 658 and management plane information about the network. More 659 specifically: 661 + Provides the mechanism to collect network data from the 662 underlay network for overall performance evaluation and the enhanced 663 VPN service planning. 665 + Provides the mechanism to collect network data about 666 each enhanced VPN for monitoring and analytics of the 667 characteristics and SLA fulfilment of enhanced VPN services. 669 4.1. Layered Architecture 671 The layered architecture of an enhanced VPN is shown in Figure 2. 673 Underpinning everything is the physical network infrastructure layer 674 which provide the underlying resources used to provision the 675 separated virtual transport networks (VTNs). This includes the 676 partitioning of link and/or node resources. Each subset of link or 677 node resource can be considered as a virtual link or virtual node 678 used to build the VTNs. 680 A 681 | | 682 +-------------------+ Centralized 683 | Network Controller| Control& Management 684 +-------------------+ 685 || 686 \/ 688 __________________________ 689 / o----o----o / 690 / / / / VTN-1 691 / o-----o-----o----o----o / 692 /_________________________/ 693 __________________________ 694 / o----o / 695 / / / \ / VTN-2 696 / o-----o----o----o-----o / 697 /_________________________/ 698 ...... ... 699 ___________________________ 700 / o----o / 701 / / / / VTN-3 702 / o-----o----o----o-----o / 703 /__________________________/ 705 ++++ ++++ ++++ 706 +--+===+--+===+--+ 707 +--+===+--+===+--+ 708 ++++ +++\\ ++++ Physical 709 || || \\ || 710 || || \\ || Network 711 ++++ ++++ ++++ \\+++ ++++ 712 +--+===+--+===+--+===+--+===+--+ Infrastructure 713 +--+===+--+===+--+===+--+===+--+ 714 ++++ ++++ ++++ ++++ ++++ 716 O Virtual Node 718 -- Virtual Link 720 ++++ 721 +--+ Physical Node with resource partition 722 +--+ 723 ++++ 724 == Physical Link with resource partition 726 Figure 2 The Layered Architecture 728 Various components and techniques discussed in Section 5 can be used 729 to enable resource partition, such as FlexE, Time Sensitive 730 Networking, Deterministic Networking, Dedicated queues, etc. These 731 partitions may be physical, or virtual so long as the SLA required 732 by the higher layers is met. 734 Based on the network resources provided by the physical network 735 infrastructure, multiple VTNs can be provisioned, each with 736 customized topology and other attributes to meet the requirement of 737 different enhanced VPNs or different groups of enhanced VPNs. To get 738 the required characteristic, each VTN needs to be mapped to a set of 739 network nodes and links in the network infrastructure. And on each 740 node or link, the VTN is associated with a set of resources which 741 are allocated for the processing of traffic in the VTN. VTN provides 742 the integration between the virtual network topology and the 743 required underlying network resources. 745 The centralized controller is used to create the VTN, and to 746 instruct the network nodes to allocate the required resources to 747 each VTN and to provision the enhanced VPN services on the VTNs. A 748 distributed control plane may also be used for the distribution of 749 the VTN topology and attribute information between nodes within the 750 VTNs. 752 The process used to create VTNs and to allocate network resources 753 for use by VTNs needs to take a holistic view of the needs of all of 754 its tenants (i.e., of all customers and their associated VTNs), and 755 to partition the resources accordingly. However, within a VTN these 756 resources can, if required, be managed via a dynamic control plane. 757 This provides the required scalability and isolation. 759 4.2. Multi-Point to Multi-Point (MP2MP) Connectivity 761 At the VPN service level, the required connectivity is usually mesh 762 or partial-mesh. To support such kinds of VPN service, the 763 corresponding VTN in underlay is also an abstract MP2MP medium. 764 Other service requirements may be expressed at different granularity, 765 some of which can be applicable to the whole service, while some 766 others may be only applicable to some pairs of end points. For 767 example, when particular level of performance guarantee is 768 required, the point-to-point path through the underlay of the 769 enhanced VPN may need to be specifically engineered to meet the 770 required performance guarantee. 772 4.3. Application Specific Network Types 774 Although a lot of the traffic that will be carried over the enhanced 775 VPN will likely be IPv4 or IPv6, the design has to be capable of 776 carrying other traffic types, in particular Ethernet traffic. This 777 is easily accomplished through the various pseudowire (PW) 778 techniques [RFC3985]. Where the underlay is MPLS, Ethernet can be 779 carried over the enhanced VPN encapsulated according to the method 780 specified in [RFC4448]. Where the underlay is IP, Layer Two 781 Tunneling Protocol - Version 3 (L2TPv3) [RFC3931] can be used with 782 Ethernet traffic carried according to [RFC4719]. Encapsulations 783 have been defined for most of the common Layer 2 types for both PW 784 over MPLS and for L2TPv3. 786 4.4. Scaling Considerations 788 VPNs are instantiated as overlays on top of an operator's network 789 and offered as services to the operator's customers. An important 790 feature of overlays is that they are able to deliver services 791 without placing per-service state in the core of the underlay 792 network. 794 Enhanced VPNs may need to install some additional state within the 795 network to achieve the additional features that they require. 796 Solutions must consider minimizing and controlling the scale of such 797 state, and deployment architectures should constrain the number of 798 enhanced VPNs that would exist where such services would place 799 additional state in the network. It is expected that the number of 800 enhanced VPN would be small in the beginning, and even in future the 801 number of enhanced VPN will be much fewer than traditional VPNs, 802 because pre-existing VPN techniques are be good enough to meet the 803 needs of most existing VPN-type services. 805 In general, it is not required that the state in the network be 806 maintained in a 1:1 relationship with the VPN+ services. It will 807 usually be possible to aggregate a set of VPN+ services so that they 808 share the same VTN and the same set of network resources (much in 809 the way that current VPNs are aggregated over transport tunnels) so 810 that collections of enhanced VPNs that require the same behaviour 811 from the network in terms of resource reservation, latency bounds, 812 resiliency, etc. are able to be grouped together. This is an 813 important feature to assist with the scaling characteristics of VPN+ 814 deployments. 816 See Section 6 for a further discussion of scalability considerations. 818 5. Candidate Technologies 820 A VPN is a network created by applying a demultiplexing technique to 821 the underlying network (the underlay) in order to distinguish the 822 traffic of one VPN from that of another. A VPN path that travels by 823 other than the shortest path through the underlay normally requires 824 state in the underlay to specify that path. State is normally 825 applied to the underlay through the use of the RSVP signaling 826 protocol, or directly through the use of an SDN controller, although 827 other techniques may emerge as this problem is studied. This state 828 gets harder to manage as the number of VPN paths increases. 829 Furthermore, as we increase the coupling between the underlay and 830 the overlay to support the enhanced VPN service, this state will 831 increase further. 833 In an enhanced VPN different subsets of the underlay resources can 834 be dedicated to different enhanced VPNs or different groups of 835 enhanced VPNs. An enhanced VPN solution thus needs tighter coupling 836 with underlay than is the case with existing VPNs. We cannot, for 837 example, share the network resource between enhanced VPNs which 838 require hard isolation. 840 5.1. Layer-Two Data Plane 842 A number of candidate Layer 2 packet or frame-based data plane 843 solutions which can be used provide the required isolation and 844 guarantees are described in following sections. 846 5.1.1. Flexible Ethernet 848 FlexE [FLEXE] provides the ability to multiplex channels over an 849 Ethernet link to create point-to-point fixed-bandwidth connections 850 in a way that provides hard isolation. FlexE also supports bonding 851 links to create larger links out of multiple low capacity links. 853 However, FlexE is only a link level technology. When packets are 854 received by the downstream node, they need to be processed in a way 855 that preserves that isolation in the downstream node. This in turn 856 requires a queuing and forwarding implementation that preserves the 857 end-to-end isolation. 859 If different FlexE channels are used for different services, then no 860 sharing is possible between the FlexE channels. This means that it 861 may be difficult to dynamically redistribute unused bandwidth to 862 lower priority services in another FlexE channel. If one FlexE 863 channel is used by one tenant, the tenant can use some methods to 864 manage the relative priority of his own traffic in the FlexE channel. 866 5.1.2. Dedicated Queues 868 DiffServ based queuing systems are described in [RFC2475] and 869 [RFC4594]. This is considered insufficient to provide isolation for 870 enhanced VPNs because DiffServ does not always provide enough 871 markers to differentiate between traffic of many enhanced VPNs, or 872 offer the range of service classes that each VPN needs to provide to 873 its tenants. This problem is particularly acute with an MPLS 874 underlay, because MPLS only provides eight Traffic Classes. 876 In addition, DiffServ, as currently implemented, mainly provides 877 per-hop priority-based scheduling, and it is difficult to use it to 878 achieve quantitive resource reservation. 880 In order to address these problems and to reduce the potential 881 interference between enhanced VPNs, it would be necessary to steer 882 traffic to dedicated input and output queues per enhanced VPN: some 883 routers have a large number of queues and sophisticated queuing 884 systems, which could support this, while some routers may struggle 885 to provide the granularity and level of isolation required by the 886 applications of enhanced VPN. 888 5.1.3. Time Sensitive Networking 890 Time Sensitive Networking (TSN) [TSN] is an IEEE project that is 891 designing a method of carrying time sensitive information over 892 Ethernet. It introduces the concept of packet scheduling where a 893 packet stream may be given a time slot guaranteeing that it 894 experiences no queuing delay or increase in latency. The mechanisms 895 defined in TSN can be used to meet the requirements of time 896 sensitive services of an enhanced VPN. 898 Ethernet can be emulated over a Layer 3 network using an IP or MPLS 899 pseudowire. However, a TSN Ethernet payload would be opaque to the 900 underlay and thus not treated specifically as time sensitive data. 901 The preferred method of carrying TSN over a Layer 3 network is 902 through the use of deterministic networking as explained in Section 903 5.2.1. 905 5.2. Layer-Three Data Plane 907 We now consider the problem of slice differentiation and resource 908 representation in the network layer. 910 5.2.1. Deterministic Networking 912 Deterministic Networking (DetNet) [RFC8655] is a technique being 913 developed in the IETF to enhance the ability of Layer 3 networks to 914 deliver packets more reliably and with greater control over the 915 delay. The design cannot use re-transmission techniques such as TCP 916 since that can exceed the delay tolerated by the applications. Even 917 the delay improvements that are achieved with Stream Control 918 Transmission Protocol Partial Reliability Extension (SCTP-PR) 919 [RFC3758] do not meet the bounds set by application demands. DetNet 920 pre-emptively sends copies of the packet over various paths to 921 minimize the chance of all copies of a packet being lost. It also 922 seeks to set an upper bound on latency, but the goal is not to 923 minimize latency. 925 5.2.2. MPLS Traffic Engineering (MPLS-TE) 927 MPLS-TE [RFC2702] [RFC3209] introduces the concept of reserving end- 928 to-end bandwidth for a TE-LSP, which can be used to provide point- 929 to-point Virtual Transport Path (VTP) across the underlay network to 930 support VPNs. VPN traffic can be carried over dedicated TE-LSPs to 931 provide reserved bandwidth for each specific connection in a VPN, 932 and VPNs with similar behaviour requirements may be multiplexed onto 933 the same TE-LSPs. Some network operators have concerns about the 934 scalability and management overhead of MPLS-TE system, and this has 935 lead them to consider other solutions for their networks. 937 5.2.3. Segment Routing 939 Segment Routing (SR) [RFC8402] is a method that prepends 940 instructions to packets at the head-end of a path. These 941 instructions are used to specify the nodes and links to be traversed 942 and allow the packets to be routed on paths other than the shortest 943 path. By encoding the state in the packet, per-path state is 944 transitioned out of the network. 946 An SR traffic engineered path operates with a granularity of a link 947 with hints about priority provided through the use of the traffic 948 class (TC) or Differentiated Services Code Point (DSCP) field in the 949 header. However to achieve the latency and isolation 950 characteristics that are sought by the enhanced VPN users, steering 951 packets through specific queues and resources will likely be 952 required. With SR, it is possible to introduce such fine-grained 953 packet steering by specifying the queues and resources through an SR 954 instruction list. 956 Note that the concept of queue is a useful abstraction for different 957 types of underlay mechanism that may be used to provide enhanced 958 isolation and latency support. How the queue satisfies the 959 requirement is implementation specific and is transparent to the 960 layer-3 data plane and control plane mechanisms used. 962 With Segment Routing, the SR instruction list could be used to build 963 a P2P path, a group of SR SIDs could also be used to represent a MP2MP 964 network. Thus the SR based mechanism could be used to provide both 965 Virtual Transport Path (VTP) and Virtual Transport Network (VTN) for 966 enhanced VPN services. 968 5.3. Non-Packet Data Plane 970 Non-packet underlay data plane technologies often have TE properties 971 and behaviours, and meet many of the key requirements in particular 972 for bandwidth guarantees, traffic isolation (with physical isolation 973 often being an integral part of the technology), highly predictable 974 latency and jitter characteristics, measurable loss characteristics, 975 and ease of identification of flows. The cost is the resources are 976 allocated on a long term and end-to-end basis. Such an arrangement 977 means that the full cost of the resources has be borne by the 978 service that is allocated with the resources. 980 5.4. Control Plane 982 Enhanced VPN would likely be based on a hybrid control mechanism, 983 which takes advantage of the logically centralized controller for 984 on-demand provisioning and global optimization, whilst still relying 985 on a distributed control plane to provide scalability, high 986 reliability, fast reaction, automatic failure recovery, etc. 987 Extension to and optimization of the distributed control plane is 988 needed to support the enhanced properties of VPN+. 990 RSVP-TE [RFC3209] provides the signaling mechanism for establishing 991 a TE-LSP in an MPLS network with end-to-end resource reservation. 992 This can be seen as a Virtual Transport Path (VTP), which could be 993 used to bind the VPN to specific network resources allocated within 994 the underlay, but there remain scalability concerns mentioned in 995 Section 5.2.2. 997 The control plane of SR [RFC8665] [RFC8667] [I-D.ietf-idr-bgp-ls- 998 segment-routing-ext] does not have the capability of signaling 999 resource reservations along the path. On the other hand, the SR 1000 approach provides a potential way of binding the underlay network 1001 resource and the enhanced VPN service without requiring per-path 1002 state to be maintained in the network. A centralized controller can 1003 perform resource planning and reservation for enhanced VPNs, while 1004 it needs to ensure that resources are correctly allocated in network 1005 nodes for the enhanced VPN service. 1007 5.5. Management Plane 1009 The management plane provides the interface between the enhanced VPN 1010 provider and the clients for the service life-cycle management (e.g. 1011 creation, modification, assurance/monitoring and decommissioning). 1012 It relies on a set of service data models for the description of the 1013 information and operations needed on the interface. 1015 In the context of 5G end-to-end network slicing [TS28530], the 1016 management of enhanced VPNs is considered as the management of the 1017 transport network part of the end-to-end network slice. 3GPP 1018 management system may provide the connectivity and performance 1019 related parameters as requirements to the management plane of the 1020 transport network. It may also require the transport network to 1021 expose the capability and status of the transport network slice. 1022 Thus, an interface between the enhanced VPN management plane and the 1023 3GPP network slice management system, and relevant service data 1024 models are needed for the coordination of end-to-end network slice 1025 management. 1027 The management plane interface and data models for enhanced VPN can 1028 be based on the service models described in Section 5.6. 1030 5.6. Applicability of Service Data Models to Enhanced VPN 1032 ACTN supports operators in viewing and controlling different domains 1033 and presenting virtualized networks to their customers. The ACTN 1034 framework [RFC8453] highlights how: 1036 o Abstraction of the underlying network resources is provided to 1037 higher-layer applications and customers. 1039 o Underlying resources are virtualized allocating those resources 1040 for the customer, application, or service. 1042 o A virtualized environment is created allowing operators to view 1043 and control multi-domain networks as a single virtualized network. 1045 o Networks can be presented to customers as a virtual network via 1046 open and programmable interfaces. 1048 The type of network virtualization enabled by ACTN managed 1049 infrastructure provides customers and applications (tenants) with 1050 the capability to utilize and independently control allocated 1051 virtual network resources as if they were physically their own 1052 resources. Service Data models are used to represent, monitor, and 1053 manage the virtual networks and services enabled by ACTN. The 1054 Customer VPN model (e.g. L3SM [RFC8299]) or an ACTN Virtual Network 1055 (VN) [I-D.ietf-teas-actn-vn-yang] model is a customer view of the 1056 ACTN managed infrastructure, and is presented by the ACTN provider 1057 as a set of abstracted services or resources. The L3VPN network 1058 model [I-D.ietf-opsawg-l3sm-l3nm] and the TE tunnel model [I-D.ietf- 1059 teas-yang-te] provide a network view of the ACTN managed 1060 infrastructure presented by the ACTN provider as a set of transport 1061 resources. 1063 5.6.1. Enhanced VPN Delivery in the ACTN Architecture 1065 ACTN provides VPN connections between multiple sites as requested 1066 via the Customer Network Controller (CNC). The CNC is managed by 1067 the customer themselves, and interacts with the network provider's 1068 Multi-Domain Service Controller (MDSC). The Provisioning Network 1069 Controllers (PNC) are responsible for network resource management, 1070 thus the PNCs are remain entirely under the management of the 1071 network provider and are not visible to the customer so that 1072 management is mostly performed by the network provider, with some 1073 flexibility delegated to the customer-managed CNC. 1075 Figure 3 presents a more general representation of how multiple 1076 enhanced VPNs may be created from the resources of multiple physical 1077 networks using the CNC, MDSC, and PNC components of the ACTN 1078 architecture. Each enhanced VPN is controlled by its own CNC. The 1079 CNCs send requests to the provider's MDSC. The provider manages two 1080 different physical networks each under the control of PNC. The MDSC 1081 asks the PNCs to allocate and provision resources to achieve the 1082 enhanced VPNs. In this figure, one enhanced VPN is constructed 1083 solely from the resources of one of the physical networks, while the 1084 the VPN uses resources from both physical networks. 1086 --------------- ( ) 1087 | CNC |---------->( VPN+ ) 1088 --------^------ ( ) 1089 | _(_________ _) 1090 --------------- ( ) ^ 1091 | CNC |----------->( VPN+ ) : 1092 ------^-------- ( ) : 1093 | | (___________) : 1094 | | ^ ^ : 1095 Boundary | | : : : 1096 Between ==========|====|===================:====:====:======== 1097 Customer & | | : : : 1098 Network Provider | | : : : 1099 v v : : : 1100 --------------- : :....: 1101 | MDSC | : : 1102 --------------- : : 1103 ^ ---^------ ... 1104 | ( ) . 1105 v ( Physical ) . 1106 ---------------- ( Network ) . 1107 | PNC |<-------->( ) ---^------ 1108 ---------------- | -------- ( ) 1109 | |-- ( Physical ) 1110 | PNC |<------------------------->( Network ) 1111 --------------- ( ) 1112 -------- 1114 Figure 3 Generic VPN+ Delivery in the ACTN Architecture 1116 5.6.2. Enhanced VPN Features with Service Data Models 1118 This section discusses how the service data models can fulfil the 1119 enhanced VPN requirements described earlier in this document within 1120 the scope of the ACTN architecture. 1122 5.6.2.1. Isolation Between VPNs 1124 The VN YANG model [I-D.ietf-teas-actn-vn-yang] and the TE-service 1125 mapping model [I-D.ietf-teas-te-service-mapping-yang] fulfil the VPN 1126 isolation requirement by providing the following features for the 1127 VPNs: 1129 o Each VPN is identified with a unique identifier (vpn-id) and 1130 can be mapped to a specific VN. Multiple VPNs may mapped to the 1131 same VN according to service requirements and operator's policy. 1133 o Each VPN is managed and controlled independent of other VPNs. 1135 o Each VPN is instantiated with an isolation requirement 1136 described by the TE-service mapping model [I-D.ietf-teas-te-service- 1137 mapping-yang]. This mapping supports all levels of isolation (hard 1138 isolation with deterministic characteristics, hard isolation, soft 1139 isolation, or no isolation). 1141 5.6.2.2. Guaranteed Performance 1143 Performance objectives of a VPN [RFC8299][RFC8466] are expressed 1144 through a QoS profile including the following properties: 1146 o Rate-limit 1148 o Bandwidth 1150 o Latency 1152 o Jitter 1154 [I-D.ietf-teas-actn-vn-yang] and [I-D.ietf-teas-yang-te-topo] allow 1155 configuration of several TE parameters that may help to meet the VPN 1156 performance objectives as follows: 1158 o Bandwidth 1160 o Objective function (e.g., min cost path, min load path, etc.) 1162 o Metric Types and their threshold: 1164 * TE cost, IGP cost, Hop count, or Unidirectional Delay (e.g., 1165 can set all path delay <= threshold) 1167 Once these requests are instantiated, the resources are committed 1168 and guaranteed through the life cycle of the VPN. 1170 5.6.2.3. Integration 1172 The L3VPN network model provides mechanism to correlate customer's 1173 VPN and the VPN service related resources (e.g., RT and RD) 1174 allocated in the provider's network. 1176 The VPN/Network performance monitoring model [I-D.www-bess-yang-vpn- 1177 service-pm] provides mechanisms to monitor and manage network 1178 Performance on the topology at different layer or the overlay 1179 topology between VPN sites. 1181 These two models provide mechanisms to correlate the customer's VPN 1182 and the actual TE tunnels instantiated in the provider's network. 1184 Service function integration with network topology (L3 and TE 1185 topology) is in progress in [I-D.ietf-teas-sf-aware-topo-model] 1186 which addresses a number of use-cases that show how TE topology 1187 supports various service functions. 1189 5.6.2.4. Dynamic and Customized Management 1191 The ACTN architecture allows the CNC to interact with the provider's 1192 MDSC. This gives the customer dynamic control of their VPNs. 1194 For example, the ACTN VN model [I-D.ietf-teas-actn-vn-yang] allows 1195 life-cycle management to create, modify, and delete VNs on demand. 1196 Customers may also be allowed more customized control of the VN 1197 topology by provisioning tunnels to connect their endpoints, and 1198 even configuring the paths of those tunnels. 1200 Another example is the L3VPN service model [RFC8299] which allows 1201 VPN lifecycle management such as VPN creation, modification, and 1202 deletion on demand. 1204 5.6.3. 5G Transport Service Delivery via Coordinated Data Modules 1206 The overview of network slice structure as defined in the 3GPP 5GS 1207 is shown in Figure 4. The terms are described in specific 3GPP 1208 documents [TS23501] [TS28530]. 1210 <================== E2E-NSI =======================> 1211 : : : : : 1212 : : : : : 1213 <====== RAN-NSSI ======><=TN-NSSI=><====== CN-NSSI ======>VL[APL] 1214 : : : : : : : : : 1215 : : : : : : : : : 1216 RW[NFs ]<=TRN-NSSI=>[NFs ]<=TN-NSSI=>[NFs ]<=TRN-NSSI=>[NFs ]VL[APL] 1217 . . . . . . . . . . . . .. . . . . . . . . . . . . .. 1218 .,----. ,----. ,----.. ,----. .,----. ,----. ,----.. 1219 UE--|RAN |---| TN |---|RAN |---| TN |--|CN |--| TN |--|CN |--[APL] 1220 .|NFs | `----' |NFs |. `----' .|NFs | `----' |NFs |. 1221 .`----' `----'. .`----' `----'. 1222 . . . . . . . . . . . . .. . . . . . . . . . . . .. 1224 RW RAN MBH CN DN 1226 *Legends 1227 UE: User Equipment 1228 RAN: Radio Access Network 1229 CN: Core Network 1230 DN: Data Network 1231 TN: Transport Network 1232 MBH: Mobile Backhaul 1233 RW: Radio Wave 1234 NF: Network Function 1235 APL: Application Server 1236 NSI: Network Slice Instance 1237 NSSI: Network Slice Subnet Instance 1239 Figure 4 Overview of Structure of Network Slice in 3GPP 5GS 1241 The L3VPN service model [RFC8299] and TEAS VN model [I-D.ietf-teas- 1242 actn-vn-yang] can both be used to describe the 5G MBB Transport 1243 Service or connectivity service. The L3VPN service model is used to 1244 describe end-to-end IP connectivity service, while the TEAS VN model 1245 is used to describe TE connectivity service between VPN sites or 1246 between RAN NFs and Core network NFs. 1248 A VN in the TEAS VN model with its support of point-to-point or 1249 multipoint-to-multipoint connectivity services can be seen as one 1250 example of a network slice. 1252 The TE Service mapping model can be used to map L3VPN service 1253 requests onto underlying network resource and TE models to get the 1254 TE network provisioned. 1256 For IP VPN service provisioning, the service parameters in the L3VPN 1257 service model [RFC8299] can be decomposed into a set of 1258 configuration parameters described in the L3VPN network model [I- 1259 D.ietf-opsawg-l3sm-l3nm] which will get the VPN network provisioned. 1261 6. Scalability Considerations 1263 Enhanced VPN provides performance guaranteed services in packet 1264 networks, but with the potential cost of introducing additional 1265 states into the network. There are at least three ways that this 1266 additional state might be presented in the network: 1268 o Introduce the complete state into the packet, as is done in SR. 1269 This allows the controller to specify a detailed series of 1270 forwarding and processing instructions for the packet as it transits 1271 the network. The cost of this is an increase in the packet header 1272 size. The cost is also that systems will have capabilities enabled 1273 in case they are called upon by a service. This is a type of latent 1274 state, and increases as we more precisely specify the path and 1275 resources that need to be exclusively available to a VPN. 1277 o Introduce the state to the network. This is normally done by 1278 creating a path using RSVP-TE, which can be extended to introduce 1279 any element that needs to be specified along the path, for example 1280 explicitly specifying queuing policy. It is possible to use other 1281 methods to introduce path state, such as via a Software Defined 1282 Network (SDN) controller, or possibly by modifying a routing 1283 protocol. With this approach there is state per path, per path 1284 characteristic that needs to be maintained over its life-cycle. 1285 This is more state than is needed using SR, but the packets are 1286 shorter. 1288 o Provide a hybrid approach. One example is based on using 1289 binding SIDs [RFC8402] to create path fragments, and bind them 1290 together with SR. Dynamic creation of a VPN service path using SR 1291 requires less state maintenance in the network core at the expense 1292 of larger packet headers. The packet size can be lower if a form of 1293 loose source routing is used (using a few nodal SIDs), and it will 1294 be lower if no specific functions or resources on the routers are 1295 specified. 1297 Reducing the state in the network is important to enhanced VPN, as 1298 it requires the overlay to be more closely integrated with the 1299 underlay than with traditional VPNs. This tighter coupling would 1300 normally mean that more state needed to be created and maintained in 1301 the network, as the state about fine granularity processing would 1302 need to be loaded and maintained in the routers. However, a segment 1303 routed approach allows much of this state to be spread amongst the 1304 network ingress nodes, and transiently carried in the packets as 1305 SIDs. 1307 6.1. Maximum Stack Depth of SR 1309 One of the challenges with SR is the stack depth that nodes are able 1310 to impose on packets [RFC8491]. This leads to a difficult balance 1311 between adding state to the network and minimizing stack depth, or 1312 minimizing state and increasing the stack depth. 1314 6.2. RSVP Scalability 1316 The traditional method of creating a resource allocated path through 1317 an MPLS network is to use the RSVP protocol. However there have 1318 been concerns that this requires significant continuous state 1319 maintenance in the network. Work to improve the scalability of 1320 RSVP-TE LSPs in the control plane can be found in [RFC8370]. 1322 There is also concern at the scalability of the forwarder footprint 1323 of RSVP as the number of paths through an LSR grows. [RFC8577] 1324 proposes to address this by employing SR within a tunnel established 1325 by RSVP-TE. 1327 6.3. SDN Scaling 1329 The centralized approach of SDN requires state to be stored in the 1330 network, but does not have the overhead of also requiring control 1331 plane state to be maintained. Each individual network node may need 1332 to maintain a communication channel with the SDN controller, but 1333 that compares favourably with the need for a control plane to 1334 maintain communication with all neighbors. 1336 However, SDN may transfer some of the scalability concerns from the 1337 network to the centralized controller. In particular, there may be 1338 a heavy processing burden at the controller, and a heavy load in the 1339 network surrounding the controller. 1341 7. OAM Considerations 1343 The enhanced VPN OAM design needs to consider the following 1344 requirements: 1346 o Instrumentation of the underlay so that the network operator can 1347 be sure that the resources committed to a tenant are operating 1348 correctly and delivering the required performance. 1350 o Instrumentation of the overlay by the tenant. This is likely to 1351 be transparent to the network operator and to use existing methods. 1352 Particular consideration needs to be given to the need to verify the 1353 isolation and the various committed performance characteristics. 1355 o Instrumentation of the overlay by the network provider to 1356 proactively demonstrate that the committed performance is being 1357 delivered. This needs to be done in a non-intrusive manner, 1358 particularly when the tenant is deploying a performance sensitive 1359 application. 1361 o Verification of the conformity of the path to the service 1362 requirement. This may need to be done as part of a commissioning 1363 test. 1365 A study of OAM in SR networks has been documented in [RFC8403]. 1367 8. Telemetry Considerations 1369 Network visibility is essential for network operation. Network 1370 telemetry has been considered as an ideal means to gain sufficient 1371 network visibility with better flexibility, scalability, accuracy, 1372 coverage, and performance than conventional OAM technologies. 1374 As defined in [I-D.ietf-opsawg-ntf], the purpose of Network 1375 Telemetry is to acquire network data remotely for network monitoring 1376 and operation. It is a general term for a large set of network 1377 visibility techniques and protocols. Network telemetry addresses 1378 the current network operation issues and enables smooth evolution 1379 toward intent-driven autonomous networks. Telemetry can be applied 1380 on the forwarding plane, the control plane, and the management plane 1381 in a network. 1383 How the telemetry mechanisms could be used or extended for the 1384 enhanced VPN service will be described in a separate document. 1386 9. Enhanced Resiliency 1388 Each enhanced VPN has a life-cycle, and may need modification during 1389 deployment as the needs of its tenant change. Additionally, as the 1390 network as a whole evolves, there may need to be garbage collection 1391 performed to consolidate resources into usable quanta. 1393 Systems in which the path is imposed such as SR, or some form of 1394 explicit routing tend to do well in these applications, because it 1395 is possible to perform an atomic transition from one path to another. 1396 This is a single action by the head-end changes the path without the 1397 need for coordinated action by the routers along the path. However, 1398 implementations and the monitoring protocols need to make sure that 1399 the new path is up and meets the required SLA before traffic is 1400 transitioned to it. It is possible for deadlocks to arise as a 1401 result of the network becoming fragmented over time, such that it is 1402 impossible to create a new path or to modify an existing path 1403 without impacting the SLA of other paths. Resolution of this 1404 situation is as much a commercial issue as it is a technical issue 1405 and is outside the scope of this document. 1407 There are, however, two manifestations of the latency problem that 1408 are for further study in any of these approaches: 1410 o The problem of packets overtaking one and other if a path latency 1411 reduces during a transition. 1413 o The problem of transient variation in latency in either direction 1414 as a path migrates. 1416 There is also the matter of what happens during failure in the 1417 underlay infrastructure. Fast reroute is one approach, but that 1418 still produces a transient loss with a normal goal of rectifying 1419 this within 50ms [RFC5654]. An alternative is some form of N+1 1420 delivery such as has been used for many years to support protection 1421 from service disruption. This may be taken to a different level 1422 using the techniques proposed by the IETF deterministic network work 1423 with multiple in-network replication and the culling of later 1424 packets [RFC8655]. 1426 In addition to the approach used to protect high priority packets, 1427 consideration has to be given to the impact of best effort traffic 1428 on the high priority packets during a transient. Specifically if a 1429 conventional re-convergence process is used there will inevitably be 1430 micro-loops and whilst some form of explicit routing will protect 1431 the high priority traffic, lower priority traffic on best effort 1432 shortest paths will micro-loop without the use of a loop prevention 1433 technology. To provide the highest quality of service to high 1434 priority traffic, either this traffic must be shielded from the 1435 micro-loops, or micro-loops must be prevented. 1437 10. Operational Considerations 1439 It is likely that enhanced VPN service will be introduced in 1440 networks which already have traditional VPN services deployed. 1441 Depends on service requirement, the tenants or the operator may 1442 choose to use traditional VPN or enhanced VPN to fulfil the service 1443 requirement. The information and parameters to assist such decision 1444 needs to be reflected on the management interface between the 1445 tenants and the operator. 1447 11. Security Considerations 1449 All types of virtual network require special consideration to be 1450 given to the isolation of traffic belonging to different tenants. 1451 That is, traffic belonging to one VPN must not be delivered to end 1452 points outside that VPN. In this regard enhanced VPNs neither 1453 introduce, no experience a greater security risks than other VPNs. 1455 However, in an enhanced Virtual Private Network service the 1456 additional service requirements need to be considered. For example, 1457 if a service requires a specific upper bound to latency then it can 1458 be damaged by simply delaying the packets through the activities of 1459 another tenant, i.e., by introducing bursts of traffic for other 1460 services. 1462 The measures to address these dynamic security risks must be 1463 specified as part to the specific solution are form part of the 1464 isolation requirements of a service. 1466 While an enhanced VPN service may be sold as offering encryption and 1467 other security features as part of the service, customers would be 1468 well advised to take responsibility for their own security 1469 requirements themselves possibly by encrypting traffic before 1470 handing it off to the service provider. 1472 The privacy of enhanced VPN service customers must be preserved. It 1473 should not be possible for one customer to discover the existence of 1474 another customer, nor should the sites that are members of an 1475 enhanced VPN be externally visible. 1477 12. IANA Considerations 1479 There are no requested IANA actions. 1481 13. Contributors 1483 Daniel King 1484 Email: daniel@olddog.co.uk 1486 Adrian Farrel 1487 Email: adrian@olddog.co.uk 1489 Jeff Tansura 1490 Email: jefftant.ietf@gmail.com 1492 Qin Wu 1493 Email: bill.wu@huawei.com 1495 Daniele Ceccarelli 1496 Email: daniele.ceccarelli@ericsson.com 1498 Mohamed Boucadair 1499 Email: mohamed.boucadair@orange.com 1501 Sergio Belotti 1502 Email: sergio.belotti@nokia.com 1504 Haomian Zheng 1505 Email: zhenghaomian@huawei.com 1507 Zhenbin Li 1508 Email: lizhenbin@huawei.com 1510 14. Acknowledgments 1512 The authors would like to thank Charlie Perkins, James N Guichard, 1513 John E Drake and Shunsuke Homma for their review and valuable 1514 comments. 1516 This work was supported in part by the European Commission funded 1517 H2020-ICT-2016-2 METRO-HAUL project (G.A. 761727). 1519 15. References 1521 15.1. Normative References 1523 [I-D.ietf-teas-actn-vn-yang] Lee, Y., Dhody, D., Ceccarelli, D., 1524 Bryskin, I., and B. Yoon, "A Yang Data Model for VN 1525 Operation", draft-ietf-teas-actn-vn-yang-07 (work in 1526 progress), October 2019. 1528 [I-D.ietf-teas-te-service-mapping-yang] Lee, Y., Dhody, D., Fioccola, 1529 G., Wu, Q., Ceccarelli, D., and J. Tantsura, "Traffic 1530 Engineering (TE) and Service Mapping Yang Model", draft- 1531 ietf-teas-te-service-mapping-yang-02 (work in progress), 1532 September 2019. 1534 [RFC2764] Gleeson, B., Lin, A., Heinanen, J., Armitage, G., and A. 1535 Malis, "A Framework for IP Based Virtual Private Networks", 1536 RFC 2764, DOI 10.17487/RFC2764, February 2000, 1537 . 1539 [RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V., 1540 and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP 1541 Tunnels", RFC 3209, DOI 10.17487/RFC3209, December 2001, 1542 . 1544 [RFC3985] Bryant, S., Ed. and P. Pate, Ed., "Pseudo Wire Emulation 1545 Edge-to-Edge (PWE3) Architecture", RFC 3985, DOI 1546 10.17487/RFC3985, March 2005, . 1549 [RFC4664] Andersson, L., Ed. and E. Rosen, Ed., "Framework for Layer 1550 2 Virtual Private Networks (L2VPNs)", RFC 4664, DOI 1551 10.17487/RFC4664, September 2006, 1554 [RFC8299] Wu, Q., Ed., Litkowski, S., Tomotaki, L., and K. Ogaki, 1555 "YANG Data Model for L3VPN Service Delivery", RFC 8299, 1556 DOI 10.17487/RFC8299, January 2018, . 1559 [RFC8402] Filsfils, C., Ed., Previdi, S., Ed., Ginsberg, L., 1560 Decraene, B., Litkowski, S., and R. Shakir, "Segment 1561 Routing Architecture", RFC 8402, DOI 10.17487/RFC8402, 1562 July 2018, . 1564 [RFC8453] Ceccarelli, D., Ed. and Y. Lee, Ed., "Framework for 1565 Abstraction and Control of TE Networks (ACTN)", RFC 8453, 1566 DOI 10.17487/RFC8453, August 2018, . 1569 [RFC8466] Wen, B., Fioccola, G., Ed., Xie, C., and L. Jalil, "A YANG 1570 Data Model for Layer 2 Virtual Private Network (L2VPN) 1571 Service Delivery", RFC 8466, DOI 10.17487/RFC8466, October 1572 2018, . 1574 15.2. Informative References 1576 [BBF-SD406] "BBF SD-406: End-to-End Network Slicing", 2016, 1577 . 1580 [DETNET] "Deterministic Networking", March , 1581 . 1583 [FLEXE] "Flex Ethernet Implementation Agreement", March 2016, 1584 . 1587 [I-D.ietf-idr-bgp-ls-segment-routing-ext] Previdi, S., Talaulikar, 1588 K., Filsfils, C., Gredler, H., and M. Chen, "BGP Link- 1589 State extensions for Segment Routing", draft-ietf-idr-bgp- 1590 ls-segment-routing-ext-16 (work in progress), June 2019. 1592 [I-D.ietf-opsawg-l3sm-l3nm] Aguado, A., Dios, O., Lopezalvarez, V., 1593 daniel.voyer@bell.ca, d., and L. Munoz, "Layer 3 VPN 1594 Network Model", draft-ietf-opsawg-l3sm-l3nm-01, (work in 1595 progress), November 2019. 1597 [I-D.ietf-opsawg-ntf] Song, H., Qin, F., Martinez-Julia, P., 1598 Ciavaglia, L., and A. Wang, "Network Telemetry Framework", 1599 draft-ietf-opsawg-ntf-02 (work in progress), October 2019. 1601 [I-D.ietf-teas-sf-aware-topo-model] Bryskin, I., Liu, X., Lee, Y., 1602 Guichard, J., Contreras, L., Ceccarelli, D., and J. 1603 Tantsura, "SF Aware TE Topology YANG Model", draft-ietf- 1604 teas-sf-aware-topo-model-04 (work in progress), November 1605 2019. 1607 [I-D.ietf-teas-yang-te] Saad, T., Gandhi, R., Liu, X., Beeram, V., 1608 and I. Bryskin, "A YANG Data Model for Traffic Engineering 1609 Tunnels and Interfaces", draft-ietf-teas-yang-te-22 (work 1610 in progress), November 2019. 1612 [I-D.ietf-teas-yang-te-topo] Liu, X., Bryskin, I., Beeram, V., Saad, 1613 T., Shah, H., and O. Dios, "YANG Data Model for Traffic 1614 Engineering (TE) Topologies", draft-ietf-teas-yang-te- 1615 topo-22 (work in progress), June 2019. 1617 [I-D.www-bess-yang-vpn-service-pm] Wang, Z., Wu, Q., Even, R., Wen, 1618 B., and C. Liu, "A YANG Model for Network and VPN Service 1619 Performance Monitoring", draft-www-bess-yang-vpn-service- 1620 pm-04 (work in progress), November 2019. 1622 [NGMN-NS-Concept] "NGMN NS Concept", 2016, 1623 . 1626 [RFC2475] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z., 1627 and W. Weiss, "An Architecture for Differentiated 1628 Services", RFC 2475, DOI 10.17487/RFC2475, December 1998, 1629 . 1631 [RFC2992] Hopps, C., "Analysis of an Equal-Cost Multi-Path 1632 Algorithm", RFC 2992, DOI 10.17487/RFC2992, November 2000, 1633 . 1635 [RFC3758] Stewart, R., Ramalho, M., Xie, Q., Tuexen, M., and P. 1636 Conrad, "Stream Control Transmission Protocol (SCTP) 1637 Partial Reliability Extension", RFC 3758, DOI 1638 10.17487/RFC3758, May 2004, . 1641 [RFC3931] Lau, J., Ed., Townsley, M., Ed., and I. Goyret, Ed., 1642 "Layer Two Tunneling Protocol - Version 3 (L2TPv3)", RFC 1643 3931, DOI 10.17487/RFC3931, March 2005, . 1646 [RFC4364] Rosen, E. and Y. Rekhter, "BGP/MPLS IP Virtual Private 1647 Networks (VPNs)", RFC 4364, DOI 10.17487/RFC4364, February 1648 2006, . 1650 [RFC4448] Martini, L., Ed., Rosen, E., El-Aawar, N., and G. Heron, 1651 "Encapsulation Methods for Transport of Ethernet over MPLS 1652 Networks", RFC 4448, DOI 10.17487/RFC4448, April 2006, 1653 . 1655 [RFC4594] Babiarz, J., Chan, K., and F. Baker, "Configuration 1656 Guidelines for DiffServ Service Classes", RFC 4594, DOI 1657 10.17487/RFC4594, August 2006, . 1660 [RFC4719] Aggarwal, R., Ed., Townsley, M., Ed., and M. Dos Santos, 1661 Ed., "Transport of Ethernet Frames over Layer 2 Tunneling 1662 Protocol Version 3 (L2TPv3)", RFC 4719, DOI 1663 10.17487/RFC4719, November 2006, . 1666 [RFC5151] Farrel, A., Ed., Ayyangar, A., and JP. Vasseur, "Inter- 1667 Domain MPLS and GMPLS Traffic Engineering - Resource 1668 Reservation Protocol-Traffic Engineering (RSVP-TE) 1669 Extensions", RFC 5151, DOI 10.17487/RFC5151, February 2008, 1670 . 1672 [RFC5654] Niven-Jenkins, B., Ed., Brungard, D., Ed., Betts, M., Ed., 1673 Sprecher, N., and S. Ueno, "Requirements of an MPLS 1674 Transport Profile", RFC 5654, DOI 10.17487/RFC5654, 1675 September 2009, 1677 [RFC7149] Boucadair, M. and C. Jacquenet, "Software-Defined 1678 Networking: A Perspective from within a Service Provider 1679 Environment", RFC 7149, DOI 10.17487/RFC7149, March 2014, 1680 . 1682 [RFC7209] Sajassi, A., Aggarwal, R., Uttaro, J., Bitar, N., 1683 Henderickx, W., and A. Isaac, "Requirements for Ethernet 1684 VPN (EVPN)", RFC 7209, DOI 10.17487/RFC7209, May 2014, 1685 . 1687 [RFC7926] Farrel, A., Ed., Drake, J., Bitar, N., Swallow, G., 1688 Ceccarelli, D., and X. Zhang, "Problem Statement and 1689 Architecture for Information Exchange between 1690 Interconnected Traffic-Engineered Networks", BCP 206, RFC 1691 7926, DOI 10.17487/RFC7926, July 2016, . 1694 [RFC8172] Morton, A., "Considerations for Benchmarking Virtual 1695 Network Functions and Their Infrastructure", RFC 8172, DOI 1696 10.17487/RFC8172, July 2017, . 1699 [RFC8370] Beeram, V., Ed., Minei, I., Shakir, R., Pacella, D., and T. 1700 Saad, "Techniques to Improve the Scalability of RSVP-TE 1701 Deployments", RFC 8370, DOI 10.17487/RFC8370, May 2018, 1702 . 1704 [RFC8403] Geib, R., Ed., Filsfils, C., Pignataro, C., Ed., and N. 1705 Kumar, "A Scalable and Topology-Aware MPLS Data-Plane 1706 Monitoring System", RFC 8403, DOI 10.17487/RFC8403, July 1707 2018, . 1709 [RFC8491] Tantsura, J., Chunduri, U., Aldrin, S., and L. Ginsberg, 1710 "Signaling Maximum SID Depth (MSD) Using IS-IS", RFC 8491, 1711 DOI 10.17487/RFC8491, November 2018, . 1714 [RFC8568] Bernardos, CJ., Rahman, A., Zuniga, JC., Contreras, LM., 1715 Aranda, P., and P. Lynch, "Network Virtualization Research 1716 Challenges", RFC 8568, DOI 10.17487/RFC8568, April 2019, 1717 . 1719 [RFC8577] Sitaraman, H., Beeram, V., Parikh, T., and T. Saad, 1720 "Signaling RSVP-TE Tunnels on a Shared MPLS Forwarding 1721 Plane", RFC 8577, DOI 10.17487/RFC8577, April 2019, 1722 . 1724 [RFC8578] Grossman, E., Ed., "Deterministic Networking Use Cases", 1725 RFC 8578, DOI 10.17487/RFC8578, May 2019, 1726 . 1728 [RFC8655] Finn, N., Thubert, P., Varga, B., and J. Farkas, 1729 "Deterministic Networking Architecture", RFC 8655, DOI 1730 10.17487/RFC8655, October 2019, . 1733 [RFC8665] Psenak, P., Previdi, S., Filsfils, C., Gredler, H., Shakir, 1734 R., Henderickx, W., and J. Tantsura, "OSPF Extensions for 1735 Segment Routing", RFC 8665, DOI 10.17487/RFC8665, December 1736 2019, . 1738 [RFC8667] Previdi, S., Ginsberg, L., Filsfils, C., Bashandy, A., 1739 Gredler, H., and B. Decraene, "IS-IS Extensions for 1740 Segment Routing", RFC 8667, DOI 10.17487/RFC8667, December 1741 2019, . 1743 [SFC] "Service Function Chaining", 1744 . 1746 [TS23501] "3GPP TS23.501", 2019, 1747 . 1750 [TS28530] "3GPP TS28.530", 2019, 1751 . 1754 [TSN] "Time-Sensitive Networking", . 1756 Authors' Addresses 1758 Jie Dong 1759 Huawei 1761 Email: jie.dong@huawei.com 1763 Stewart Bryant 1764 Futurewei 1766 Email: stewart.bryant@gmail.com 1768 Zhenqiang Li 1769 China Mobile 1771 Email: lizhenqiang@chinamobile.com 1773 Takuya Miyasaka 1774 KDDI Corporation 1776 Email: ta-miyasaka@kddi.com 1778 Young Lee 1779 Sung Kyun Kwan University 1781 Email: younglee.tx@gmail.com