idnits 2.17.1 draft-xia-latency-critical-communication-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (October 30, 2017) is 2369 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- No issues found here. Summary: 0 errors (**), 0 flaws (~~), 1 warning (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Internet Draft J. Xia 2 Intended status: Informational Huawei 3 Expires: April 2018 October 30, 2017 5 Architectural Considerations for 6 Delivering Latency Critical Communication over the Internet 7 draft-xia-latency-critical-communication-00.txt 9 Abstract 11 There is clear demand for Internet applications requiring critical 12 low-latency and reliable communication - Latency Critical 13 Communication (LCC). 15 This document is intended to stimulate LCC discussion and is not 16 expected to be published as an RFC. 18 Status of this Memo 20 This Internet-Draft is submitted to IETF in full conformance with 21 the provisions of BCP 78 and BCP 79. 23 Internet-Drafts are working documents of the Internet Engineering 24 Task Force (IETF), its areas, and its working groups. Note that 25 other groups may also distribute working documents as Internet- 26 Drafts. 28 Internet-Drafts are draft documents valid for a maximum of six 29 months and may be updated, replaced, or obsoleted by other documents 30 at any time. It is inappropriate to use Internet-Drafts as 31 reference material or to cite them other than as "work in progress." 33 The list of current Internet-Drafts can be accessed at 34 http://www.ietf.org/ietf/1id-abstracts.txt 36 The list of Internet-Draft Shadow Directories can be accessed at 37 http://www.ietf.org/shadow.html. 39 This Internet-Draft will expire on April 30, 2018. 41 Copyright Notice 43 Copyright (c) 2017 IETF Trust and the persons identified as the 44 document authors. All rights reserved. 46 This document is subject to BCP 78 and the IETF Trust's Legal 47 Provisions Relating to IETF Documents 48 (http://trustee.ietf.org/license-info) in effect on the date of 49 publication of this document. Please review these documents 50 carefully, as they describe your rights and restrictions with 51 respect to this document. Code Components extracted from this 52 document must include Simplified BSD License text as described in 53 Section 4.e of the Trust Legal Provisions and are provided without 54 warranty as described in the Simplified BSD License. 56 Table of Contents 58 1. Introduction ................................................. 2 59 2. The Need for Low Latency Communications ...................... 3 60 3. Quantifying Latency .......................................... 4 61 3.1. Determinism ............................................. 4 62 3.2. Network KPIs ............................................ 4 63 3.3. Service KQIs ............................................ 6 64 3.4. Correlating KQI and KPI ................................. 6 65 3.5. Application Use Cases ................................... 7 66 3.5.1. Cloud-based Virtual Reality ........................ 8 67 3.5.1.1. Quality of Experience Requirements ............ 8 68 3.5.2. Remote Surgery ..................................... 8 69 3.5.2.1. Quality of Experience Requirements ............ 8 70 3.5.3. Live-TV Distribution in Virtualized CDN environments 9 71 3.5.3.1. Quality of Experience Requirements .............9 72 4. Measurement of Latency ...................................... 10 73 4.1. End-to-end Latency ..................................... 11 74 4.2. Per Link Latency ....................................... 12 75 4.3. Per Node Latency ....................................... 12 76 4.4. Reporting Per Link and Per Node Latency ................ 12 77 4.5. Isolating Latency Disruption ........................... 13 78 5. Mechanisms to achieve low latency flows ..................... 13 79 5.1. Path Computation ....................................... 13 80 5.2. Traffic Engineering .................................... 13 81 5.3. Coloring ............................................... 14 82 5.4. Queue Management ....................................... 14 83 5.5. Latency Management ..................................... 15 84 6. Functional Architecture for LCC ............................. 15 85 6.1. LCC Functional Components .............................. 16 86 7. Alternatives to Low Latency Networking ...................... 16 87 7.1. Mitigation ............................................. 16 88 7.2. Resource Placement ..................................... 16 89 7.3. Application and Resource Pipelining .................... 17 90 7.4. Prediction ............................................. 17 91 7.5. Buffering .............................................. 17 92 8. Security Considerations ..................................... 17 93 8.1 Privacy and Regulatory Issues ........................... 17 94 9. IANA Considerations ......................................... 17 95 10. References ................................................. 18 96 10.1. Normative References .................................. 18 97 10.2. Informative References ................................ 18 98 11. Acknowledgments ............................................ 18 100 1. Introduction 102 Latency Critical Communication (LCC) applications are increasingly 103 important, requiring guaranteed low-latency communication high- 104 reliability and ensuring quality of user experience. 106 Several on-going mechanisms exist for delivering LCC services within 107 multiple Standards Development Organizations, including: Time- 108 Sensitive Networking Task Group [TSN8021] in IEEE 802.1, 5G 109 requirements for next-generation access technology [TS38913] in 3GPP 110 and Broadband Assured IP Service [BAS-Architecture] in the BBF. 112 This draft identifies common service requirements in heterogeneous 113 networks for delivering LCC services, and outlines specific uses 114 across a spectrum of applications, specifically: cloud-based virtual 115 reality, remote surgery, and live-TV distribution in virtualized CDN 116 environments. 118 We may scope LCC application requirements by explicitly focusing on 119 end-to-end (E2E) service characteristics and capability requirements 120 for delivering each specific use case. Furthermore, as the E2E 121 service usually traverses multiple domains and involves multiple 122 layers. Yet, existing standards and current discussion typically 123 focuses on a specific layer, protocol, or link layer technology. 124 This focused view lacks an integrated approach or system view on 125 solving the LCC problem space. 127 This document is intended to stimulate discussion and outlines 128 specific LCC application requirements, and proposes an architecture 129 and enabling functional components to address the common 130 requirements discussed in each use case. 132 2. The Need for Low Latency Communications 134 Fundamentally, latency is a time interval between the stimulation 135 and response, or, from a more general point of view, a time delay 136 between the cause and the effect of change in the system being 137 observed. 139 Network latency in packet networks is measured either one-way (the 140 time from the source sending a packet to the destination receiving 141 it), or round-trip delay time (the one-way latency from source to 142 destination plus the one-way latency from the destination back to 143 the source). Some packets will be dropped, i.e., never delivered, 144 due to buffer overflows, synchronization failures, etc. Moreover, we 145 assume that packets that are decoded in error are also dropped 146 either by the protocol itself or by higher layers. Using the 147 convention that dropped packets have infinite latency, we can define 148 the reliability as the probability that the latency does not exceed 149 a pre-described value. 151 Our community has recognized low latency networking as an important 152 research problem, and devoted much attention to tackle the issue 153 from various perspectives, these include: 155 o Processing delays 157 o Buffer delays 159 o Transmission delays 161 o Packet loss 163 o Propagation delays 165 There are a number of common requirements across low latency use 166 cases (including 3GPP on Cloud RAN, front haul, back haul and by 167 various application layers use cases. Additional useful documents 168 exist that provide background and motivation for low latency 169 networks, including [I-D.arkko-arch-low-latency] and [I-D.dunbar- 170 e2e-latency-arch-view-and-gaps]. 172 3. Quantifying Latency 174 LCC Applications exist for a variety of deployments, use cases are 175 assigned into the following categories: 177 o Extreme Mobile Broadband (xMBB): high speed and low latency mobile 178 broadband; 180 o Ultra-reliable Machine-type Communication (uMTC): reliability is 181 the key service requirement of these services; 183 o Massive Machine-Type Communication (mMTC) and Massive IoT (mIoT): 184 massive M2M and IoT connectivity; 186 o Critical Connections/ Ultra Reliable Low Latency Connections 187 (CriC/URLLC): low latency and ultra-reliable communications. 189 The focus of this document is to outline requirements for CriC/URLLC 190 use cases, specifically: 192 o Cloud-based virtual reality; 193 o Remote surgery; 194 o Live-TV distribution in virtualized CDN environments. 196 3.1. Determinism 198 o Difference between predictable and reliable bounds. 200 o Behavior of packet flow, and loss, and/or packets allowed outside 201 of the bounds. 203 3.2. Network KPIs 205 For each category of use case, specific KPIs are identified for 206 clustering requirements: 208 Device density: 209 o High: 10000 devices per km2 210 o Medium: 1000 - 10000 devices per km2 211 o Low: < 1000 devices per km2 213 Mobility: 214 o No: static users 215 o Low: pedestrians (0-3 km/h) 216 o Medium: slow moving vehicles (3 - 50 km/h) 217 o High: fast moving vehicles, e.g. cars and trains (> 50 km/h) 219 Infrastructure: 220 o Limited: no infrastructure available or only macro cell coverage 221 o Medium density: Small number of small cells 222 o Highly available infrastructure: Big number of small cells 223 available 225 Traffic type: 226 o Continuous 227 o Bursty 228 o Event driven 229 o Periodic 230 o All types 232 User data rate: 233 o Very high data rate: 1 Gbps 234 o High: 100 Mbps - 1 Gbps 235 o Medium: 50 - 100 Mbps 236 o Low: < 50 Mbps 238 Latency 239 o High: > 50 ms 240 o Medium: 10 - 50 ms 241 o Low: 1 - 10 ms 243 Reliability 244 o Low: < 95% 245 o Medium: 95 - 99% 246 o High: > 99% 248 Availability (related to coverage) 249 o Low: < 95% 250 o Medium: 95 - 99% 251 o High: > 99% 253 3.3. Service KQIs 255 Application requirements, can be modelled by user experience (QoE), 256 and qualified by service KQI. From users' point of view, QoE is the 257 overall performance of a system. It is a measure of E2E performance 258 at the services level from the user perspective and an indication of 259 how well the system meets the user's needs. There are many factors 260 affecting QoE, such as user expectations, end-to-end system effects, 261 etc. It is essential to establish a relation between user 262 expectations and QoS, considered as the ability of the network to 263 provide a service at a guaranteed performance level. 265 Network's performance can be evaluated with network KPIs such as 266 delay, jitter, packet loss, etc. For URLLC services, existing KPIs 267 are insufficient to forecast the service quality and reflect end- 268 users' QoE. Hence, it is important to identify useful KPIs to 269 quantify end-users' experiences and build the connections between 270 network KPI and service KQI, as shown in Figure 1. The KQI for a 271 given service can be expressed as a function/combination of the 272 KPIs, and can be expressed as follow: KQI=f(KPI1, KPI2,...,KPIn). 274 3.4. Correlating KQI and KPI 276 +-------------+ 277 |Requirements | 278 +--------------+------+---------+ 279 | | | 280 | | | 281 +-----+-----+ +----+---+ +------+----+ 282 Service KQI | KQI1 | | KQI2 | | KQI3 | 283 +-----+-----+ +--------+ +-----------+ 284 | 285 +-------------------------+---------------+ 286 | | | | 287 Network+---v-+ +---v--+ +--------v--------+ +-v---+ 288 KPI |KPI1 | | KPI2 | | KPI3 | | ... | 289 +-----+ +------+ +-----------------+ +-----+ 291 Figure 1: KQI-KPI Correlation 293 The emerging LCC application services have led to composite KQIs 294 use, providing network measurement of specific application service 295 aspects (i.e., the performance of the application or service). As 296 there is limited experience to guide how to deliver the new LCC 297 services, the mapping between the KPI and KQI will require specific 299 3.5. Application Use Cases 301 3.5.1. Cloud-based Virtual Reality 303 Virtual Reality (VR), also known as immersive multimedia or 304 computer-simulated reality, is a computer technology that replicates 305 an environment, real or imagined, and simulates a user's physical 306 presence and environment to allow for user interaction. 308 Although some aspects of VR are becoming promising, there is still 309 bottleneck that prevents it from being popular. High cost, 310 especially for higher-end systems that try to reproduce a high- 311 quality experience, is one barrier to success for VR. One way to 312 reduce the cost of local VR computing, to make it more affordable 313 and increase its popularity and general usage, is to offload the 314 computations to a cloud-based server. This especially fits the 315 cloud-based VR gaming environment when connecting with multiple 316 parties. 318 +----------+ 319 | | | 320 | VR | | 321 | +------| 322 | Device | | 323 | | | 324 +----------+ | ------ 325 | /// \\\ 326 | | VR | 327 +--------+ | 328 | | Cloud | 329 +----------+ | | | 330 | | | \\\\ //// 331 | VR | | ---- 332 | +------| 333 | Device | | 334 | | | 335 +----------+ 337 Figure 2: Cloud-based VR Scenario 339 But then, additional stringent requirements for the underlying 340 network are being introduced into the system, including high 341 bandwidth, low latency and low packet loss ratio. 343 3.5.1.1. Quality of Experience Requirements 345 To make the VR world realistic, the VR motion-to-photon latency is 346 recommended to be less than 20ms. However, the network transmission 347 latency is limited within 5ms because other VR processing (i.e., 348 tracking, rendering, and displaying) latency almost consumes 15ms. 349 To achieve this, the VR cloud is proposed to be deployed at the 350 edge of operator network. 352 Regarding bandwidth requirements, high-end VR systems typically use 353 a display frame rate of 75-90 frames per second on dual HD or 4K 354 displays, which can result in traffic rates four to eight times that 355 for regular HD or 4K video respectively. Of course, low packet loss 356 is required to prevent video freezes or dropouts. 358 +-------------------+------------------+ 359 | Name | Elements | 360 +-------------------+------------------+ 361 |Service type | CriC/URLLC | 362 +-------------------+------------------+ 363 |Bandwidth [Mb/s] | 4K 25Mb/s | 364 | | 8K 100 Mb/s | 365 | | 12K 418 Mb/s | 366 +-------------------+------------------+ 367 | | 4K 16 Mbps | 368 |Bitrate(Mbps) | 8K 64 Mbps | 369 | | 12K 279 Mbps | 370 +-------------------+------------------+ 371 |Latency | 5 ms | 372 +-------------------+------------------+ 373 |Reliability | High (five 9) | 374 +-------------------+------------------+ 375 Figure 3: Cloud VR Service Type 377 3.5.2. Remote Surgery 379 Remote surgery (also known as telesurgery) is the ability for a 380 doctor to perform surgery on a patient even though they are not 381 physically in the same location. It further includes the high-speed 382 communication networks, connecting the surgical robot in one 383 location to the surgeon console at another location manipulating the 384 robot through an operation. 386 Remote telesurgery allows the specialized surgeons to be available 387 to the patients worldwide, without the need for patients to travel 388 beyond their local hospital. Imagine a doctor in an urban city, 389 performing an urgent procedure on a patient in an inaccessible rural 390 area. 392 3.5.2.1. Quality of Experience Requirements 393 In order to ensure a telesurgery procedure to be highly safe, a 394 particularly unique demand is required on the network, at least 395 including very reliable connection (99.999% availability), 396 sufficient bandwidth to ensure adequate video resolution as required 397 by the remote surgeon controlling the robot, as little as possible 398 latency allowing the feel of instantaneous reaction to the actions 399 of the surgeons and of course as little as possible latency 400 variation (i.e., jitter) allowing system or human correction of the 401 latency. 402 +-------------------+------------------+ 403 | Name | Elements | 404 +-------------------+------------------+ 405 |Service type | CriC/URLLC | 406 +-------------------+------------------+ 407 |Bandwidth [Mb/s] | Up to 1Mb/s for | 408 | | control commands | 409 +-------------------+------------------+ 410 |Bitrate(Mbps) | 8K 64 Mbps | 411 +-------------------+------------------+ 412 |Latency | 30 ms | 413 +-------------------+------------------+ 414 |Reliability | High (five 9) | 415 +-------------------+------------------+ 416 Figure 4: Remote Surgery Service Type 418 3.5.3. Live-TV Distribution in Virtualized CDN environments 420 Live-TV signal distribution is a growing service that a network 421 operator needs to support. The bandwidth needed to convey a video 422 stream is determined by its quality. Evolution from standard 423 definition (SD) and high definition (HD) quality formats towards 424 Ultra-High Definition (UHD) formats, including 2K and 4K UHD will 425 have to be carried across an IP network, thus requiring the 426 migration from traditional Serial Digital Interfaces (SDI) 427 transmission to all-IP environments. 429 Various paradigms exist to provide cost-effective scalable live-TV 430 distribution. Specifically, in live-TV distribution, uncompressed 431 video stream formats are used before the video is produced. Once the 432 video has been produced, distribution to end-users is based on 433 compressed video streams, which quality is adapted to the one that 434 fits better the user's device (i.e., compressed SD, HD or UHD 435 formats). 437 Content Delivery Networks (CDN) can be considered as a suitable 438 option for live-TV content delivery by means of the standardized 439 MPEG Dynamic Adaptive Streaming over HTTP (MPEG-DASH) technique. 441 3.5.3.1. Quality of Experience Requirements 442 Transport quality (packet loss, jitter) highly impacts on users' 443 quality of experience (QoE). Undesired effects such as pixelization, 444 tiling, frame freezing, or blue screen can appear if transport 445 quality is slightly degraded. 447 Monitoring at different levels (network, computing, service) and 448 applying local/global KDD procedures enable dynamic adaptive CDN 449 reconfiguration, i.e. scaling up/down HTTP servers, reassigning 450 users, increasing CDN links capacity, etc. 452 +-------------------+------------------+ 453 | Name | Elements | 454 +-------------------+------------------+ 455 |Service type | CriC/URLLC | 456 +-------------------+------------------+ 457 |Bandwidth [Mb/s] | Low 1-4 Mb/s SD | 458 | | Med 10 Mb/s HD | 459 | | High 25 Mb/s UHD | 460 +-------------------+------------------+ 461 |Latency | High 50-100s ms | 462 +-------------------+------------------+ 463 |Jitter | Stringent <1 ms | 464 +-------------------+------------------+ 465 |Reliability | High (five 9) | 466 +-------------------+------------------+ 467 |Availability | Moderate (99%) | 468 +-------------------+------------------+ 469 |Mobility - UE Speed| Up to 50km/h | 470 +-------------------+------------------+ 471 |Area Traffic | Normal 1s Gb/s | 472 | | Hotspot 10s Gb/s | 473 +-------------------+------------------+ 474 |Sensor Network | No | 475 +-------------------+------------------+ 476 |Massive Type | No | 477 +-------------------+------------------+ 478 |Device Direct | No | 479 +-------------------+------------------+ 480 |Coverage Required | Standard | 481 +-------------------+------------------+ 482 |Energy Consumption | No | 483 | Critical | | 484 +-------------------+------------------+ 485 |Type of Use Equip. | Conventional | 486 +-------------------+------------------+ 488 4. Measurement of Latency 490 Various Internet measurement methods have been proposed to identify 491 latency between end hosts. Active network measurements, which 492 involve sending a stream of measurement data traversed along 493 arbitrary paths over a network, including the Internet, are amongst 494 the more popular methods to provision end-to-end quality-of-service. 496 Accurate network measurement would require mesh measurement of all 497 network links to collect sufficient network latency information for 498 network path construction based on active measurement methods. It 499 takes a longer time; thus, it may be possible for a small group of 500 nodes but not for larger number of nodes. Inaccurate measurement 501 over lossy network with long inter-packet delays would become an 502 issue, and not support real-time applications that require time 503 sensitive information for network path decisions. 505 In the [I-D.dunbar-e2e-latency-arch-view-and-gaps], several key 506 latency factors are listed as below: 508 o Generation: delay between physical event and availability of data 510 o Transmission: signal propagation, initial signal encoding 512 o Processing: Forwarding, encoding/decoding, NAT, encryption, 513 authentication, compress, error coding, signal translation 515 o Multiplexing: Delays needed to support sharing; Shared channel 516 acquisition, output queuing, connection establishment 518 o Grouping: Reduces frequency of control information and processing; 519 Packetization, message aggregation 521 From the network point of view, only the last four latency factors 522 are highly relevant to the network characteristic and need to be 523 measured. 525 The E2E performance has been focused on connection-less 526 technologies, the requirements of measuring and maintaining "flow" 527 state for end-user have gaps. 529 Measurement of network delay, performance guarantees, dynamic path 530 adaption, and throughput optimization, all exist but are generally 531 technology specific. 533 4.1. End-to-end Latency 535 A One-way Delay Metric for IPPM is defined for packets across 536 Internet paths based on notions introduced in the IPPM framework 537 with detailed introduction on the measurement methodology, error 538 analysis and relevant statistics. The result can be used to 539 indicate the performance of specific application and the 540 congestion state on the path traversed. 542 IP Flow Information Export (IPFIX) Protocol serves as a 543 means for transmitting Traffic Flow information over the network 544 from an IPFIX Exporting Process to an IPFIX Collecting Process. 546 IPPM or IPFIX should be sufficient for the controller of distributed 547 control plane to make the necessary optimization or bandwidth 548 control, assuming IPFIX and IPPM can measure segment, interface, and 549 chassis/fabric time. But if not, the extension of existing IPPM 550 (metrics) may be needed. 552 In addition, other existing technologies, such as One-Way Active 553 Measurement Protocol (OWAMP) and Two-Way Active Measurement Protocol 554 (TWAMP), are focused on providing one way and two way IP performance 555 metrics. Latency is one of metrics that can be used for End-to-End 556 deterministic latency provisioning. 558 Using OWAMP/TWAMP protocols or extension on that to support 559 measurement of flow latency performance is also feasible. 561 4.2. Per Link Latency 563 Latency related to link can be computed as the ratio between the 564 link length and the propagation speed over the specific medium of 565 the link. 567 The link capacities along the path as well as the way in which the 568 available capacity is used can have impact on the latency. Whenever 569 the link capacity is low, the time of getting data out of network 570 card to onto the medium will be high. Furthermore, capacity is often 571 shared and only a small proportion of the capacity may be available 572 to a specific flow, this is the case when links are congested. 574 4.3. Per Node Latency 576 The links along a network path are connected by network nodes, such 577 as core switches and routers. Transit through each node adds to the 578 path latency, this type of latency is referred to as 579 switching/forwarding latency. 581 To achieve optimized end-to-end low latency services, each network 582 node along the path needs to measure the latency metric on it. Using 583 OWAMP /TWAMP and/or extension on that, each network node needs to 584 record accurate measurements and thus requires accurate time 585 synchronization, which also contributes latency on the network node. 587 4.4. Reporting Per Link and Per Node Latency 589 Basically, the latency information needs to be reported from the 590 network node to the controller or OAM handler [RFC7491] to keep the 591 end-to-end latency under bound. A template that defines the LCC 592 connection attributes, latency, loss and etc, must be used. 594 In addition, an interface or mechanism to report such latency 595 performance information is necessary. A simple approach can be an 596 interface from network device to controller, which collects all the 597 latency performance information of each network node, and then make 598 a decision how to serve the flow at each network node. 600 4.5. Isolating Latency Disruption 602 When congestion occurs, it is often not being detected until it has 603 already induced latency. Early detection of the onset of congestion 604 allows the controllers to reduce their transmission rate quickly. 605 This could use delay based inference of congestion or early explicit 606 notification of congestion by the network. 608 However, the congestion occurred link should be separated with other 609 links and thus will not disrupt the other links. One feasible way is 610 to reserve dedicated network resources to the specific link (for a 611 specific application) and thus isolate the usage of the dedicated 612 network resources from other links. 614 5. Mechanisms to achieve low latency flows 616 The network infrastructure will need advanced interaction with LLC 617 applications. The network will need insight into which types of 618 applications are being transported, and traffic classification and 619 path control to ensure SLAs expected by the applications are met. 620 Several techniques exist to achieve this, and are discussed in the 621 following sub-sections. 623 5.1. Path Computation 625 The Path Computation Element (PCE) was developed to provide path 626 computation services for path controlled networks. The may be used 627 to provide path computation and policy enforcement for LCC 628 applications and services. 630 The PCE operates on a view of the network topology stored in the 631 Traffic Engineering Database (TED). The TED is a data store of 632 topology information about network nodes and links, and capacity and 633 metrics such as link performance (latency, latency-variation, and 634 packet loss). The TED may be further augmented with status 635 information about existing services as well. 637 The PCE would facilitate the setting up of LCC application paths by 638 computing a path based on the end-to-end network performance 639 criteria. 641 5.2. Traffic Engineering 642 Traffic engineering techniques, including Multiprotocol Label 643 Switching (MPLS) Traffic Engineering (MPLS-TE), are commonly 644 deployed by operators. 646 MPLS-TE allows for a TE scheme where the ingress node of a label 647 switched path (LSP) can calculate the most efficient route 648 (including latency minimization) through the network toward the 649 egress router of the LSP. 651 The operator typically has a pre-planning task to monitor the 652 physical layout of the network for capacity planning and network 653 visualization followed by estimation of possible TE settings of the 654 links, knowing how much an IGP setting affects the traffic flow and 655 path. Modification of TE settings to reduce latency based on network 656 conditions is possible, but introduces potential network instability 657 if changes are frequent. 659 Overall, TE technologies come with limitations such as scalability, 660 operational complexity, protocol overhead, and supervised network 661 optimization. Although, recent enhancements to MPLS-TE exist, the 662 interaction between applications and network infrastructure is still 663 not sufficient for the LLC challenges. 665 5.3. Coloring 667 It is possible to build colored paths through the network with the 668 colors representing low bandwidth, low delay, high cost, affinities. 669 Application traffic can then be assigned to those paths based on 670 traffic placement profile. 672 Link coloring could be used to classify specific low latency links 673 for LLC applications, and assigned to a logical topology for the 674 delay-sensitive application traffic. 676 MPLS-TE also supports this function, often described as 677 administrative groups "colors" or "link colors". Furthermore, link 678 coloring is supported in IP networks with the use of MT-aware IGPs. 680 5.4. Queue Management 682 Deploying queue management techniques, such as Active Queue 683 Management (AQM), in the network may facilitate latency reduction, 684 reduce network latency. It may be useful to distinguish between two 685 related classes of algorithms: "queue management" versus 686 "scheduling" algorithms. 688 o Queue management algorithms manage the length of packet queues by 689 marking or dropping packets when necessary or appropriate 690 o Scheduling algorithms determine which packet to send next and are 691 used primarily to manage the allocation of bandwidth among flows. 693 The two mechanisms are loosely related, they address different 694 performance issues and operate on different timescales. 696 As interactive applications (e.g. voice over IP, real time video 697 streaming and financial transactions) run in the Internet, high 698 latency and latency variation degrade application performance. 700 Deploying intelligent queue management and scheduling schemes to 701 control latency and latency variation, would provide desirable and 702 predictable behavior to end-to-end connections for applications 704 5.5. Latency Management 706 Latency management techniques include: 708 o XoDel (Controlled Delay) and FQ-CoDel (FlowQueue-CoDel) Controlled 709 Delay (CoDel) are queue management technologies to set limits per 710 packet for delay 712 o FlowQueue-CoDel (FQ-CoDel) is a hybrid packet scheduler/AQM 713 algorithm for fighting application latency across the Internet. It 714 is based on a two-tier Deficit Round Robin (DRR) queue scheduler, 715 with the CoDel AQM algorithm operating on each sub-queue. 717 6. Functional Architecture for LCC 719 A basic architecture for LCC operation will be required. These will 720 include the necessary components to manage the latency service, 721 underlay packet and transport communications infrastructure. 723 +-------------------------------------------------------------------+ 724 | OSS/NMS/Application Service Coordinator | 725 +--+--------+-------------------------+-------------------------+---+ 726 | | | | 727 | | | | 728 | | | | 729 | +--+---+ +----------------+----------------+ +--+----+ 730 | |Policy+----+ DLN Service Orchestrator +-----+ OAM | 731 | |Agent | +---+------------+-------------+--+ |Handler| 732 | ++-----+ | | | +-----+-+ 733 | | | | | | 734 | | | | | | 735 +---------++ +------+---+ +----+-----+ +---+------+ | 736 |Databases | | Service | | Packet | |Transport | | 737 |Latency DB+------+Controller| |Controller| |Controller| | 738 +--+------++ | +-----+----+ +----+-----+ +---+------+ | 739 | | | | | | | 740 | +--+--+--+ | | | | 741 | | | | | | | 742 | | DLNPC | | | | | 743 | +---+----+ | | | | 744 | | | | | | 745 | | | | | | 746 +---+-------+------------+-------------+-------------+-----------+ 747 | Network elements | 748 +----------------------------------------------------------------+ 750 6.1. LCC Functional Components 752 7. Alternatives to Low Latency Networking 754 7.1. Mitigation 756 Several strategies and techniques exist for reducing or negating 757 network latency for some time sensitive applications. 759 7.2. Resource Placement 761 There is a trend of placing resources in locations that would reduce 762 or negate service and application latency. 764 One approach to support more dynamic placement of functions, 765 enabling the LLC application, close to the user is to introduce 766 Virtualized Network Functions (NFV) at the edge of the network, near 767 the LCC application users to reduce end-to-end latency, time-to- 768 response, and unnecessary utilization of the core network 769 infrastructure. 771 Specific technology threads developing edge resource placement, via 772 virtual functions, include Mobile Edge Computing (MEC) and Fog 773 Computing. 775 7.3. Application and Resource Pipelining 777 To be discussed. 779 7.4. Prediction 781 To be discussed. 783 7.5. Buffering 785 Reducing switch queue length, or buffer occupancy, is the most 786 direct way to tackle the latency problem. Packet forwarders could 787 use deep buffers to handle bursty traffic. However, they must ensure 788 that this does not become detrimental to latency performance. As TCP 789 relies on packet drops for congestion control, it introduces 790 overhead for the application. 792 8. Security Considerations 794 The following list provides some security challenges and 795 considerations in designing and building network infrastructure for 796 LLC applications: 798 o Identification and authentication of the entities involved in the 799 LLC service 801 o Access control to the different entities that need to be connected 802 and capable of creating LLC services 804 o Processes and mechanisms to guarantee availability of LLC network 805 resources and protect them against attack 807 8.1 Privacy and Regulatory Issues 809 o Identification of endpoints 811 o Data protection to guarantee the security (confidentiality, 812 integrity, availability, authenticity) and privacy of the 813 information carried by the network for the LCC service 815 9. IANA Considerations 817 10. References 818 10.1. Normative References 820 10.2. Informative References 822 [TSN8021] "Time-Sensitive Networking Task Group", IEEE 823 (http://www.ieee802.org/1/pages/tsn.html). 825 [BAS-Architecture] Y.L. Jiang, "Broadband Assured IP Services 826 Architecture", draft WT-387-00, broadband forum (BBF), July, 2016. 828 [TS38913] "3rd Generation Partnership Project; Technical 829 Specification Group Radio Access Network; Study on Scenarios and 830 Requirements for Next Generation Access Technologies; (Release 14)" 832 [I-D.arkko-arch-low-latency] J. Arkko, "Low Latency Applications and 833 the Internet Architecture", draft-arkko-arch-low-latency (work in 834 progress), 2017. 836 [I-D.dunbar-e2e-latency-arch-view-and-gaps] Dunbar, L., 837 "Architectural View of E2E Latency and Gaps", draft-dunbar-e2e- 838 latency-arch-view-and-gaps (work in progress), 2017. 840 [MEC_White_Paper] ETSI, "Mobile-Edge Computing - Introductory 841 Technical White Paper", 2014. 843 [RFC7491] D. King and A. Farrel, "A PCE-Based Architecture for 844 Application-Based Network Operations ", RFC 7491, March 2015, 845 . 847 11. Acknowledgments 849 Authors' Addresses 851 Jinwei Xia 852 Huawei 853 101 Software Avenue, Yuhua District 854 Nanjing, Jiangsu 210012 855 China 857 Email: xiajinwei@huawei.com 859 Contributors 861 Ning Zong 862 Huawei Technologies 864 Email: zongning@huawei.com 866 Daniel King 867 Lancaster University 869 Email: d.king@lancaster.ac.uk