idnits 2.17.1 draft-arkko-arch-low-latency-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack a Security Considerations section. ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (October 30, 2017) is 2370 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-02) exists of draft-dunbar-e2e-latency-arch-view-and-gaps-01 == Outdated reference: A later version (-34) exists of draft-ietf-quic-transport-07 == Outdated reference: A later version (-28) exists of draft-ietf-sfc-nsh-27 == Outdated reference: A later version (-28) exists of draft-ietf-tls-tls13-21 -- Obsolete informational reference (is this intentional?): RFC 6555 (Obsoleted by RFC 8305) -- Obsolete informational reference (is this intentional?): RFC 6824 (Obsoleted by RFC 8684) -- Obsolete informational reference (is this intentional?): RFC 7540 (Obsoleted by RFC 9113) Summary: 2 errors (**), 0 flaws (~~), 5 warnings (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force J. Arkko 3 Internet-Draft Ericsson 4 Intended status: Informational J. Tantsura 5 Expires: May 3, 2018 Futurewei, Future Networks 6 October 30, 2017 8 Low Latency Applications and the Internet Architecture 9 draft-arkko-arch-low-latency-02 11 Abstract 13 Some recent Internet technology developments relate to improvements 14 in communications latency. For instance, improvements in radio 15 communications or the recent work in IETF transport, security, and 16 web protocols. There are also potential applications where latency 17 would play a more significant role than it has traditionally been in 18 the Internet communications. Modern networking systems offer many 19 tools for building low-latency networks, from highly optimised 20 individual protocol components to software controlled, virtualised 21 and tailored network functions. This memo views the developments 22 from a system viewpoint, and considers the potential future stresses 23 that the strive for low-latency support for applications may bring. 25 Status of This Memo 27 This Internet-Draft is submitted in full conformance with the 28 provisions of BCP 78 and BCP 79. 30 Internet-Drafts are working documents of the Internet Engineering 31 Task Force (IETF). Note that other groups may also distribute 32 working documents as Internet-Drafts. The list of current Internet- 33 Drafts is at https://datatracker.ietf.org/drafts/current/. 35 Internet-Drafts are draft documents valid for a maximum of six months 36 and may be updated, replaced, or obsoleted by other documents at any 37 time. It is inappropriate to use Internet-Drafts as reference 38 material or to cite them other than as "work in progress." 40 This Internet-Draft will expire on May 3, 2018. 42 Copyright Notice 44 Copyright (c) 2017 IETF Trust and the persons identified as the 45 document authors. All rights reserved. 47 This document is subject to BCP 78 and the IETF Trust's Legal 48 Provisions Relating to IETF Documents 49 (https://trustee.ietf.org/license-info) in effect on the date of 50 publication of this document. Please review these documents 51 carefully, as they describe your rights and restrictions with respect 52 to this document. Code Components extracted from this document must 53 include Simplified BSD License text as described in Section 4.e of 54 the Trust Legal Provisions and are provided without warranty as 55 described in the Simplified BSD License. 57 Table of Contents 59 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 60 2. Applications with Special Focus on Low Latency . . . . . . . 3 61 3. Role of Low-Latency vs. Other Communications . . . . . . . . 4 62 4. Selected Improvements to Communications Latency . . . . . . . 5 63 5. Architectural Considerations . . . . . . . . . . . . . . . . 6 64 5.1. Background . . . . . . . . . . . . . . . . . . . . . . . 6 65 5.2. Implications . . . . . . . . . . . . . . . . . . . . . . 7 66 5.2.1. Service Distribution . . . . . . . . . . . . . . . . 7 67 5.2.2. Edge Computing . . . . . . . . . . . . . . . . . . . 8 68 5.2.3. Routing and tunnels . . . . . . . . . . . . . . . . . 8 69 5.2.4. Alternative Paths and Control Tension . . . . . . . . 9 70 5.2.5. Quality-of-Service . . . . . . . . . . . . . . . . . 9 71 5.2.6. Cross-Layer Optimisations . . . . . . . . . . . . . . 10 72 5.3. Recommendations for Further Work . . . . . . . . . . . . 11 73 6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 12 74 7. Informative References . . . . . . . . . . . . . . . . . . . 12 75 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 15 77 1. Introduction 79 Some recent Internet technology developments relate to improvements 80 in communications latency. For instance, improvements in radio 81 communications or the recent work in IETF transport, security, and 82 web protocols. 84 There are also potential applications where latency would play a more 85 significant role than it has traditionally been in the Internet 86 communications. 88 New applications or technologies do not necessarily imply that 89 latency should be the main driving concern, or that any further 90 efforts are needed, beyond those already ongoing. Indeed, modern 91 networking systems offer many tools for building low-latency 92 networks, across the stack. At the IETF, for instance, there has 93 been a recent increase in work related to transport, security, and 94 web application protocols, in part to make significant improvements 95 in latency and connection set-up times. Similar efforts for other 96 components of communications technology exist in 3GPP, IEEE, and 97 other standards organisations. 99 Despite a large number of specific developments, it may be 100 interesting to view the developments from a system viewpoint, and to 101 consider the potential future stresses that the strive for low- 102 latency support applications may bring. 104 The rest of this memo is organised as follows: Section 2 discusses 105 potential applications for low-latency communications. Section 4 106 reviews some of the recent work across the stack, related to latency 107 improvements. Finally, Section 5 discusses some of the implications 108 (and non-implications) from an architectural perspective. 110 2. Applications with Special Focus on Low Latency 112 Most Internet applications enjoy significant benefits from low- 113 latency communications in the form of faster setup and response times 114 as well as higher bandwidth communications enabled by transport 115 protocol behaviour [RFC7323]. 117 There are also potential applications where latency would play an 118 even more significant role. For instance, embedding communications 119 technology in automation or traffic systems, or consumer applications 120 such as augmented or virtual reality where due to the human brain's 121 perceptual limits variability in latency may not be feasible, i.e., 122 render the service unusable due to motion sickness caused. 124 Many of the Internet-of-Things and critical services use cases in 5G, 125 for instance, have been listed as requiring low latency and high 126 reliability for communications [ER2015] [HU2015] [NGMN2015] [NO2015] 127 [QU2016] [IMT2020]. 129 Some example use cases include optimisation of utility services such 130 as electricity networks, connected automation systems in factories, 131 remote control of machinery such as mining equipment, or embedded 132 technology in road or railway traffic. 134 The different applications vary in terms of their needs. Some may be 135 very focused on high-speed local area communication, others need to 136 connect at optimal speed over a wide-area network, and yet others 137 need to find the right ways to provide global services without 138 incurring unreasonable delays. 140 For these reasons it is difficult to specify what "low latency" means 141 in terms of specific delays. Applications and network scenarios 142 differ. Reaching a 50ms latency may be enough for some applications 143 while others may require 50us. Obviously, latency is ultimately 144 limited by physics, location, and topology. Individual link 145 characteristics are important, but system level communication needs 146 both in terms of what is being communicated and between what parties 147 matter more. 149 Note that when we say "low-latency capabilities", there is no intent 150 to imply any specific implementation of those capabilities. In 151 particular, we look at the low-latency requirements from a broader 152 perspective than Quality-of-Service guarantees or separating traffic 153 onto different classes. Indeed, while today's virtualisation and 154 software-driven technologies give us more tools to deal with those 155 kinds of arrangements as well, past experience on deploying Quality- 156 of-Service mechanisms in the Internet should give us a pause 157 [CC2015]. 159 It is not the purpose of this memo to analyse the application 160 requirements for low-latency applications much further; for our 161 purposes it suffices to note that there are applications that are 162 enabled by low-latency capabilities of the underlying network 163 infrastructure. 165 3. Role of Low-Latency vs. Other Communications 167 There are some limited applications that rely solely on local 168 communication. One example of such an application is vehicles 169 communicating braking status to nearby ones. 171 Also, while many applications run in the global Internet, some are 172 designed for specialised networks that may not have full or even any 173 Internet connectivity, but yet use IP technology. 175 However, many applications will include also wide-area communication. 176 If the factory automation machines are not talking other than with 177 themselves, at least their control systems are doing so in order to 178 ensure parts orders, monitoring and maintenance by equipment 179 manufacturers, and so on. This does not imply that these perhaps 180 critical applications are openly accessible from the Internet, but 181 many of them are likely to communicate outside their immediate 182 surroundings. 184 Many applications also rely on wide-area connectivity for software 185 updates. 187 As a result, this document recommends that when building 188 architectures for low-latency applications it is important to take 189 into account that these applications can also benefit from 190 communications elsewhere. Or at the very least, the existence of a 191 specialised communications link or network should not be immediately 192 taken to mean that no other communications are needed. 194 4. Selected Improvements to Communications Latency 196 It should be noted that latency is a very broad topic in 197 communications protocol design, almost as broad as "security", or 198 even "correctness". 200 Implementation techniques to satisfy these requirements vary, some 201 applications can be built with sufficient fast local networking 202 capabilities, others may require, for instance, building world-wide, 203 distributed content delivery mechanisms. 205 Modern networking systems offer many tools for building low-latency 206 networks, from highly optimised individual protocol components 207 [I-D.ietf-tls-tls13] [I-D.ietf-quic-transport] [RFC7413] [RFC7540] to 208 software controlled, virtualised and tailored network functions 209 [NFV2012] [RFC7665] [I-D.ietf-sfc-nsh] [OF2008]. Data- and software- 210 driven network management and orchestration tools enable networks to 211 be built to serve particular needs as well as to optimize workload 212 placement in a way low-latency requirements could be met. 214 Obviously, low-latency communications are not merely a matter of 215 protocols and their optimisation. Implementation techniques matter, 216 from the placement of network functions and nodes in the right 217 places, to the quality of individual function implementations. 218 Today's technology allows much freedom on placement of (virtual) 219 functions at chosen locations and many options for all functions 220 ranging from load balancing to storage to packet processing to 221 management tools. Good design can provide significant gains by 222 reducing latency in and between network components, reducing the 223 necessary control traffic and state synchronization, and so on. 225 Across the stack there are also many other protocol tools, as well as 226 tools being in development, e.g., a new transport design [L4S] at the 227 IETF. 229 On the lower layers, improvements in radio communications are being 230 made. For instance, the IEEE 802.1 Time-Sensitive Networking Task 231 Group [TSN8021] has worked to define precise time synchronization 232 mechanisms for a local area network, and scheduling mechanisms to 233 enable different classes of traffic to use the same network while 234 minimising jitter and latency. At the IETF, the DETNET working group 235 is taking these capabilities and applying them for layer 3 networking 236 [DETNET]. 238 The 3GPP 5G requirements for next-generation access technology are 239 stringent, and are leading to the optimization of the radio 240 interfaces. The requirements specify a one-way latency limit of 241 0.5ms for ultra-reliable low-latency communications [TS38913]. But 242 again, mere latency numbers mean very little without the context of a 243 system and what an application needs to communicate and with whom. 245 5. Architectural Considerations 247 Despite a large number of specific developments, it may be 248 interesting to view the developments from a system viewpoint, and to 249 consider the potential future stresses that the strive for low- 250 latency support for applications may bring. 252 5.1. Background 254 To begin with, it may be useful to observe that the requirements and 255 developments outlined above do not necessarily imply that any 256 specific new technology is needed or that the nature of 257 communications in the Internet would somehow fundamentally change. 258 And certainly not that latency should be the only or primary concern 259 in technology development. 261 With the drive for a new class of applications, there is often an 262 expectation that this means significant changes. However, all 263 changes need to stand on their own, be justifiable and deployable on 264 a global network. For instance, the discussion around the 265 introduction of the newest 4K or 8K high-definition video streaming 266 applications is reminiscent of the discussions about the introduction 267 of VoIP applications in the Internet. At the time, there was some 268 expectation that special arrangements and Quality-of-Service 269 mechanisms might be needed to support this new traffic class. This 270 turned out to be not true, at least not in general networks. 272 Experience tells us, for instance, that deploying Quality-of-Service 273 mechanisms in the Internet is hard, not so much because of the 274 technology itself, but due to lack of forces that would be able to 275 drive the necessary business changes in the ecosystem for the 276 technology to be feasibly deployable [CC2015]. As claffy and Clark 277 note: 279 "Although the Internet has a standards body (the IETF) to resolve 280 technical issues, it lacks any similar forum to discuss business 281 issues such as how to allocate revenues among competing ISPs 282 offering enhanced services. In the U.S., ISPs feared such 283 discussions would risk anti-trust scrutiny. Thus, lacking a way 284 to negotiate the business implications of QoS, it was considered a 285 cost rather than a potential source of revenue. Yet, the 286 relentless growth of a diversity of applications with widely 287 varying performance requirements continued on the public Internet, 288 with ISPs using relatively primitive, and not always completely 289 benign, mechanisms for handling them." 291 These difficulties should not be read as prohibiting all changes. Of 292 course, change can also seem unlikely even in cases where it becomes 293 absolutely necessary or the forces necessary to make a change have 294 actually built up. As a result, statements regarding change in the 295 Internet should be carefully evaluated on their merits from both 296 technical and ecosystem perspective. 298 Secondly, we often consider characteristics from a too narrow 299 viewpoint. In the case of latency, it is easy to focus on a 300 particular protocol or link, whereas from the user perspective 301 latency is a property of the system, not a property of an individual 302 component. 304 For instance, improvements on the performance of one link on a 305 communications path can be insignificant, if the other parts make up 306 a significant fraction of the system-level latency. That may seem 307 obvious, but many applications are highly dependent on communications 308 between a number of different parties which may reside in different 309 places. For instance, a third party may perform authentication for a 310 cloud-based service that also interacts with user's devices and a 311 number of different sensors and actuators. 313 We cannot change the speed of light, and a single exchange with 314 another part of the world may result in a 100ms delay, or about 200 315 times longer than the expected 5G radio link delay for critical 316 applications. It is clear that designing applications from a system 317 perspective is very important. 319 5.2. Implications 321 This section discusses a selected set of architectural effects and 322 design choices within applications that desire low latency 323 communications. 325 5.2.1. Service Distribution 327 As noted above, low-latency applications need to pay particular 328 attention to the placement of services in the global network. 329 Operations that are on the critical path for the low-latency aspects 330 of an application are unlikely to work well if those communications 331 need to traverse half of the Internet. 333 Many widely used services are already distributed and replicated 334 throughout the world, to minimise communications latency. But many 335 other services are not distributed in this manner. For low-latency 336 applications such distribution becomes necessary. Hosting a global 337 service in one location is not feasible due to latency, even when 338 from a scale perspective a single server might otherwise suffice for 339 the service. All major public cloud providers offer CDN services to 340 their customers - AWS's CloudFront, Google's Cloud CDN and Azure's 341 CDN to mention a few. 343 Content-Delivery Networks (CDNs) and similar arrangements are likely 344 to flourish because of this. These arrangements can bring content 345 close to end-users, and have a significant impact on latency. 346 Typical CDN arrangements provide services that are on a global scale 347 nearby, e.g., in the same country or even at the ISP's datacenter. 349 Today's CDNs are of course just one form of distributed service 350 implementation. Previous generations, such as web caching, have 351 existed as well, and it is likely that the current arrangements will 352 evolve in the future. CDN evolution is also naturally affected not 353 only by the need to provide services closer to the user, but also 354 through the fine-grained control and visibility mechanisms that it 355 gives to the content owners. Such factors continue to affect also 356 future evolution, e.g., any information-centric networking solutions 357 that might emerge. 359 5.2.2. Edge Computing 361 Recent advances in "edge computing" take the more traditional type 362 service like CDN as well as a new class of services that require 363 "local compute" capabilities placement even further by providing 364 services near the users. This would enable more extreme uses cases 365 where latency from, say, ISP datacenter to the users is considered 366 too high. An important consideration is what is considered an edge, 367 however. From Internet perspective edge usually refers to the IP 368 point of presence or the first IP hop. But given the centralised 369 nature of many access networks, some of the discussions around the 370 use of edge computing also involve components at the edge that are 371 much closer to user than the first IP hop. Special arrangements are 372 needed to enable direct IP connectivity from the user equipment to 373 these components. 375 5.2.3. Routing and tunnels 377 How the communications are routed also matters. For instance, 378 architectures based on tunneling to a central point may incur extra 379 delay. One way to address this pressure is to use SDN- and 380 virtualisation-based networks that can be provisioned in the desired 381 manner, so that, for instance, instances of tunnel servers can be 382 placed in the topologically optimal place for a particular 383 application. 385 5.2.4. Alternative Paths and Control Tension 387 Recent developments in multipath transport protocols [RFC6824] also 388 provide application- and service-level control of some of the 389 networking behaviour. Similar choices among alternative paths also 390 exist in simpler techniques, ranging from server selection algorithms 391 to IPv6 "Happy Eyeballs" algorithms [RFC6555]. In all of these cases 392 an application makes some observations of the environment and decides 393 to use an alternative path or target that is perceived to be best 394 suited for the application's needs. 396 In all of these multipath and alternative selection techniques there 397 is tension between application control (often by content providers) 398 and network control (often by network operators). 400 One special case where that tension has appeared in the past is 401 whether there should be ways to provide information from applications 402 to networks on how packets should be treated. This was extensively 403 discussed during the discussion stemming from implications of 404 increased use of encryption in the Internet, and how that affects 405 operators [I-D.nrooney-marnew-report]. 407 Another case where there is tension is between mechanisms designed 408 for a single link or network vs. end-to-end mechanisms. Many of the 409 stated requirements for low-latency applications are explicitly about 410 end-to-end characteristics and capabilities. Yet, the two mechanisms 411 are very different, and most of the deployment difficulties reported 412 in [CC2015] relate to end-to-end mechanisms. 414 Note that some of the multipath techniques can be used either by 415 endpoints or by the network. Proxy-based Multipath TCP is one 416 example of this [I-D.boucadair-mptcp-plain-mode]. 418 5.2.5. Quality-of-Service 420 Existing approaches have not necessarily proven to be technical 421 deficient in any way, but it seems that it is reasonable to draw some 422 conclusions about the lack of feasibility for deploying mechanisms 423 that require a high degree of coordination among multiple parties. 424 Without significant changes in the marketplace or other conditions, 425 these types of solutions do not seem likely to get traction. 427 The other observation that may be worth noting is that many networks 428 have focused on providing highly tuned services for a relatively 429 small fractions of traffic. While this may be justifiable from the 430 perspective of special applications needing that support, this does 431 not seem to produce a good bang-for-the-buck ratio. There's 432 relatively small amount of work on mechanisms that would help a large 433 fraction of applications. For instance, in many access networks the 434 over-the-top content is by far the biggest source of traffic. 435 Solutions that are business-wise deployable for such traffic would 436 seem preferrable. 438 5.2.6. Cross-Layer Optimisations 440 In the search for even faster connection setup times one obvious 441 technique is cross-layer optimisation. We have seen some of this in 442 the IETF in the rethinking of the layers for transport, transport 443 layer security, and application framework protocols. By taking into 444 account the protocol layer interactions or even bundling the protocol 445 design together, it is relatively easy to optimise the connection 446 setup time, as evidenced by recent efforts to look for "0-RTT" 447 designs in various protocols. 449 But while cross-layer optimisation can bring benefits, it also has 450 downsides. In particular, it connects different parts of the stack 451 in additional ways. This can lead to difficulties in further 452 evolution of the technology, if done wrong. 454 In the case of the IETF transport protocol evolution, significant 455 improvements were made to ensure better evolvability of the 456 protocols than what we've experienced with TCP, starting from an 457 ability to implement the new protocols in applications rather than 458 in the kernel. 460 While the connection setup is an obvious example, cross-layer 461 optimisations are not limited to them. Interfaces between 462 application, transport, networking, and link layers can provide 463 information and set parameters that improve latency. For instance, 464 setting DSCP values or requesting a specialised L2 service for a 465 particular application. Cross-Layer optimisations between lower 466 layers will be discussed in the upcoming versions of the draft. 468 The effects of badly designed cross-layer optimisation are a 469 particular form of Internet ossification. The general networking 470 trend, however, is for greater flexibility and programmability. 471 Arguably, the ease at which networks can evolve is probably even more 472 important than their specific characteristics. 474 These comments about cross-layer optimisation should not be 475 interpreted to mean that protocol design should not take into account 476 how other layers behave. The IETF has a long tradition of discussing 477 link layer design implications for Internet communications (see, 478 e.g., the results of the PILC working group [RFC3819]. 480 5.3. Recommendations for Further Work 482 Low-latency applications continue to be a hot topic in networking. 483 The following topics in particular deserve further work from an 484 architectural point of view: 486 o Application architectures for globally connected but low-latency 487 services. 489 o What are the issues with inter-domain Quality-of-Service 490 mechanisms? Are there approaches that would offer progress on 491 this field? Work on Quality-of-Service mechanisms that are 492 deployable for common cases, and without excessive need for 493 technical and non-technical coordination across the ecosystem. 495 o Network architectures that employ tunneling, and mitigations 496 against the delay impacts of tunnels (such as tunnel server 497 placement or "local breakout" techniques). Low latency often 498 implies high reliability, special care is to be taken of network 499 convergence, and other, relevant characteristics of the underlying 500 infrastructure. 502 o The emergence of cross-layer optimisations and how that affects 503 the Internet architecture and its future evolution. 505 o Inter-organisatorial matters, e.g., to what extent different 506 standards organisations need to talk about low latency effects and 507 ongoing work, to promote system-level understanding? 509 Overall, this memo stresses the importance of the system-level 510 understanding of Internet applications and their latency issues. 511 Efforts to address specific sub-issues are unlikely to be fruitful 512 without a holistic plan. 514 In the authors' opinion, the most extreme use cases (e.g., 1ms or 515 smaller latencies) are not worth building general-purpose networks 516 for. But having the necessary tools so that networks can be flexible 517 for the more general cases is very useful, as there are many 518 applications that can benefit from the added flexibility. The key 519 tools for this include ability to manage network function placement 520 and topology. 522 6. Acknowledgements 524 The author would like to thank Brian Trammell, Mirja Kuehlewind, 525 Linda Dunbar, Goran Rune, Ari Keranen, James Kempf, Stephen Farrell, 526 Mohamed Boucadair, Kumar Balachandran, Jani-Pekka Kainulainen, Goran 527 AP Eriksson, and many others for interesting discussions in this 528 problem space. 530 The author would also like to acknowledge the important contribution 531 that [I-D.dunbar-e2e-latency-arch-view-and-gaps] made in this topic. 533 7. Informative References 535 [CC2015] claffy, kc. and D. Clark, "Adding Enhanced Services to the 536 Internet: Lessons from History", September 2015 537 (https://www.caida.org/publications/papers/2015/ 538 adding_enhanced_services_internet/ 539 adding_enhanced_services_internet.pdf). 541 [DETNET] "Deterministic Networking (DETNET) Working Group", March 542 2016 (https://tools.ietf.org/wg/detnet/charters). 544 [ER2015] Yilmaz, O., "5G Radio Access for Ultra-Reliable and Low- 545 Latency Communications", Ericsson Research Blog, May 2015 546 (https://www.ericsson.com/research-blog/5g/5g-radio- 547 access-for-ultra-reliable-and-low-latency- 548 communications/). 550 [HU2015] "5G Vision: 100 Billion connections, 1 ms Latency, and 10 551 Gbps Throughput", Huawei 2015 552 (http://www.huawei.com/minisite/5g/en/defining-5g.html). 554 [I-D.boucadair-mptcp-plain-mode] 555 Boucadair, M., Jacquenet, C., Bonaventure, O., Behaghel, 556 D., stefano.secci@lip6.fr, s., Henderickx, W., Skog, R., 557 Vinapamula, S., Seo, S., Cloetens, W., Meyer, U., 558 Contreras, L., and B. Peirens, "Extensions for Network- 559 Assisted MPTCP Deployment Models", draft-boucadair-mptcp- 560 plain-mode-10 (work in progress), March 2017. 562 [I-D.dunbar-e2e-latency-arch-view-and-gaps] 563 Dunbar, L., "Architectural View of E2E Latency and Gaps", 564 draft-dunbar-e2e-latency-arch-view-and-gaps-01 (work in 565 progress), March 2017. 567 [I-D.ietf-quic-transport] 568 Iyengar, J. and M. Thomson, "QUIC: A UDP-Based Multiplexed 569 and Secure Transport", draft-ietf-quic-transport-07 (work 570 in progress), October 2017. 572 [I-D.ietf-sfc-nsh] 573 Quinn, P., Elzur, U., and C. Pignataro, "Network Service 574 Header (NSH)", draft-ietf-sfc-nsh-27 (work in progress), 575 October 2017. 577 [I-D.ietf-tls-tls13] 578 Rescorla, E., "The Transport Layer Security (TLS) Protocol 579 Version 1.3", draft-ietf-tls-tls13-21 (work in progress), 580 July 2017. 582 [I-D.nrooney-marnew-report] 583 Rooney, N., "IAB Workshop on Managing Radio Networks in an 584 Encrypted World (MaRNEW) Report", draft-nrooney-marnew- 585 report-03 (work in progress), June 2017. 587 [IMT2020] "Framework and overall objectives of the future 588 development of IMT for 2020 and beyond", ITU 589 Recommendation M.2083-0, September 2015 590 (http://www.itu.int/rec/R-REC-M.2083-0-201509-I/en). 592 [L4S] "Low Latency Low Loss Scalable throughput (L4S) Birds-of- 593 Feather Session", July 2016 594 (https://datatracker.ietf.org/wg/l4s/charter/). 596 [NFV2012] "Network Functions Virtualisation - Introductory White 597 Paper", ETSI, 598 http://portal.etsi.org/NFV/NFV_White_Paper.pdf, October 599 2012. 601 [NGMN2015] 602 "5G White Paper", NGMN Alliance, February 2015 603 (https://www.ngmn.org/fileadmin/ngmn/content/downloads/ 604 Technical/2015/NGMN_5G_White_Paper_V1_0.pdf). 606 [NO2015] Doppler, K., "5G the next major wireless standard", DREAMS 607 Seminar, January 2015 608 (https://chess.eecs.berkeley.edu/pubs/1084/ 609 doppler-DREAMS_5G_jan15.pdf). 611 [OF2008] McKeown, N., Anderson, T., Balakrishnan, H., Parulkar, G., 612 Peterson, L., Rexford, J., Shenker, S., and J. Turner, 613 "OpenFlow: Enabling Innovation in Campus Networks", ACM 614 SIGCOMM Computer Communication Review, Volume 38, Issue 2, 615 pp. 69-74 2008. 617 [QU2016] "Leading the world to 5G", Qualcomm, February 2016 618 (https://www.qualcomm.com/media/documents/files/ 619 qualcomm-5g-vision-presentation.pdf). 621 [RFC3819] Karn, P., Ed., Bormann, C., Fairhurst, G., Grossman, D., 622 Ludwig, R., Mahdavi, J., Montenegro, G., Touch, J., and L. 623 Wood, "Advice for Internet Subnetwork Designers", BCP 89, 624 RFC 3819, DOI 10.17487/RFC3819, July 2004, 625 . 627 [RFC6555] Wing, D. and A. Yourtchenko, "Happy Eyeballs: Success with 628 Dual-Stack Hosts", RFC 6555, DOI 10.17487/RFC6555, April 629 2012, . 631 [RFC6824] Ford, A., Raiciu, C., Handley, M., and O. Bonaventure, 632 "TCP Extensions for Multipath Operation with Multiple 633 Addresses", RFC 6824, DOI 10.17487/RFC6824, January 2013, 634 . 636 [RFC7323] Borman, D., Braden, B., Jacobson, V., and R. 637 Scheffenegger, Ed., "TCP Extensions for High Performance", 638 RFC 7323, DOI 10.17487/RFC7323, September 2014, 639 . 641 [RFC7413] Cheng, Y., Chu, J., Radhakrishnan, S., and A. Jain, "TCP 642 Fast Open", RFC 7413, DOI 10.17487/RFC7413, December 2014, 643 . 645 [RFC7540] Belshe, M., Peon, R., and M. Thomson, Ed., "Hypertext 646 Transfer Protocol Version 2 (HTTP/2)", RFC 7540, 647 DOI 10.17487/RFC7540, May 2015, 648 . 650 [RFC7665] Halpern, J., Ed. and C. Pignataro, Ed., "Service Function 651 Chaining (SFC) Architecture", RFC 7665, 652 DOI 10.17487/RFC7665, October 2015, 653 . 655 [TS38913] "3rd Generation Partnership Project; Technical 656 Specification Group Radio Access Network; Study on 657 Scenarios and Requirements for Next Generation Access 658 Technologies; (Release 14)", 3GPP Technical Report TR 659 38.913 V14.2.0, March 2017 660 (https://portal.3gpp.org/desktopmodules/Specifications/ 661 SpecificationDetails.aspx?specificationId=2996). 663 [TSN8021] "Time-Sensitive Networking Task Group", IEEE 664 (http://www.ieee802.org/1/pages/tsn.html). 666 Authors' Addresses 668 Jari Arkko 669 Ericsson 670 Kauniainen 02700 671 Finland 673 Email: jari.arkko@piuha.net 675 Jeff Tantsura 676 Futurewei, Future Networks 678 Email: jefftant.ietf@gmail.com