idnits 2.17.1 draft-li-overlayed-path-segment-forwarding-ps-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 143 has weird spacing: '...merging multi...' == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (September 26, 2018) is 2032 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'RFC8174' is mentioned on line 113, but not defined == Unused Reference: 'RFC4122' is defined on line 368, but no explicit reference was found in the text -- Obsolete informational reference (is this intentional?): RFC 6830 (Obsoleted by RFC 9300, RFC 9301) Summary: 0 errors (**), 0 flaws (~~), 5 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 TSVWG Y. Li 3 INTERNET-DRAFT X. Zhou 4 Intended Status: Informational Huawei 5 Expires: March 30, 2019 September 26, 2018 7 Overlayed Path Segment Forwarding (OPSF) Problem Statement 8 draft-li-overlayed-path-segment-forwarding-ps-00 10 Abstract 12 Various overlays are used in networks including WAN, enterprise 13 campus and others. End to end path are divided into multiple segments 14 some of which are overlay encapsulated to achieve better path 15 selection, lower latency and so on. Traditional end-to-end transport 16 layer is not very responding to microburst and non-congestive packet 17 loss caused by the different characteristics of the path segments. 18 With the potential transport enhancement for the existing or 19 purposely created overlayed path segment, end to end throughput can 20 be improved. This document illustrates the problems in some use cases 21 and tries to inspire more about whether and how to solve them by 22 introducing a reliable, efficient and non-intrusive transport 23 forwarding over the overlayed path segment(s). 25 Status of this Memo 27 This Internet-Draft is submitted to IETF in full conformance with the 28 provisions of BCP 78 and BCP 79. 30 Internet-Drafts are working documents of the Internet Engineering 31 Task Force (IETF), its areas, and its working groups. Note that 32 other groups may also distribute working documents as 33 Internet-Drafts. 35 Internet-Drafts are draft documents valid for a maximum of six months 36 and may be updated, replaced, or obsoleted by other documents at any 37 time. It is inappropriate to use Internet-Drafts as reference 38 material or to cite them other than as "work in progress." 40 The list of current Internet-Drafts can be accessed at 41 http://www.ietf.org/1id-abstracts.html 43 The list of Internet-Draft Shadow Directories can be accessed at 44 http://www.ietf.org/shadow.html 46 Copyright and License Notice 48 Copyright (c) 2018 IETF Trust and the persons identified as the 49 document authors. All rights reserved. 51 This document is subject to BCP 78 and the IETF Trust's Legal 52 Provisions Relating to IETF Documents 53 (http://trustee.ietf.org/license-info) in effect on the date of 54 publication of this document. Please review these documents 55 carefully, as they describe your rights and restrictions with respect 56 to this document. Code Components extracted from this document must 57 include Simplified BSD License text as described in Section 4.e of 58 the Trust Legal Provisions and are provided without warranty as 59 described in the Simplified BSD License. 61 Table of Contents 63 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 64 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . 3 65 3. Use Cases and Problems . . . . . . . . . . . . . . . . . . . . 3 66 3.1 Microburst in Long Haul Network . . . . . . . . . . . . . . 4 67 3.2 Non-congestive Loss in WiFi Accessed Campus Overlay . . . . 6 68 3.3 Higher Reliability and Low Latency for Interactive 69 Application . . . . . . . . . . . . . . . . . . . . . . . . 8 70 4. Features to be Considered for OPSF (Overlayed Path Segment 71 Forwarding) . . . . . . . . . . . . . . . . . . . . . . . . . . 8 72 5. Security Considerations . . . . . . . . . . . . . . . . . . . . 10 73 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 10 74 7. References . . . . . . . . . . . . . . . . . . . . . . . . . . 10 75 7.1 Normative References . . . . . . . . . . . . . . . . . . . 10 76 7.2 Informative References . . . . . . . . . . . . . . . . . . 10 77 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 11 79 1. Introduction 81 Overlay tunnels are widely deployed for various networks, including 82 long haul WAN interconnection, enterprise wireless access networks, 83 etc. End to end connection are normally broken into multiple path 84 segments for different purposes, for instance, selecting a better 85 overlay path over the WAN or deliver the packets over the 86 heterogenous networks like enterprise access and core networks. 88 TCP-like transport layer provides end to end flow control and 89 congestion control for path reliability and high throughput. Such an 90 approach has the problems of slow congestion responding and non- 91 congestive loss misinterpretation at the sender and does not achieve 92 the optimal performance in certain cases. 94 Some of the problems have been well known over years. With new 95 technologies are emerging like NFV (Network Function Virtualization) 96 and various flexible overlay protocols, forwarding over the specific 97 overlayed path segment(s) can be considered to be enhanced by 98 providing a reliable and non-intrusive transport to improve the 99 throughput to solve those problems. 101 This document illustrates the problems in some use cases and tries to 102 inspire more about whether and how to solve them by introducing a 103 reliable, efficient and non-intrusive transport forwarding for the 104 overlayed path segment(s). 106 2. Terminology 108 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 109 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 110 "OPTIONAL" in this document are to be interpreted as described in BCP 111 14 [RFC2119] [RFC8174] when, and only when, they appear in all 112 capitals, as shown here. 114 OPSF: Overlayed Path Segment Forwarding 116 3. Use Cases and Problems 118 The following subsections presents use cases from different scenarios 119 using overlay tunnels with a common need of higher performance and 120 reliable overlayed path segment in best effort networks. 122 3.1 Microburst in Long Haul Network 124 Internet is a huge sized network of networks. The interconnections of 125 end devices using this global network are normally provided by ISPs 126 (Internet Service Provider). This ISP provided huge network is 127 considered as traditional Internet. CSPs (Cloud Service Provider) are 128 connecting their data centers using Internet or self-constructed 129 networks/links. This expands Internet's infrastructure and together 130 with the original ISP's infrastructure, forms the Internet underlay. 132 NFV further makes it easier to dynamically provision a new virtual 133 node as a work load in a cloud for CPU/storage intensive functions. 134 With the aid of various mechanisms such as kernel bypassing and 135 Virtual IO, forwarding based on virtual node is becoming more and 136 more effective. The interconnections among the purposely positioned 137 virtual nodes and/or the existing nodes with vitalization functions 138 potentially form "the Second Plane" or overlay of Internet. It is 139 called the Cloud-Internet Overlay Network (CION) in this document. 141 CION makes use of overlay technologies to direct the traffic going to 142 the specific path regardless the underlying physical topology to 143 achieve better service delivery. Figure 1 shows an emerging multi- 144 segment overlay over large geographic distances. It purposely creates 145 or selects overlay nodes (ON) from clouds/Internet. Segment here is 146 the virtual hop between two ONs. By directing the traffic to be 147 forwarded along those virtual nodes rather than the default path, 148 better delivery in terms of throughput and delay can be achieved. 149 When a large number of potential virtual nodes are available, there 150 is a high chance that a better path could be found [CRONets]. 152 _____________ 153 / domain 1 \ 154 / \ 155 ___/ -------------\ 156 / \ 157 PoP1 ->--ON1 \ 158 | | ON4------>-- PoP2 159 | | ON2 ___|__/ 160 \__|_ |->| _____ / | 161 | \|__|__ / \ / | 162 | | | \____/ \__/ | 163 \|/ | | _____ | 164 | | | ___/ \ | 165 | | \|/ / \_____ | 166 | | | / domain 2 \ /|\ 167 | | | | ON3 | | 168 | | | \ |->| | | 169 | | | \_____|__|_______/ | 170 | /|\ | | \|/ | 171 | | | | | | 172 | | | /|\ | | 173 +--------------------------------------------------+ 174 | | | | | | | Internet | 175 | o--o o---o->---o o---o->--o--o underlay | 176 +--------------------------------------------------+ 178 Figure 1. Cloud-Internet Overlay Network (CION) 180 Microburst is an unexpected data bursts within a very small time 181 window (probably in micro-seconds). Some research shows microbursts 182 happen even for underutilized link [BurstyAna]. The short spikes 183 caused by microburst result in higher jitter and sometimes packet 184 loss in a network. Such loss may trigger the congestion control like 185 reducing the sending rate at the TCP sender as it exhibits the normal 186 pattern of congestion loss in terms of duplicate acknowledgements 187 and/or RTT increases. As microburst is extremely short, the packet 188 loss caused by it is non-persistent and rather random. Therefore it 189 does not necessarily require the sender to reduce its sending rate. 190 Invoking the congestion control at the sender may unnecessarily make 191 the average sending rate low and degrades the throughput in long haul 192 CION. In addition, long haul transmission may take hundreds of 193 milliseconds. The packet loss response at the sender to microburst 194 over the long haul transmission is not timely. Sender's reaction does 195 not really respond to the current instantaneous path situation. 197 Overlay nodes in the middle can potentially offer new possibilities, 198 e.g. retransmission over ONs, to better response to microbursts. Such 199 enhancement can be enabled based on the individual overlayed path 200 segment rather than on the entire end to end path to improve the 201 response time and performance from the packet loss/re-order caused by 202 microburst. Such enhancement should avoid racing with higher layer 203 transport protocols. 205 3.2 Non-congestive Loss in WiFi Accessed Campus Overlay 207 Different path segments have different characteristics. The 208 probabilities of packet loss over every and each segments have a 209 large variance. The non-congestive packet loss usually occurs in some 210 specific overlayed path segments. End to end TCP-like transport 211 protocols do not take this factor into careful account. It assumes 212 that packet loss for any reason is almost evenly distributed across 213 the entire path, and adjusts the sender to accommodate the packet 214 loss of the bottleneck segment. This results in non-optimal sending 215 rate in some cases. 217 Figure 2 shows the WiFi accessed enterprise campus. AP connects to 218 its edge switch normally using Cat5/5e twisted-pair cable which 219 typically provides less than 10G bandwidth. The data packets are 220 tunneled using various overlay mechanisms, like VXLAN [RFC7348], LISP 221 [RFC6830] or CAPWAP [RFC5415]. Two edge switches use another overlay 222 segment over campus core network to deliver the packets which 223 provides more functions like policy enforcement and mobility 224 enhancement. This overlay is usually over fiber which provides higher 225 bandwidth. 227 _____________________________ 228 +-----+ /| |\ +-----+ 229 |edge1| | | | | |edge2| 230 +-----+ \|____________________________|/ +-----+ 231 _ _ 232 /_\ fiber /_\ 233 | | | | 234 | | | | 235 | | Cat5/5e | | Cat5/5e 236 | | cable | | cable 237 |_| |_| 238 \_/ \_/ 239 +-----+ +-----+ 240 | AP1 | | AP2 | 241 +-----+ +-----+ 242 | | 243 | WiFi access | WiFi access 244 | | 245 +----+ +----+ 246 |STA1| |STA2| 247 +----+ +----+ 249 Figure 2. WiFi accessed Campus Overlay Network 251 Cat5/5e cables, especially UTP (Unshielded twisted pair), are 252 susceptible to distance, interference, and bundling. The environment 253 and the way they are deployed cause drastic changes in random loss 254 rate. The overlay tunnel running over it will have more transmission 255 unreliability than the overlay running on the fiber. Current 256 transport layer is not able to identify such specific problematic 257 segment and simply leaves it for the end to end congestion control to 258 handle it so that the sender may be kept at a lower sending rate and 259 the throughput is not optimal. 261 In addition to the uplink of the AP, the non-congestive packet loss 262 generated by the wireless access link itself accounts for the largest 263 proportion in the end-to-end path. Wifi access is affected by fades, 264 interference, attenuation and corruption. Non-congestive loss is 265 common. Its link layer has mechanisms to do the packet recovery. 266 However the number of local link layer retransmission is usually 267 based on the empirical value or the static configuration. When the 268 value is not properly chosen, the TCP sender can be unnecessarily 269 exposed by the random packet loss and reduce the sending rate. It is 270 hard to make the link layer frame recovery work in concert with the 271 current end to end transport layer. 273 3.3 Higher Reliability and Low Latency for Interactive Application 275 Mobile gaming and VoIP like application normally can not tolerate a 276 retransmission even over a path segment. When two divergent overlay 277 segments are available like shown in figure 3 for path from ON1 to 278 ON2, purposely duplicating packets over two segments provides more 279 reliability. Two disjoint segments can usually be obtained by 280 measuring to find segments with very low mathematical correlation in 281 latency change. When the number of overlay nodes is large, it is easy 282 to find such disjoint segments. Random node or memory failure may 283 also benefit from the duplicating packets over disjoint segments. 285 ON-A 286 +----------o------------------+ 287 | | 288 | | 289 A -----o ON1 ON2o----- B 290 | | 291 +-----------------------o-----+ 292 ON-B 294 Figure 3. Multiple Overlayed Path Segments for Higher Reliability 296 4. Features to be Considered for OPSF (Overlayed Path Segment 297 Forwarding) 299 The diagram shown in Figure 4 illustrates a typical scenario with an 300 overlayed path segment. Transport layer provide the end to end flow 301 control between two end host. When an overlayed path segment exists 302 or is purposely created between two overlay nodes, an enhanced 303 forwarding over that segment can potentially solve some problems of 304 end to end transport performance issues and at the same time provides 305 more reliability and flexibility to traffic path. 307 ON=overlay node 308 UN=underlay node 310 +-----------+ +-----------+ 311 |Application|<-------------- end-to-end -------------> |Application| 312 +-----------+ +-----------+ 313 | Transport |<-------------- end-to-end -------------> | Transport | 314 | | | | 315 +-----------+ overlayed +-----------+ 316 | | +----+ path segment+----+ | | 317 | | | ON |<----------->| ON | | | 318 | Network | +----+ +----+ +----+ +----+ | Network | 319 | |<->| UN |<->| UN |<->| UN |<-->| UN |<--->| | 320 +-----------+ +----+ +----+ +----+ +----+ +-----------+ 321 End Host End Host 323 Figure 4. A Simple Overlayed Path Segment Forwarding Usage Scenario 325 Features need more investigations include, 327 - Enhancement for the overlayed path segment, like retransmission, 328 FEC(forward error correction), duplicating packet over the segments, 329 lightweighted congestion control, etc. When the segment is a small 330 portion of the whole end to end path, the retransmission over it has 331 more benefit. Retransmission over the path segment has to be 332 carefully designed to avoid the racing condition with the upper 333 layer. The segment enabled retransmission may measure the segment RTT 334 by itself to determine the appropriate retransmission attempts. On 335 the other hand, the upper layers including the applications can 336 indicate the credit as the safe band time that allows for the 337 overlayed path segment to do the retransmission. At the same time, 338 the persistent congestion caused packet loss should be exposed to the 339 upper transport layer, so that the sender's congestion control can 340 work properly. The timing of activation of the enhancement scheme, 341 parameters such as the threshold setting of retransmission are worthy 342 of further determination. 344 - Measurement based path selection for better performance, backup or 345 load balancing. Overlay nodes have to be continuously monitored in 346 order to find one or more appropriate overlayed paths. Such 347 measurement can be in-band or out of band of data packets. When more 348 than one overlayed segment with the same ingress and egress are used, 349 it has to be determined how the traffic are split and merged. 351 5. Security Considerations 353 TBD 355 6. IANA Considerations 357 No IANA action is required. 359 7. References 361 7.1 Normative References 363 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 364 Requirement Levels", BCP 14, RFC 2119, March 1997. 366 7.2 Informative References 368 [RFC4122] Leach, P., Mealling, M., and R. Salz, "A Universally 369 Unique IDentifier (UUID) URN Namespace", RFC 4122, July 370 2005. 372 [RFC5415] Calhoun, P., Ed., Montemurro, M., Ed., and D. Stanley, 373 Ed., "Control And Provisioning of Wireless Access Points 374 (CAPWAP) Protocol Specification", RFC 5415, March 2009. 376 [BurstyAna] Chung S., Agrawal D., Kim M., Hong J., and Park K. 377 "Analysis of bursty packet loss characteristics on 378 underutilized links using SNMP", IEEE/IFIP E2EMON, 2004. 380 [CRONets] Cai, C. X., Le, F., Sun, X., Xie, G. G., Jamjoom, H., and 381 Campbell, R. H. CRONets: Cloud-Routed Overlay Networks. In 382 36th International Conference on Distributed Computing 383 Systems (ICDCS) (2016), IEEE, pp. 67--77. 385 [RFC6830] Farinacci, D., Fuller, V., Meyer, D., and D. Lewis, "The 386 Locator/ID Separation Protocol (LISP)", RFC 6830, January 387 2013. 389 [RFC7348] Mahalingam, M., Dutt, D., Duda, K., Agarwal, P., Kreeger, 390 L., Sridhar, T., Bursell, M., and C. Wright, "Virtual 391 eXtensible Local Area Network (VXLAN): A Framework for 392 Overlaying Virtualized Layer 2 Networks over Layer 3 393 Networks", RFC 7348, August 2014. 395 Authors' Addresses 397 Yizhou Li 398 Huawei Technologies 399 101 Software Avenue, 400 Nanjing 210012 401 China 403 Phone: +86-25-56624584 404 EMail: liyizhou@huawei.com 406 Xingwang Zhou 407 Huawei Technologies 408 Huawei Technologies 409 101 Software Avenue, 410 Nanjing 210012 411 China 413 EMail: zhouxingwang@huawei.com