Network Working Group T. Dreibholz Internet-Draft SimulaMet Intended status: Informational S. Ferlin Expires: June 5, 2020 Simula Research Laboratory O. Alay A. Elmokashfi I. Livadariu SimulaMet X. Zhou Hainan University December 3, 2019 MPTCP Experiences in the NorNet Testbed draft-dreibholz-mptcp-nornet-experience-05 Abstract This document collects some experiences of Multi-Path TCP (MPTCP) evaluations in the NorNet testbed. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on June 5, 2020. Copyright Notice Copyright (c) 2019 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect Dreibholz, et al. Expires June 5, 2020 [Page 1] Internet-Draft MPTCP Experiences in the NorNet Testbed December 2019 to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 1.1. Abbreviations . . . . . . . . . . . . . . . . . . . . . . 2 1.2. Multi-Path TCP . . . . . . . . . . . . . . . . . . . . . 2 1.3. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2. Testbed Evaluation . . . . . . . . . . . . . . . . . . . . . 3 2.1. The NorNet Testbed . . . . . . . . . . . . . . . . . . . 3 2.2. Multi-Path TCP in NorNet . . . . . . . . . . . . . . . . 3 2.3. NetPerfMeter . . . . . . . . . . . . . . . . . . . . . . 3 3. Research Results and Work in Progress . . . . . . . . . . . . 4 4. Security Considerations . . . . . . . . . . . . . . . . . . . 5 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 5 6. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 5 7. References . . . . . . . . . . . . . . . . . . . . . . . . . 6 7.1. Normative References . . . . . . . . . . . . . . . . . . 6 7.2. Informative References . . . . . . . . . . . . . . . . . 6 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 11 1. Introduction 1.1. Abbreviations o TCP: Transmission Control Protocol o MPTCP: Multi-Path TCP 1.2. Multi-Path TCP The Multi-Path TCP (MPTCP) extension for the Transmission Control Protocol (TCP) has been defined in [3], [4], [1], [2], [5]. There are also detailed introductions provided for example by [8], [9] as well as lots of further information material on [6]. MPTCP is therefore not introduced in more detail here. 1.3. Scope The scope of this document is to collect some experiences with the usage of MPTCP in the NorNet testbed, a large-scale Internet testbed for multi-homed systems. Dreibholz, et al. Expires June 5, 2020 [Page 2] Internet-Draft MPTCP Experiences in the NorNet Testbed December 2019 2. Testbed Evaluation 2.1. The NorNet Testbed The NorNet testbed (https://www.nntb.no) introduced in [11], [15], [10] is a programmable testbed platform with focus on the evaluation of multi-homed systems. It consists of programmable nodes that are distributed all over Norway as well as further locations in other countries. NotNet is built and operated by the Simula Research Laboratory and financed by Forskningsraedet (the Research Council of Norway) through their INFRASTRUKTUR program (project number 208798/ F50). NorNet has two main components: NorNet Core and NorNet Edge. NorNet Core consists of currently 20 programmable sites (https://www.nntb.no/pub/nornet-configuration/NorNetCore-Sites.html), most of them multi-homed to several network providers. Details can be found in [12], [14], [11] and [16]. NorNet Edge consists of several hundreds of smaller nodes connected to all mobile broadband providers in Norway (http://robustenett.no/map), details can be found in [13]. Together, these two components offer a unique platform for experimental networking research. NorNet is made available to the international networking research community. Further details on NorNet can be found on the NorNet website [17]. 2.2. Multi-Path TCP in NorNet The NorNet nodes are Linux-based and therefore run the Linux MPTCP implementation by Universite catholique de Louvain-la-Neuve (see [18]). With support by all NorNet Core nodes (circa 100 nodes at currently 20 sites) and most NorNet Edge nodes (several hundreds), NorNet probably provides the world's largest MPTCP experimentation platform. 2.3. NetPerfMeter NetPerfMeter [19], [7] is a network performance meter for the UDP, TCP, MPTCP, SCTP and DCCP transport protocols over IPv4 and IPv6. It simultaneously transmits bidirectional flows to an endpoint and measures the resulting flow bandwidths and QoS. The results are written as vector and scalar files. NetPerfMeter is provided in NorNet to allow for transport protocol comparisons and transport protocol performance evaluations. Dreibholz, et al. Expires June 5, 2020 [Page 3] Internet-Draft MPTCP Experiences in the NorNet Testbed December 2019 3. Research Results and Work in Progress So far, work has been done on evaluating MPTCP in the real-world Internet. The following list is a short overview of current research: o [20] examines the different properties of 3G paths (UMTS as well as CDMA2000) in NorNet Edge. These results provide an overview of the QoS characteristics that can be expected by multi-path transport protocols (and particularly by MPTCP) on different paths. o [21] analyzes the impact of buffer bloat on MPTCP connections over mobile broadband (2G, 3G) and WLAN paths. Furthermore, it proposes and evaluates Multi-Path Transport Bufferbloat Mitigation (MPT-BM), a bufferbloat mitigation algorithm to improve MPTCP performance in buffer-bloated wireless networks. o [22] examines the performance benefits of multi-path transport with MPTCP under heterogeneous wireless networks. It furthermore introduces and evaluates the Dynamic Relative Path Scoring (DRePaS) algorithm that optimizes the path management in such setups. o [23] examines the performance benefits of the state-of-the-art Linux MPTCP implementation in a large-scale NorNet setup, covering sites in multiple countries on different continents. It particularly also shortly introduces the NetPerfMeter measurement tool that is used to perform the measurements, and particularly its extension for MPTCP. o [24] examines how the IPv4/IPv6 identity duality can be utilized with MPTCP in order to improve performance even in case of only a single ISP connection. o [25] provides an introduction to the NetPerfMeter tool for MPTCP experiments. o [26] presents MPTCP's architecture and multi-path congestion control algorithm concepts. Then, it examines three test scenarios in the NorNet testbed, particularly highlighting the performance difference between using uncoupled and coupled congestion controls in multi-homed, real-world Internet setups. o [27] introduces a scanning infrastructure to search for MPTCP- capable hosts in the Internet. In a study, it used the top-1M Alexa servers to test the platform and gain insight about server support for MPTCP. Dreibholz, et al. Expires June 5, 2020 [Page 4] Internet-Draft MPTCP Experiences in the NorNet Testbed December 2019 o [28] proposes a practical shared bottleneck detection (SBD) algorithm for MPTCP, namely MPTCP-SBD. Through extensive emulations, it is shown that MPTCP-SBD outperforms all currently deployed MPTCP coupled congestion controls by accurately detecting bottlenecks resulting in throughput gains in the absence of shared bottlenecks, while remaining fair to TCP in shared bottlenecks scenarios. o [29] proposes a send-window BLocking ESTimation scheduler, BLEST, which aims to minimise head-of-line-blocking in heterogeneous networks. BLEST increases the potential for capacity aggregation by reducing the number of spurious retransmissions. o [30] examines path management strategies for MPTCP in real-world, multi-homed Internet setups. Particular goal is to highlight the performance impact of different path management and congestion control settings in such realistic scenarios. o [31] analyses the actual buffer size requirements for MPTCP in heterogeneous, multi-homed Internet setups. Particular goal is to provide some guidelines on buffer size requirements for achieving a reasonable throughput, while on the other hand not wasting resources. o [32] examines the relationship of buffer size with throughput and congestion control algorithms, based on the statistical predictive modelling method. o [33] examines the application of MPTCP for real-time video streaming for future Mobile Edge Computing (MEC) scenarios. 4. Security Considerations Security considerations on MPTCP are described in [3]. 5. IANA Considerations This document has no actions for IANA. 6. Acknowledgments The authors would like to thank Hakim Adhari, Fa Fu, Feng Zhou and Kun Wang for discussions and support. Dreibholz, et al. Expires June 5, 2020 [Page 5] Internet-Draft MPTCP Experiences in the NorNet Testbed December 2019 7. References 7.1. Normative References [1] Raiciu, C., Handley, M., and D. Wischik, "Coupled Congestion Control for Multipath Transport Protocols", RFC 6356, DOI 10.17487/RFC6356, October 2011, . [2] Ford, A., Raiciu, C., Handley, M., and O. Bonaventure, "TCP Extensions for Multipath Operation with Multiple Addresses", RFC 6824, DOI 10.17487/RFC6824, January 2013, . 7.2. Informative References [3] Bagnulo, M., "Threat Analysis for TCP Extensions for Multipath Operation with Multiple Addresses", RFC 6181, DOI 10.17487/RFC6181, March 2011, . [4] Ford, A., Raiciu, C., Handley, M., Barre, S., and J. Iyengar, "Architectural Guidelines for Multipath TCP Development", RFC 6182, DOI 10.17487/RFC6182, March 2011, . [5] Scharf, M. and A. Ford, "Multipath TCP (MPTCP) Application Interface Considerations", RFC 6897, DOI 10.17487/RFC6897, March 2013, . [6] Dreibholz, T., "Thomas Dreibholz's MPTCP Page", Online: https://www.uni-due.de/~be0001/mptcp/, 2016, . [7] Dreibholz, T., "NetPerfMeter - A TCP/MPTCP/UDP/SCTP/DCCP Network Performance Meter Tool", Online: https://www.uni-due.de/~be0001/netperfmeter/, 2016, . [8] Becke, M., "Revisiting the IETF Multipath Extensions on Transport Layer", November 2014, . Dreibholz, et al. Expires June 5, 2020 [Page 6] Internet-Draft MPTCP Experiences in the NorNet Testbed December 2019 [9] Ruengeler, I., "SCTP - Evaluating, Improving and Extending the Protocol for Broader Deployment", December 2009, . [10] Dreibholz, T., "The NorNet Testbed for Multi-Homed Systems - Introduction and Status", Invited Talk at Princeton University, Department of Computer Science, May 2014, . [11] Dreibholz, T., "NorNet at NICTA - An Introduction to the NorNet Testbed", Invited Talk at National Information Communications Technology Australia (NICTA), January 2016, . [12] Gran, E., Dreibholz, T., and A. Kvalbein, "NorNet Core - A Multi-Homed Research Testbed", Computer Networks, Special Issue on Future Internet Testbeds Volume 61, Pages 75-87, ISSN 1389-1286, DOI 10.1016/j.bjp.2013.12.035, March 2014, . [13] Kvalbein, A., Baltrunas, D., Evensen, K., Xiang, J., Elmokashfi, A., and S. Ferlin, "The NorNet Edge Platform for Mobile Broadband Measurements", Computer Networks, Special Issue on Future Internet Testbeds Volume 61, Pages 88-101, ISSN 1389-1286, DOI 10.1016/j.bjp.2013.12.036, March 2014, . [14] Dreibholz, T. and E. Gran, "Design and Implementation of the NorNet Core Research Testbed for Multi-Homed Systems", Proceedings of the 3nd International Workshop on Protocols and Applications with Multi-Homing Support (PAMS) Pages 1094-1100, ISBN 978-0-7695-4952-1, DOI 10.1109/WAINA.2013.71, March 2013, . Dreibholz, et al. Expires June 5, 2020 [Page 7] Internet-Draft MPTCP Experiences in the NorNet Testbed December 2019 [15] Dreibholz, T., "Multi-Path Transport - From Simulations to Real-World Internet Measurements", Keynote Talk at the Universidad de Castilla-La Mancha, Instituto de Investigacion Informatica de Albacete, February 2017, . [16] Dreibholz, T., "An Experiment Tutorial for the NorNet Core Testbed at the the Universidad de Castilla-La Mancha", Tutorial at the Universidad de Castilla-La Mancha, Instituto de Investigacion Informatica de Albacete, February 2017, . [17] Dreibholz, T., "NorNet - A Real-World, Large-Scale Multi- Homing Testbed", Online: https://www.nntb.no/, 2017, . [18] Paasch, C. and S. Barre, "Multipath TCP implementation in the Linux kernel", Online: http://www.multipath-tcp.org, 2016, . [19] Dreibholz, T., Becke, M., Adhari, H., and E. Rathgeb, "Evaluation of A New Multipath Congestion Control Scheme using the NetPerfMeter Tool-Chain", Proceedings of the 19th IEEE International Conference on Software, Telecommunications and Computer Networks (SoftCOM) Pages 1-6, ISBN 978-953-290-027-9, September 2011, . [20] Ferlin, S., Dreibholz, T., Alay, Oe., and A. Kvalbein, "Measuring the QoS Characteristics of Operational 3G Mobile Broadband Networks", Proceedings of the 4th International Workshop on Protocols and Applications with Multi-Homing Support (PAMS) Pages 753-758, ISBN 978-1-4799-2652-7, DOI 10.1109/WAINA.2014.123, May 2014, . [21] Ferlin, S., Dreibholz, T., and Oe. Alay, "Tackling the Challenge of Bufferbloat in Multi-Path Transport over Heterogeneous Wireless Networks", Proceedings of the IEEE/ ACM International Symposium on Quality of Service (IWQoS) Pages 123-128, ISBN 978-1-4799-4852-9, DOI 10.1109/IWQoS.2014.6914310, May 2014, . Dreibholz, et al. Expires June 5, 2020 [Page 8] Internet-Draft MPTCP Experiences in the NorNet Testbed December 2019 [22] Ferlin, S., Dreibholz, T., and Oe. Alay, "Multi-Path Transport over Heterogeneous Wireless Networks: Does it really pay off?", Proceedings of the IEEE Global Communications Conference (GLOBECOM) Pages 5005-5011, ISBN 978-1-4799-3512-3, December 2014, . [23] Dreibholz, T., Zhou, X., and F. Fa, "Multi-Path TCP in Real-World Setups - An Evaluation in the NorNet Core Testbed", 5th International Workshop on Protocols and Applications with Multi-Homing Support (PAMS), March 2015, . [24] Livadariu, I., Ferlin, S., Alay, Oe., Dreibholz, T., Dhamdhere, A., and A. Elmokashfi, "Leveraging the IPv4/ IPv6 Identity Duality by using Multi-Path Transport", Proceedings of the 18th IEEE Global Internet Symposium (GI), April 2015, . [25] Dreibholz, T., "NetPerfMeter: A Network Performance Metering Tool", Multipath TCP Blog, September 2015, . [26] Fa, F., Zhou, X., Dreibholz, T., Wang, K., Zhou, F., and Q. Gan, "Performance Comparison of Congestion Control Strategies for Multi-Path TCP in the NorNet Testbed", Proceedings of the 4th IEEE/CIC International Conference on Communications in China (ICCC) Pages 607-612, ISBN 978-1-5090-0243-6, November 2015, . [27] Mehani, O., Holz, R., Ferlin, S., and R. Boreli, "An Early Look at Multipath TCP Deployment in the Wild", Proceedings of the 6th International Workshop on Hot Topics in Planet- Scale Measurement ISBN 978-1-4503-3534-8, DOI 10.1145/2798087.2798088, 2015, . Dreibholz, et al. Expires June 5, 2020 [Page 9] Internet-Draft MPTCP Experiences in the NorNet Testbed December 2019 [28] Ferlin, S., Alay, Oe., Hayes, D., Dreibholz, T., and M. Welzl, "Revisiting Congestion Control for Multipath TCP with Shared Bottleneck Detection", Proceedings of the 35th IEEE International Conference on Computer Communications (INFOCOM), April 2016. [29] Ferlin, S., Alay, Oe., Mehani, O., and R. Boreli, "BLEST: Blocking Estimation-Based MPTCP Scheduler for Heterogeneous Networks", Proceedings of the IFIP Networking Conference Pages 431-439, DOI 10.1109/IFIPNetworking.2016.7497206, May 2016, . [30] Wang, K., Dreibholz, T., Zhou, X., Fu, F., Tan, Y., Cheng, X., and Q. Tan, "On the Path Management of Multi-Path TCP in Internet Scenarios based on the NorNet Testbed", Proceedings of the IEEE International Conference on Advanced Information Networking and Applications (AINA) ISBN 978-1-5090-6028-3, March 2017, . [31] Zhou, F., Dreibholz, T., Zhou, X., Fu, F., Tan, Y., and Q. Gan, "The Performance Impact of Buffer Sizes for Multi- Path TCP in Internet Setups", Proceedings of the IEEE International Conference on Advanced Information Networking and Applications (AINA) ISBN 978-1-5090-6028-3, March 2017, . [32] Tan, Q., Yang, X., Zhao, L., Zhou, X., and T. Dreibholz, "A Statistic Procedure to Find Formulae for Buffer Size in MPTCP", Proceedings of the 3rd IEEE Advanced Information Technology, Electronic and Automation Control Conference (IAEAC) Pages 900-907, ISBN 978-1-5386-4509-3, October 2018, . [33] Luo, Y., Zhou, X., Dreibholz, T., and H. Kuang, "A Real- Time Video Streaming System over IPv6+MPTCP Technology", Proceedings of the 1st International Workshop on Recent Advances for Multi-Clouds and Mobile Edge Computing (M2EC) in conjunction with the 33rd International Conference on Advanced Information Networking and Applications (AINA), March 2019, . Dreibholz, et al. Expires June 5, 2020 [Page 10] Internet-Draft MPTCP Experiences in the NorNet Testbed December 2019 Authors' Addresses Thomas Dreibholz Simula Metropolitan Centre for Digital Engineering Pilestredet 52 0167 Oslo, Oslo Norway Phone: +47-6782-8200 Fax: +47-6782-8201 Email: dreibh@simula.no URI: https://www.simula.no/people/dreibh Simone Ferlin Simula Research Laboratory, Networks Department Martin Linges vei 17 1364 Fornebu, Akershus Norway Phone: +47-4072-0702 Fax: +47-4072-0702 Email: ferlin@simula.no URI: https://www.simula.no/people/ferlin Ozgu Alay Simula Metropolitan Centre for Digital Engineering Pilestredet 52 0167 Oslo, Oslo Norway Phone: +47-9848-5362 Fax: +47-9848-5362 Email: ozgu@simula.no URI: https://www.simula.no/people/ozgu Ahmed Elmokashfi Simula Metropolitan Centre for Digital Engineering Pilestredet 52 0167 Oslo, Oslo Norway Phone: +47-474-52-315 Fax: +47-6782-8201 Email: ahmed@simula.no URI: https://www.simula.no/people/ahmed Dreibholz, et al. Expires June 5, 2020 [Page 11] Internet-Draft MPTCP Experiences in the NorNet Testbed December 2019 Ioana Alexandrina Livadariu Simula Metropolitan Centre for Digital Engineering Pilestredet 52 0167 Oslo, Oslo Norway Phone: +47-453-98-686 Fax: +47-6782-8201 Email: ioana@simula.no URI: https://www.simula.no/people/ioana Xing Zhou Hainan University, College of Information Science and Technology Renmin Avenue 58 570228 Haikou, Hainan China Phone: +86-898-66279141 Email: zhouxing@hainu.edu.cn URI: http://www.hainu.edu.cn/stm/xinxi/2011330/10409758.shtml Dreibholz, et al. Expires June 5, 2020 [Page 12]