idnits 2.17.1 draft-ietf-mptcp-api-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (November 29, 2010) is 4895 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- ** Obsolete normative reference: RFC 793 (ref. '1') (Obsoleted by RFC 9293) == Outdated reference: A later version (-05) exists of draft-ietf-mptcp-architecture-02 == Outdated reference: A later version (-12) exists of draft-ietf-mptcp-multiaddressed-02 == Outdated reference: A later version (-08) exists of draft-ietf-mptcp-threat-04 == Outdated reference: A later version (-07) exists of draft-ietf-mptcp-congestion-00 == Outdated reference: A later version (-17) exists of draft-ietf-shim6-multihome-shim-api-13 == Outdated reference: A later version (-32) exists of draft-ietf-tsvwg-sctpsocket-23 == Outdated reference: A later version (-15) exists of draft-ietf-mif-problem-statement-04 == Outdated reference: A later version (-12) exists of draft-ietf-mif-current-practices-01 Summary: 1 error (**), 0 flaws (~~), 9 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force M. Scharf 3 Internet-Draft Alcatel-Lucent Bell Labs 4 Intended status: Informational A. Ford 5 Expires: June 2, 2011 Roke Manor Research 6 November 29, 2010 8 MPTCP Application Interface Considerations 9 draft-ietf-mptcp-api-00 11 Abstract 13 Multipath TCP (MPTCP) adds the capability of using multiple paths to 14 a regular TCP session. Even though it is designed to be totally 15 backward compatible to applications, the data transport differs 16 compared to regular TCP, and there are several additional degrees of 17 freedom that applications may wish to exploit. This document 18 summarizes the impact that MPTCP may have on applications, such as 19 changes in performance. Furthermore, it discusses compatibility 20 issues of MPTCP in combination with non-MPTCP-aware applications. 21 Finally, the document describes a basic application interface for 22 MPTCP-aware applications that provides access to multipath address 23 information and a level of control equivalent to regular TCP. 25 Status of This Memo 27 This Internet-Draft is submitted in full conformance with the 28 provisions of BCP 78 and BCP 79. 30 Internet-Drafts are working documents of the Internet Engineering 31 Task Force (IETF). Note that other groups may also distribute 32 working documents as Internet-Drafts. The list of current Internet- 33 Drafts is at http://datatracker.ietf.org/drafts/current/. 35 Internet-Drafts are draft documents valid for a maximum of six months 36 and may be updated, replaced, or obsoleted by other documents at any 37 time. It is inappropriate to use Internet-Drafts as reference 38 material or to cite them other than as "work in progress." 40 This Internet-Draft will expire on June 2, 2011. 42 Copyright Notice 44 Copyright (c) 2010 IETF Trust and the persons identified as the 45 document authors. All rights reserved. 47 This document is subject to BCP 78 and the IETF Trust's Legal 48 Provisions Relating to IETF Documents 49 (http://trustee.ietf.org/license-info) in effect on the date of 50 publication of this document. Please review these documents 51 carefully, as they describe your rights and restrictions with respect 52 to this document. Code Components extracted from this document must 53 include Simplified BSD License text as described in Section 4.e of 54 the Trust Legal Provisions and are provided without warranty as 55 described in the Simplified BSD License. 57 Table of Contents 59 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 60 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 61 3. Comparison of MPTCP and Regular TCP . . . . . . . . . . . . . 5 62 3.1. Performance Impact . . . . . . . . . . . . . . . . . . . . 6 63 3.1.1. Throughput . . . . . . . . . . . . . . . . . . . . . . 6 64 3.1.2. Delay . . . . . . . . . . . . . . . . . . . . . . . . 6 65 3.1.3. Resilience . . . . . . . . . . . . . . . . . . . . . . 7 66 3.2. Potential Problems . . . . . . . . . . . . . . . . . . . . 7 67 3.2.1. Impact of Middleboxes . . . . . . . . . . . . . . . . 7 68 3.2.2. Outdated Implicit Assumptions . . . . . . . . . . . . 8 69 3.2.3. Security Implications . . . . . . . . . . . . . . . . 8 70 4. Operation of MPTCP with Legacy Applications . . . . . . . . . 8 71 4.1. Overview of the MPTCP Network Stack . . . . . . . . . . . 8 72 4.2. Address Issues . . . . . . . . . . . . . . . . . . . . . . 9 73 4.2.1. Specification of Addresses by Applications . . . . . . 9 74 4.2.2. Querying of Addresses by Applications . . . . . . . . 10 75 4.3. Socket Option Issues . . . . . . . . . . . . . . . . . . . 11 76 4.3.1. General Guideline . . . . . . . . . . . . . . . . . . 11 77 4.3.2. Disabling of the Nagle Algorithm . . . . . . . . . . . 11 78 4.3.3. Buffer Sizing . . . . . . . . . . . . . . . . . . . . 11 79 4.3.4. Other Socket Options . . . . . . . . . . . . . . . . . 12 80 4.4. Default Enabling of MPTCP . . . . . . . . . . . . . . . . 12 81 4.5. Summary of Advices to Application Developers . . . . . . . 12 82 5. Basic API for MPTCP-aware Applications . . . . . . . . . . . . 13 83 5.1. Design Considerations . . . . . . . . . . . . . . . . . . 13 84 5.2. Requirements on the Basic MPTCP API . . . . . . . . . . . 13 85 5.3. Sockets Interface Extensions by the Basic MPTCP API . . . 15 86 5.3.1. Overview . . . . . . . . . . . . . . . . . . . . . . . 15 87 5.3.2. Enabling and Disabling of MPTCP . . . . . . . . . . . 16 88 5.3.3. Binding MPTCP to Specified Addresses . . . . . . . . . 17 89 5.3.4. Querying the MPTCP Subflow Addresses . . . . . . . . . 17 90 5.3.5. Getting a Unique Connection Identifier . . . . . . . . 18 91 6. Other Compatibility Issues . . . . . . . . . . . . . . . . . . 18 92 6.1. Usage of the SCTP Socket API . . . . . . . . . . . . . . . 18 93 6.2. Incompatibilities with other Multihoming Solutions . . . . 18 94 6.3. Interactions with DNS . . . . . . . . . . . . . . . . . . 19 95 7. Security Considerations . . . . . . . . . . . . . . . . . . . 19 96 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 19 97 9. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . 19 98 10. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 20 99 11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 20 100 11.1. Normative References . . . . . . . . . . . . . . . . . . . 20 101 11.2. Informative References . . . . . . . . . . . . . . . . . . 21 102 Appendix A. Requirements on a Future Advanced MPTCP API . . . . . 21 103 A.1. Design Considerations . . . . . . . . . . . . . . . . . . 21 104 A.2. MPTCP Usage Scenarios and Application Requirements . . . . 22 105 A.3. Potential Requirements on an Advanced MPTCP API . . . . . 24 106 Appendix B. Change History of the Document . . . . . . . . . . . 25 108 1. Introduction 110 Multipath TCP adds the capability of using multiple paths to a 111 regular TCP session [1]. The motivations for this extension include 112 increasing throughput, overall resource utilisation, and resilience 113 to network failure, and these motivations are discussed, along with 114 high-level design decisions, as part of the Multipath TCP 115 architecture [4]. The MPTCP protocol [5] offers the same reliable, 116 in-order, byte-stream transport as TCP, and is designed to be 117 backward compatible with both applications and the network layer. It 118 requires support inside the network stack of both endpoints. 120 This document first presents the impacts that MPTCP may have on 121 applications, such as performance changes compared to regular TCP. 122 Second, it defines the interoperation of MPTCP and applications that 123 are unaware of the multipath transport. MPTCP is designed to be 124 usable without any application changes, but some compatibility issues 125 have to be taken into account. Third, this memo specifies a basic 126 Application Programming Interface (API) for MPTCP-aware applications. 127 The API presented here is an extension to the regular TCP API to 128 allow an MPTCP-aware application the equivalent level of control and 129 access to information of an MPTCP connection that would be possible 130 with the standard TCP API on a regular TCP connection. 132 An advanced API for MPTCP is outside the scope of this document. 133 Such an advanced API could offer a more fine-grained control over 134 multipath transport functions and policies. The appendix includes a 135 brief, non-compulsory list of potential features of such an advanced 136 API. 138 The de facto standard API for TCP/IP applications is the "sockets" 139 interface. This document provides an abstract definition of MPTCP- 140 specific extensions to this interface. These are operations that can 141 be used by an application to get or set additional MPTCP-specific 142 information on a socket, in order to provide an equivalent level of 143 information and control over MPTCP as exists for an application using 144 regular TCP. It is up to the applications, high-level programming 145 languages, or libraries to decide whether to use these optional 146 extensions. For instance, an application may want to turn on or off 147 the MPTCP mechanism for certain data transfers, or limit its use to 148 certain interfaces. The abstract specification is in line with the 149 Posix standard [8] as much as possible. 151 There are also various related extensions of the sockets interface: 152 [12] specifies sockets API extensions for a multihoming shim layer. 153 The API enables interactions between applications and the multihoming 154 shim layer for advanced locator management and for access to 155 information about failure detection and path exploration. 157 Experimental extensions to the sockets API are also defined for the 158 Host Identity Protocol (HIP) [13] in order to manage the bindings of 159 identifiers and locator. Further related API extensions exist for 160 IPv6 [10], Mobile IP [11], and SCTP [14]. There can be interactions 161 or incompatibilities of these APIs with MPTCP, which are discussed 162 later in this document. 164 Some network stack implementations, specially on mobile devices, have 165 centralized connection managers or other higher-level APIs to solve 166 multi-interface issues, as surveyed in [16]. Their interaction with 167 MPTCP is outside the scope of this note. 169 The target readers of this document are application developers whose 170 software may benefit significantly from MPTCP. This document also 171 provides the necessary information for developers of MPTCP to 172 implement the API in a TCP/IP network stack. 174 2. Terminology 176 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 177 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 178 document are to be interpreted as described in [3]. 180 This document uses the MPTCP terminology introduced in [5]. 182 Concerning the API towards applications, the following terms are 183 distinguished: 185 o Legacy API: The interface towards TCP that is currently used by 186 applications. This document explains the impact of MPTCP for such 187 applications, as well as resulting issues. 189 o Basic API: A simple extension of TCP's interface for applications 190 that are aware of MPTCP. This document abstractly describes this 191 interface, which provides access to multipath address information 192 and a level of control equivalent to regular TCP. 194 o Advanced API: An API that offers more fine-grained control over 195 the MPTCP behaviour. Its detailed specification is outside scope 196 of this document. 198 3. Comparison of MPTCP and Regular TCP 200 This section discusses the impact that the use of MPTCP will have on 201 applications, in comparison to what may be expected from the use of 202 regular TCP. 204 3.1. Performance Impact 206 One of the key goals of adding multipath capability to TCP is to 207 improve the performance of a transport connection by load 208 distribution over separate subflows across potentially disjoint 209 paths. Furthermore, it is an explicit goal of MPTCP that it should 210 not provide a worse performing connection that would have existed 211 through the use of single-path TCP. A corresponding congestion 212 control algorithm is described in [7]. The following sections 213 summarize the performance impact of MPTCP as seen by an application. 215 3.1.1. Throughput 217 The most obvious performance improvement that will be gained with the 218 use of MPTCP is an increase in throughput, since MPTCP will pool more 219 than one path (where available) between two endpoints. This will 220 provide greater bandwidth for an application. If there are shared 221 bottlenecks between the flows, then the congestion control algorithms 222 will ensure that load is evenly spread amongst regular and multipath 223 TCP sessions, so that no end user receives worse performance than 224 single-path TCP. 226 This performance increase additionally means that an MPTCP session 227 could achieve throughput that is greater than the capacity of a 228 single interface on the device. If any applications make assumptions 229 about interfaces due to throughput (or vice versa), they must take 230 this into account (although an MPTCP implementation must always 231 respect an application's request for a particular interface). 233 Furthermore, the flexibility of MPTCP to add and remove subflows as 234 paths change availability could lead to a greater variation, and more 235 frequent change, in connection bandwidth. Applications that adapt to 236 available bandwidth (such as video and audio streaming) may need to 237 adjust some of their assumptions to most effectively take this into 238 account. 240 The transport of MPTCP signaling information results in a small 241 overhead. If multiple subflows share a same bottleneck, this 242 overhead slightly reduces the capacity that is available for data 243 transport. Yet, this potential reduction of throughput will be 244 neglectible in many usage scenarios, and the protocol contains 245 optimisations in its design so that this overhead is minimal. 247 3.1.2. Delay 249 If the delays on the constituent subflows of an MPTCP connection 250 differ, the jitter perceivable to an application may appear higher as 251 the data is spread across the subflows. Although MPTCP will ensure 252 in-order delivery to the application, the application must be able to 253 cope with the data delivery being burstier than may be usual with 254 single-path TCP. Since burstiness is commonplace on the Internet 255 today, it is unlikely that applications will suffer from such an 256 impact on the traffic profile, but application authors may wish to 257 consider this in future development. 259 In addition, applications that make round trip time (RTT) estimates 260 at the application level may have some issues. Whilst the average 261 delay calculated will be accurate, whether this is useful for an 262 application will depend on what it requires this information for. If 263 a new application wishes to derive such information, it should 264 consider how multiple subflows may affect its measurements, and thus 265 how it may wish to respond. In such a case, an application may wish 266 to express its scheduling preferences, as described later in this 267 document. 269 3.1.3. Resilience 271 The use of multiple subflows simultaneously means that, if one should 272 fail, all traffic will move to the remaining subflow(s), and 273 additionally any lost packets can be retransmitted on these subflows. 275 Subflow failure may be caused by issues within the network, which an 276 application would be unaware of, or interface failure on the node. 277 An application may, under certain circumstances, be in a position to 278 be aware of such failure (e.g. by radio signal strength, or simply an 279 interface enabled flag), and so must not make assumptions of an MPTCP 280 flow's stablity based on this. An MPTCP implementation must never 281 override an application's request for a given interface, however, so 282 the cases where this issue may be applicable are limited. 284 3.2. Potential Problems 286 3.2.1. Impact of Middleboxes 288 MPTCP has been designed in order to pass through the majority of 289 middleboxes. Empirical evidence suggests that new TCP options can 290 successfully be used on most paths in the Internet. Nevertheless 291 some middleboxes may still refuse to pass MPTCP messages due to the 292 presence of TCP options, or they may strip TCP options. If this is 293 the case, MPTCP should fall back to regular TCP. Although this will 294 not create a problem for the application (its communication will be 295 set up either way), there may be additional (and indeed, user- 296 perceivable) delay while the first handshake fails. A detailed 297 discussion of the various fallback mechanisms, for failures occurring 298 at different points in the connection, is presented in [5]. 300 There may also be middleboxes that transparently change the length of 301 content. If such middleboxes are present, MPTCP's reassembly of the 302 byte stream in the receiver is difficult. Still, MPTCP can detect 303 such middleboxes and then fall back to regular TCP. An overview of 304 the impact of middleboxes is presented in [4] and MPTCP's mechanisms 305 to work around these are presented and discussed in [5]. 307 MPTCP can also have other unexpected implications. For instance, 308 intrusion detection systems could be triggered. A full analysis of 309 MPTCP's impact on such middleboxes is for further study after 310 deployment experiments. 312 3.2.2. Outdated Implicit Assumptions 314 In regular TCP, there is a one-to-one mapping of the socket interface 315 to a flow through a network. Since MPTCP can make use of multiple 316 subflows, applications cannot implicitly rely on this one-to-one 317 mapping any more. Applications that require the transport along a 318 single path can disable the use of MPTCP as described later in this 319 document. Examples include monitoring tools that want to measure the 320 available bandwidth on a path, or routing protocols such as BGP that 321 require the use of a specific link. 323 3.2.3. Security Implications 325 The support for multiple IP addresses within one MPTCP connection can 326 result in additional security vulnerabilities, such as possibilities 327 for attackers to hijack connections. The protocol design of MPTCP 328 minimizes this risk. An attacker on one of the paths can cause harm, 329 but this is hardly an additional security risk compared to single- 330 path TCP, which is vulnerable to man-in-the-middle attacks, too. A 331 detailed thread analysis of MPTCP is published in [6]. 333 4. Operation of MPTCP with Legacy Applications 335 4.1. Overview of the MPTCP Network Stack 337 MPTCP is an extension of TCP, but it is designed to be backward 338 compatible for legacy applications. TCP interacts with other parts 339 of the network stack by different interfaces. The de facto standard 340 API between TCP and applications is the sockets interface. The 341 position of MPTCP in the protocol stack can be illustrated in 342 Figure 1. 344 +-------------------------------+ 345 | Application | 346 +-------------------------------+ 347 ^ | 348 ~~~~~~~~~~~|~Socket Interface|~~~~~~~~~~~ 349 | v 350 +-------------------------------+ 351 | MPTCP | 352 + - - - - - - - + - - - - - - - + 353 | Subflow (TCP) | Subflow (TCP) | 354 +-------------------------------+ 355 | IP | IP | 356 +-------------------------------+ 358 Figure 1: MPTCP protocol stack 360 In general, MPTCP can affect all interfaces that make assumptions 361 about the coupling of a TCP connection to a single IP address and TCP 362 port pair, to one sockets endpoint, to one network interface, or to a 363 given path through the network. 365 This means that there are two classes of applications: 367 o Legacy applications: These applications are unaware of MPTCP and 368 use the existing API towards TCP without any changes. This is the 369 default case. 371 o MPTCP-aware applications: These applications indicate support for 372 an enhance MPTCP interface. This document specified a minimum set 373 of API extensions for such applications. 375 In the following, it is discussed to which extent MPTCP affects 376 legacy applications using the existing sockets API. The existing 377 sockets API implies that applications deal with data structures that 378 store, amongst others, the IP addresses and TCP port numbers of a TCP 379 connection. A design objective of MPTCP is that legacy applications 380 can continue to use the established sockets API without any changes. 381 However, in MPTCP there is a one-to-many mapping between the socket 382 endpoint and the subflows. This has several subtle implications for 383 legacy applications using sockets API functions. 385 4.2. Address Issues 387 4.2.1. Specification of Addresses by Applications 389 During binding, an application can either select a specific address, 390 or bind to INADDR_ANY. Furthermore, on some systems other socket 391 options (e.g., SO_BINDTODEVICE) can be used to bind to a specific 392 interface. If an application uses a specific address or binds to a 393 specific interface, then MPTCP MUST respect this and not interfere in 394 the application's choices. The binding to a specific address or 395 interface implies that the application is not aware of MPTCP and will 396 disable the use of MPTCP on this connection. An application that 397 wishes to bind to a specific set of addresses with MPTCP must use 398 multipath-aware calls to achieve this (as described in 399 Section 5.3.3). 401 If an application binds to INADDR_ANY, it is assumed that the 402 application does not care which addresses to use locally. In this 403 case, a local policy MAY allow MPTCP to automatically set up multiple 404 subflows on such a connection. 406 The basic sockets API of MPTCP-aware applications allows to express 407 further preferences in an MPTCP-compatible way (e.g. bind to a subset 408 of interfaces only). 410 4.2.2. Querying of Addresses by Applications 412 Applications can use the getpeername() or getsockname() functions in 413 order to retrieve the IP address of the peer or of the local socket. 414 These functions can be used for various purposes, including security 415 mechanisms, geo-location, or interface checks. The socket API was 416 designed with an assumption that a socket is using just one address, 417 and since this address is visible to the application, the application 418 may assume that the information provided by the functions is the same 419 during the lifetime of a connection. However, in MPTCP, unlike in 420 TCP, there is a one-to-many mapping of a connection to subflows, and 421 subflows can be added and removed while the connections continues to 422 exist. Therefore, MPTCP cannot expose addresses by getpeername() or 423 getsockname() that are both valid and constant during the 424 connection's lifetime. 426 This problem is addressed as follows: If used by a legacy 427 application, the MPTCP stack MUST always return the addresses of the 428 first subflow of an MPTCP connection, in all circumstances, even if 429 that particular subflow is no longer in use. 431 As this address may not be valid any more if the first subflow is 432 closed, the MPTCP stack MAY close the whole MPTCP connection if the 433 first subflow is closed (i.e. fate sharing between the initial 434 subflow and the MPTCP connection as a whole). Whether to close the 435 whole MPTCP connection by default SHOULD be controlled by a local 436 policy. Further experiments are needed to investigate its 437 implications. 439 The functions getpeername() and getsockname() SHOULD also always 440 return the addresses of the first subflow if the socket is used by an 441 MPTCP-aware application, in order to be consistent with MPTCP-unaware 442 applications, and, e. g., also with SCTP. Instead of getpeername() 443 or getsockname(), MPTCP-aware applications can use new API calls, 444 documented later, in order to retrieve the full list of address pairs 445 for the subflows in use. 447 4.3. Socket Option Issues 449 4.3.1. General Guideline 451 The existing sockets API includes options that modify the behavior of 452 sockets and their underlying communications protocols. Various 453 socket options exist on socket, TCP, and IP level. The value of an 454 option can usually be set by the setsockopt() system function. The 455 getsockopt() function gets information. In general, the existing 456 sockets interface functions cannot configure each MPTCP subflow 457 individually. In order to be backward compatible, existing APIs 458 therefore SHOULD apply to all subflows within one connection, as far 459 as possible. 461 4.3.2. Disabling of the Nagle Algorithm 463 One commonly used TCP socket option (TCP_NODELAY) disables the Nagle 464 algorithm as described in [2]. This option is also specified in the 465 Posix standard [8]. Applications can use this option in combination 466 with MPTCP exactly in the same way. It then SHOULD disable the Nagle 467 algorithm for the MPTCP connection, i.e., all subflows. 469 In addition, the MPTCP protocol instance MAY use a different path 470 scheduler algorithm if TCP_NODELAY is present. For instance, it 471 could use an algorithm that is optimized for latency-sensitive 472 traffic. Specific algorithms are outside the scope of this document. 474 4.3.3. Buffer Sizing 476 Applications can explicitly configure send and receive buffer sizes 477 by the sockets API (SO_SNDBUF, SO_RCVBUF). These socket options can 478 also be used in combination with MPTCP and then affect the buffer 479 size of the MPTCP connection. However, when defining buffer sizes, 480 application programmers should take into account that the transport 481 over several subflows requires a certain amount of buffer for 482 resequencing in the receiver. MPTCP may also require more storage 483 space in the sender, in particular, if retransmissions are sent over 484 more than one path. In addition, very small send buffers may prevent 485 MPTCP from efficiently scheduling data over different subflows. 486 Therefore, it does not make sense to use MPTCP in combination with 487 small send or receive buffers. 489 An MPTCP implementation MAY set a lower bound for send and receive 490 buffers and treat a small buffer size request as an implicit request 491 not to use MPTCP. 493 4.3.4. Other Socket Options 495 Some network stacks also provide other implementation-specific socket 496 options or interfaces that affect TCP's behavior. If a network stack 497 supports MPTCP, it must be ensured that these options do not 498 interfere. 500 4.4. Default Enabling of MPTCP 502 It is up to a local policy at the end system whether a network stack 503 should automatically enable MPTCP for sockets even if there is no 504 explicit sign of MPTCP awareness of the corresponding application. 505 Such a choice may be under the control of the user through system 506 preferences. 508 The enabling of MPTCP, either by application or by system defaults, 509 does not necessarily mean that MPTCP will always be used. Both 510 endpoints must support MPTCP, and there must be multiple addresses at 511 at least one endpoint, for MPTCP to be used. Even if those 512 requirements are met, however, MPTCP may not be immediately used on a 513 connection. It may make sense for multiple paths to be brought into 514 operation only after a given period of time, or if the connection is 515 saturated. 517 4.5. Summary of Advices to Application Developers 519 o Using the default MPTCP configuration: Like TCP, MPTCP is designed 520 to be efficient and robust in the default configuration. 521 Application developers should not explicitly configure TCP (or 522 MPTCP) features unless this is really needed. 524 o Socker buffet dimensioning: Multipath transport requires larger 525 buffers in the receiver for resequencing, as already explained. 526 Applications should use reasonably buffer sizes (such as the 527 operating system default values) in order to fully benefit from 528 MPTCP. A full discussion of buffer sizing issues is given in [5]. 530 o Facilitating stack-internal heuristics: The path management and 531 data scheduling by MPTCP is realized by stack-internal algorithms 532 that may implicitly try to self-optimize their behavior according 533 to assumed application needs. For instance, an MPTCP 534 implementation may use heuristics to determine whether an 535 application requires delay-sensitive or bulk data transport, using 536 for instance port numbers, the TCP_NODELAY socket options, or the 537 application's read/write patterns as input parameters. An 538 application developer can facilitate the operation of such 539 heuristics by avoiding atypical interface use cases. For 540 instance, for long bulk data transfers, it does neither make sense 541 to enable the TCP_NODELAY socket option, nor is it reasonable to 542 use many small subsequent socket "send()" calls with small amounts 543 of data only. 545 5. Basic API for MPTCP-aware Applications 547 5.1. Design Considerations 549 While applications can use MPTCP with the unmodified sockets API, 550 multipath transport results in many degrees of freedom. MPTCP 551 manages the data transport over different subflows automatically. By 552 default, this is transparent to the application, but an application 553 could use an additional API to interface with the MPTCP layer and to 554 control important aspects of the MPTCP implementation's behaviour. 556 This document describes a basic MPTCP API. The API contains a 557 minimum set of functions that provide an equivalent level of control 558 and information as exists for regular TCP. It maintains backward 559 compatibility with legacy applications. 561 An advanced MPTCP API is outside the scope of this document. The 562 basic API does not allow a sender or a receiver to express 563 preferences about the management of paths or the scheduling of data, 564 even if this can have a significant performance impact and if an 565 MPTCP implementation could benefit from additional guidance by 566 applications. A list of potential further API extensions is provided 567 in the appendix. The specification of such an advanced API is for 568 further study and may partly be implementation-specific. 570 MPTCP mainly affects the sending of data. Therefore, the basic API 571 only affects the sender side of a data transfer. A receiver may also 572 have preferences about data transfer choices, and it may have 573 performance requirements, too. A receiver may also have preferences 574 about data transfer choices, and it may have performance 575 requirements, too. Yet, the configuration of such preferences is 576 outside of the scope of the basic API. 578 5.2. Requirements on the Basic MPTCP API 580 Because of the importance of the sockets interface there are several 581 fundamental design objectives for the basic interface between MPTCP 582 and applications: 584 o Consistency with existing sockets APIs must be maintained as far 585 as possible. In order to support the large base of applications 586 using the original API, a legacy application must be able to 587 continue to use standard socket interface functions when run on a 588 system supporting MPTCP. Also, MPTCP-aware applications should be 589 able to access the socket without any major changes. 591 o Sockets API extensions must be minimized and independent of an 592 implementation. 594 o The interface should both handle IPv4 and IPv6. 596 The following is a list of the core requirements for the basic API: 598 REQ1: Turn on/off MPTCP: An application should be able to request to 599 turn on or turn off the usage of MPTCP. This means that an 600 application should be able to explicitly request the use of 601 MPTCP if this is possible. Applications should also be able 602 to request not to enable MPTCP and to use regular TCP 603 transport instead. This can be implicit in many cases, since 604 MPTCP must disabled by the use of binding to a specific 605 address. MPTCP may also be enabled if an application uses a 606 dedicated multipath address family (such as AF_MULTIPATH, 607 [9]). 609 REQ2: An application should be able to restrict MPTCP to binding to 610 a given set of addresses. 612 REQ3: An application should be able obtain information on the 613 addresses used by the MPTCP subflows. 615 REQ4: An application should be able to extract a unique identifier 616 for the connection (per endpoint). 618 The first requirement is the most important one, since some 619 applications could benefit a lot from MPTCP, but there are also cases 620 in which it hardly makes sense. The existing sockets API provides 621 similar mechanisms to enable or disable advanced TCP features. The 622 second requirement corresponds to the binding of addresses with the 623 bind() socket call, or, e.g., explicit device bindings with a 624 SO_BINDTODEVICE option. The third requirement ensures that there is 625 an equivalent to getpeername() or getsockname() that is able to deal 626 with more than one subflow. Finally, it should be possible for the 627 application to retrieve a unique connection identifier (local to the 628 endpoint on which it is running) for the MPTCP connection. This is 629 equivalent to using the (address, port) pair for a connection 630 identifier in single-path TCP, which is no longer static in MPTCP. 632 An application can continue to use getpeername() or getsockname() in 633 addition to the basic MPTCP API. In that case, both functions return 634 the corresponding addresses of the first subflow, as already 635 explained. 637 5.3. Sockets Interface Extensions by the Basic MPTCP API 639 5.3.1. Overview 641 The abstract, basic MPTCP API consists of a set of new values that 642 are associated with an MPTCP socket. Such values may be used for 643 changing properties of an MPTCP connection, or retrieving 644 information. These values could be accessed by new symbols on 645 existing calls such as setsockopt() and getsockopt(), or could be 646 implemented as entirely new function calls. This implementation 647 decision is out of scope for this document. The following list 648 presents symbolic names for these MPTCP socket settings. 650 o TCP_MULTIPATH_ENABLE: Enable/disable MPTCP 652 o TCP_MULTIPATH_ADD: Bind MPTCP to a set of given local addresses, 653 or add a new local address to an existing MPTCP connection 655 o TCP_MUTLIPATH_REMOVE: Remove a local address from a MPTCP 656 connection 658 o TCP_MULTIPATH_SUBFLOWS: Get the pairs of addresses currently used 659 by the MPTCP subflows 661 o TCP_MULTIPATH_CONNID: Get the local connection identifier for this 662 MPTCP connection 664 Table Table 1 shows a list of the abstract socket operations for the 665 basic configuration of MPTCP. The first column gives the symbolic 666 name of the operation. The second and third columns indicate whether 667 the operation provides values to be read ("Get") or takes values to 668 configure ("Set"). The fourth column lists the type of data 669 associated with this operation. 671 +------------------------+-----+-----+----------------------------+ 672 | Name | Get | Set | Data type | 673 +------------------------+-----+-----+----------------------------+ 674 | TCP_MULTIPATH_ENABLE | o | o | boolean | 675 | TCP_MULTIPATH_ADD | | o | list of addresses | 676 | TCP_MULTIPATH_REMOVE | | o | list of addresses | 677 | TCP_MULTIPATH_SUBFLOWS | o | | list of pairs of addresses | 678 | TCP_MULTIPATH_CONNID | o | | 32-bit integer | 679 +------------------------+-----+-----+----------------------------+ 681 Table 1: MPTCP Socket Operations 683 There are restrictions when these new socket operations can be used: 685 o TCP_MULTIPATH_ENABLE: This value SHOULD only be set before the 686 establishment of a TCP connection. Its value SHOULD only be read 687 after the establishment of a connection. 689 o TCP_MULTIPATH_ADD: This operation can be both applied before 690 connection setup or during a connection. If used before, it 691 controls the local addresses that an MPTCP connection can use. In 692 the latter case, it allows MPTCP to use an additional local 693 address, if there has been a restriction before connection setup. 695 o TCP_MULTIPATH_REMOVE: This operation only has meaning after 696 connection setup. 698 o TCP_MULTIPATH_SUBFLOWS: This value is read-only and SHOULD only be 699 used after connection setup. 701 o TCP_MULTIPATH_CONNID: This value is read-only and SHOULD only be 702 used after connection setup. 704 5.3.2. Enabling and Disabling of MPTCP 706 An application can explicitly indicate multipath capability by 707 setting TCP_MULTIPATH_ENABLE to a value larger than 0. In this case, 708 the MPTCP implementation SHOULD try to negitiate MPTCP for that 709 connection. Note that multipath transport will not necessarily be 710 enabled, as it requires multiple addresses and support in the other 711 end-system and potentially also on middleboxes. 713 An application can disable MPTCP setting TCP_MUTLIPATH_ENABLE to a 714 value of 0. In that case, MPTCP MUST NOT be used on that connection. 716 After connection establishment, an application can get the value of 717 TCP_MULTIPATH_ENABLE. A value of 0 then means lack of MPTCP support. 718 Any value equal to or larger than 1 means that MPTCP is supported. 720 As alternative to setting an explicit value, an application could 721 also use a new, separate address family called AF_MULTIPATH [9]. 722 This separate address family can be used to exchange multiple 723 addresses between an application and the standard sockets API, and 724 additionally acts as an explicit indication that an application is 725 MPTCP-aware, i.e., that it can deal with the semantic changes of the 726 sockets API, in particular concerning getpeername() and 727 getsockname(). The usage of AF_MULTIPATH is also more flexible with 728 respect to multipath transport, either IPv4 or IPv6, or both in 729 parallel [9]. 731 5.3.3. Binding MPTCP to Specified Addresses 733 Before connection establishment, an application can use 734 TCP_MULTIPATH_ADD socket option to indicate a set of local IP 735 addresses that MPTCP may bind to. By extension, this operation will 736 also control the list of addresses that can be advertised to the peer 737 via MPTCP signalling. 739 This operation can also be used to modify the address list in use 740 during the lifetime of an MPTCP connection. In this case, it is used 741 to indicate a set of additional local addresses that the MPTCP 742 connection can make use of, and which can be signalled to the peer. 743 It should be noted that this signal is only a hint, and an MPTCP 744 implementation MAY only use a subset of the addresses. 746 The TCP_MULTIPATH_REMOVE operation can be used to remove a (set of) 747 local addresses from an MPTCP connection. MPTCP MUST close any 748 corresponding subflows (i.e. those using the local address that is no 749 longer present), and signal the removal of the address to the peer. 750 If alternative paths are available using the supplied address list 751 but MPTCP is not currently using them, an MPTCP implementation SHOULD 752 establish alternative subflows before undertaking the address 753 removal. 755 It should be remembered that these operations SHOULD support both 756 IPv4 and IPv6 addresses, potentially in the same call. 758 5.3.4. Querying the MPTCP Subflow Addresses 760 An application can get a list of the addresses used by the currently 761 established subflows by means of the read-only TCP_MULTIPATH_SUBFLOWS 762 operation. The return value is a list of pairs of IP addresses. In 763 one pair, the first address refers to the local IP address, and the 764 second one to the remote IP address used by the subflow. The list 765 MUST only include established subflows. Each address pair MUST be 766 either a pair of IPv4 addresses or a pair of IPv6 addresses. 768 5.3.5. Getting a Unique Connection Identifier 770 An application that wants a unique identifier for the connection, 771 analogous to an (address, port) pair in regular TCP, can query the 772 TCP_MULTIPATH_CONNID value to get a local connection identifier for 773 the MPTCP connection. 775 This is a 32-bit number, and SHOULD be the same as the local 776 connection identifier sent in the MPTCP handshake. 778 6. Other Compatibility Issues 780 6.1. Usage of the SCTP Socket API 782 For dealing with multi-homing, several socket API extensions have 783 been defined for SCTP [14]. As MPTCP realizes multipath transport 784 from and to multi-homed endsystems, some of these interface function 785 calls are actually applicable to MPTCP in a similar way. 787 The following functions that are defined for SCTP have similar 788 functionality to the MPTCP API extensions defined earlier: 790 o sctp_bindx() 792 o sctp_connectx() 794 o sctp_getladdrs() 796 o sctp_getpaddrs() 798 The syntax and semantics of these functions are described in [14]. 800 API developers MAY wish to integrate SCTP and MPTCP calls to provide 801 a consistent interface to the application. Yet, it must be 802 emphasized that the transport service provided by MPTCP is different 803 to SCTP, and this is why not all SCTP API functions can be mapped 804 directly to MPTCP. Furthermore, a network stack implementing MPTCP 805 does not necessarily support SCTP and its specific socket interface 806 extensions. This is why the basic API of MPTCP defines additional 807 socket options only, which are a backward compatible extension of 808 TCP's application interface. 810 6.2. Incompatibilities with other Multihoming Solutions 812 The use of MPTCP can interact with various related sockets API 813 extensions. The use of a multihoming shim layer conflicts with 814 multipath transport such as MPTCP or SCTP [12]. Care should be taken 815 for the usage not to confuse with the overlapping features of other 816 APIs: 818 o SHIM API [12]: This API specifies sockets API extensions for the 819 multihoming shim layer. 821 o HIP API [13]: The Host Identity Protocol (HIP) also results in a 822 new API. 824 o API for Mobile IPv6 [11]: For Mobile IPv6, a significantly 825 extended socket API exists as well. 827 In order to avoid any conflict, multiaddressed MPTCP SHOULD NOT be 828 enabled if a network stack uses SHIM6, HIP, or Mobile IPv6. 829 Furthermore, applications should not try to use both the MPTCP API 830 and another multihoming or mobility layer API. 832 It is possible, however, that some of the MPTCP functionality, such 833 as congestion control, could be used in a SHIM6 or HIP environment. 834 Such operation is outside the scope of this document. 836 6.3. Interactions with DNS 838 In multihomed or multiaddressed environments, there are various 839 issues that are not specific to MPTCP, but have to be considered, 840 too. These problems are summarized in [15]. 842 Specifically, there can be interactions with DNS. Whilst it is 843 expected that an application will iterate over the list of addresses 844 returned from a call such as getaddrinfo(), MPTCP itself MUST NOT 845 make any assumptions about multiple A or AAAA records from the same 846 DNS query referring to the same host, as it is possible that multiple 847 addresses refer to multiple servers for load balancing purposes. 849 TODO: Elaborate on DNS 851 7. Security Considerations 853 Will be added in a later version of this document. 855 8. IANA Considerations 857 No IANA considerations. 859 9. Conclusion 861 This document discusses MPTCP's application implications and 862 specifies a basic MPTCP API. For legacy applications, it is ensured 863 that the existing sockets API continues to work. MPTCP-aware 864 applications can use the basic MPTCP API that provides some control 865 over the transport layer equivalent to regular TCP. A more fine- 866 granular interaction between applications and MPTCP requires an 867 advanced MPTCP API, which is not specified in this document. 869 10. Acknowledgments 871 Authors sincerely thank to the following people for their helpful 872 comments to the document: Costin Raiciu, Philip Eardley 874 Michael Scharf is supported by the German-Lab project 875 (http://www.german-lab.de/) funded by the German Federal Ministry of 876 Education and Research (BMBF). Alan Ford is supported by Trilogy 877 (http://www.trilogy-project.org/), a research project (ICT-216372) 878 partially funded by the European Community under its Seventh 879 Framework Program. The views expressed here are those of the 880 author(s) only. The European Commission is not liable for any use 881 that may be made of the information in this document. 883 11. References 885 11.1. Normative References 887 [1] Postel, J., "Transmission Control Protocol", STD 7, RFC 793, 888 September 1981. 890 [2] Braden, R., "Requirements for Internet Hosts - Communication 891 Layers", STD 3, RFC 1122, October 1989. 893 [3] Bradner, S., "Key words for use in RFCs to Indicate Requirement 894 Levels", BCP 14, RFC 2119, March 1997. 896 [4] Ford, A., Raiciu, C., Handley, M., and J. Iyengar, 897 "Architectural Guidelines for Multipath TCP Development", 898 draft-ietf-mptcp-architecture-02 (work in progress), 899 October 2010. 901 [5] Ford, A., Raiciu, C., and M. Handley, "TCP Extensions for 902 Multipath Operation with Multiple Addresses", 903 draft-ietf-mptcp-multiaddressed-02 (work in progress), 904 October 2010. 906 [6] Bagnulo, M., "Threat Analysis for Multi-addressed/Multi-path 907 TCP", draft-ietf-mptcp-threat-04 (work in progress), 908 November 2010. 910 [7] Raiciu, C., Handley, M., and D. Wischik, "Coupled Multipath- 911 Aware Congestion Control", draft-ietf-mptcp-congestion-00 (work 912 in progress), July 2010. 914 [8] "IEEE Std. 1003.1-2008 Standard for Information Technology -- 915 Portable Operating System Interface (POSIX). Open Group 916 Technical Standard: Base Specifications, Issue 7, 2008.". 918 11.2. Informative References 920 [9] Sarolahti, P., "Multi-address Interface in the Socket API", 921 draft-sarolahti-mptcp-af-multipath-01 (work in progress), 922 March 2010. 924 [10] Stevens, W., Thomas, M., Nordmark, E., and T. Jinmei, "Advanced 925 Sockets Application Program Interface (API) for IPv6", 926 RFC 3542, May 2003. 928 [11] Chakrabarti, S. and E. Nordmark, "Extension to Sockets API for 929 Mobile IPv6", RFC 4584, July 2006. 931 [12] Komu, M., Bagnulo, M., Slavov, K., and S. Sugimoto, "Socket 932 Application Program Interface (API) for Multihoming Shim", 933 draft-ietf-shim6-multihome-shim-api-13 (work in progress), 934 February 2010. 936 [13] Komu, M. and T. Henderson, "Basic Socket Interface Extensions 937 for Host Identity Protocol (HIP)", draft-ietf-hip-native-api-12 938 (work in progress), January 2010. 940 [14] Stewart, R., Poon, K., Tuexen, M., Yasevich, V., and P. Lei, 941 "Sockets API Extensions for Stream Control Transmission 942 Protocol (SCTP)", draft-ietf-tsvwg-sctpsocket-23 (work in 943 progress), July 2010. 945 [15] Blanchet, M. and P. Seite, "Multiple Interfaces Problem 946 Statement", draft-ietf-mif-problem-statement-04 (work in 947 progress), May 2010. 949 [16] Wasserman, M. and P. Seite, "Current Practices for Multiple 950 Interface Hosts", draft-ietf-mif-current-practices-01 (work in 951 progress), June 2010. 953 Appendix A. Requirements on a Future Advanced MPTCP API 955 A.1. Design Considerations 957 Multipath transport results in many degrees of freedom. The basic 958 MPTCP API only defines a minimum set of the API extensions for the 959 interface between the MPTCP layer and applications, which does not 960 offer much control of the MPTCP implementation's behaviour. A 961 future, advanced API could address further features of MPTCP and 962 provide more control. 964 Applications that use TCP may have different requirements on the 965 transport layer. While developers have become used to the 966 characteristics of regular TCP, new opportunities created by MPTCP 967 could allow the service provided to be optimised further. An 968 advanced API could enable MPTCP-aware applications to specify 969 preferences and control certain aspects of the behavior, in addition 970 to the simple control provided by the basic interface. An advanced 971 API could also address aspects that are completely out-of-scope of 972 the basic API, for example, the question whether a receiving 973 application could influence the sending policy. 975 Furthermore, an advanced MPTCP API could be part of a new overall 976 interface between the network stack and applications that addresses 977 other issues as well, such as the split between identifiers and 978 locators. An API that does not use IP addresses (but, instead e.g. a 979 connectbyname() function) would be useful for numerous purposes, 980 independent of MPTCP. 982 This appendix documents a list of potential usage scenarios and 983 requirements for the advanded API. The specification and 984 implementation of a corresponding API is outside the scope of this 985 document. 987 A.2. MPTCP Usage Scenarios and Application Requirements 989 There are different MPTCP usage scenarios. An application that 990 wishes to transmit bulk data will want MPTCP to provide a high 991 throughput service immediately, through creating and maximising 992 utilisation of all available subflows. This is the default MPTCP use 993 case. 995 But at the other extreme, there are applications that are highly 996 interactive, but require only a small amount of throughput, and these 997 are optimally served by low latency and jitter stability. In such a 998 situation, it would be preferable for the traffic to use only the 999 lowest latency subflow (assuming it has sufficient capacity), maybe 1000 with one or two additional subflows for resilience and recovery 1001 purposes. The key challenge for such a strategy is that the delay on 1002 a path may fluctuate significantly and that just always selecting the 1003 path with the smallest delay might result in instability. 1005 The choice between bulk data transport and latency-sensitive 1006 transport affects the scheduler in terms of whether traffic should 1007 be, by default, sent on one subflow or across several ones. Even if 1008 the total bandwidth required is less than that available on an 1009 individual path, it is desirable to spread this load to reduce stress 1010 on potential bottlenecks, and this is why this method should be the 1011 default for bulk data transport. However, that may not be optimal 1012 for applications that require latency/jitter stability. 1014 In the case of the latter option, a further question arises: Should 1015 additional subflows be used whenever the primary subflow is 1016 overloaded, or only when the primary path fails (hot-standby)? In 1017 other words, is latency stability or bandwidth more important to the 1018 application? This results in two different options: Firstly, there 1019 is the single path which can overflow into an additional subflow; and 1020 secondly there is single-path with hot-standby, whereby an 1021 application may want an alternative backup subflow in order to 1022 improve resilience. In case that data delivery on the first subflow 1023 fails, the data transport could immediately be continued on the 1024 second subflow, which is idle otherwise. 1026 Yet another complication is introduced with the potential that MPTCP 1027 introduces for changes in available bandwidth as the number of 1028 available subflows changes. Such jitter in bandwidth may prove 1029 confusing for some applications such as video or audio streaming that 1030 dynamically adapt codecs based on available bandwidth. Such 1031 applications may prefer MPTCP to attempt to provide a consistent 1032 bandwidth as far as is possible, and avoid maximising the use of all 1033 subflows. 1035 A further, mostly orthogonal question is whether data should be 1036 duplicated over the different subflows, in particular if there is 1037 spare capacity. This could improve both the timeliness and 1038 reliability of data delivery. 1040 In summary, there are at least three possible performance objectives 1041 for multipath transport (not necessarily disjoint): 1043 1. High bandwidth 1045 2. Low latency and jitter stability 1047 3. High reliability 1049 In an advanced API, applications could provide high-level guidance to 1050 the MPTCP implementation concerning these performance requirements, 1051 for instance, which is considered to be the most important one. The 1052 MPTCP stack would then use internal mechanisms to fulfill this 1053 abstract indication of a desired service, as far as possible. This 1054 would both affect the assignment of data (including retransmissions) 1055 to existing subflows (e.g., 'use all in parallel', 'use as overflow', 1056 'hot standby', 'duplicate traffic') as well as the decisions when to 1057 set up additional subflows to which addresses. In both cases 1058 different policies can exist, which can be expected to be 1059 implementation-specific. 1061 Therefore, an advanced API could provide a mechanism how applications 1062 can specify their high-level requirements in an implementation- 1063 independent way. One possibility would be to select one "application 1064 profile" out of a number of choices that characterize typical 1065 applications. Yet, as applications today do not have to inform TCP 1066 about their communication requirements, it requires further studies 1067 whether such an approach would be realistic. 1069 Of course, independent of an advanced API, such functionality could 1070 also partly be achieved by MPTCP-internal heuristics that infer some 1071 application preferences e.g. from existing socket options, such as 1072 TCP_NODELAY. Whether this would be reliable, and indeed appropriate, 1073 is for further study, too. 1075 A.3. Potential Requirements on an Advanced MPTCP API 1077 The following is a list of potential requirements for an advanced 1078 MPTCP API beyond the features of the basic API. It is included here 1079 for information only: 1081 REQ5: An application should be able to establish MPTCP connections 1082 without using IP addresses as locators. 1084 REQ6: An application should be able obtain usage information and 1085 statistics about all subflows (e.g., ratio of traffic sent 1086 via this subflow). 1088 REQ7: An application should be able to request a change in the 1089 number of subflows in use, thus triggering removal or 1090 addition of subflows. An even finer control granularity 1091 would be a request for the establishment of a new subflow to 1092 a provided destination, or a request for the termination of a 1093 specified, existing subflow. 1095 REQ8: An application should be able to inform the MPTCP 1096 implementation about its high-level performance requirements, 1097 e.g., in form of a profile. 1099 REQ9: An application should be able to control the automatic 1100 establishment/termination of subflows. This would imply a 1101 selection among different heuristics of the path manager, 1102 e.g., 'try as soon as possible', 'wait until there is a bunch 1103 of data', etc. 1105 REQ10: An application should be able to set preferred subflows or 1106 subflow usage policies. This would result in a selection 1107 among different configurations of the multipath scheduler. 1109 REQ11: An application should be able to control the level of 1110 redundancy by telling whether segments should be sent on more 1111 than one path in parallel. 1113 An advanced API fulfilling these requirements would allow application 1114 developers to more specifically configure MPTCP. It could avoid 1115 suboptimal decisions of internal, implicit heuristics. However, it 1116 is unclear whether all of these requirements would have a significant 1117 benefit to applications, since they are going above and beyond what 1118 the existing API to regular TCP provides. 1120 Appendix B. Change History of the Document 1122 Changes compared to version 03: 1124 o Removal of explicit references to "socket options" and getsockopt/ 1125 setsockopt. 1127 o Change of TCP_MULTIPATH_BIND to TCP_MULTIPATH_ADD and 1128 TCP_MULTIPATH_REMOVE. 1130 o Mention of stability of bandwidth as another potential QoS 1131 parameter for the advanced API. 1133 o Address comments received from Philip Eardley: Explanation of the 1134 API terminology, more explicit statement concerning applications 1135 that bind to a specific address, and some smaller editorial fixes 1137 Changes compared to version 02: 1139 o Definition of the behavior of getpeername() and getsockname() when 1140 being called by an MPTCP-aware application. 1142 o Discussion of the possiblity that an MPTCP implementation could 1143 support the SCTP API, as far as it is applicable to MPTCP. 1145 o Various editorial fixes. 1147 Changes compared to version 01: 1149 o Second half of the document completely restructured 1151 o Separation between a basic API and an advanced API: The focus of 1152 the document is the basic API only; all text concerning a 1153 potential extended API is moved to the appendix 1155 o Several clarifications, e. g., concerning buffer sizeing and the 1156 use of different scheduling strategies triggered by TCP_NODELAY 1158 o Additional references 1160 Changes compared to version 00: 1162 o Distinction between legacy and MPTCP-aware applications 1164 o Guidance concerning default enabling, reaction to the shutdown of 1165 the first subflow, etc. 1167 o Reference to a potential use of AF_MULTIPATH 1169 o Additional references to related work 1171 Authors' Addresses 1173 Michael Scharf 1174 Alcatel-Lucent Bell Labs 1175 Lorenzstrasse 10 1176 70435 Stuttgart 1177 Germany 1179 EMail: michael.scharf@alcatel-lucent.com 1181 Alan Ford 1182 Roke Manor Research 1183 Old Salisbury Lane 1184 Romsey, Hampshire SO51 0ZN 1185 UK 1187 Phone: +44 1794 833 465 1188 EMail: alan.ford@roke.co.uk