idnits 2.17.1 draft-tuexen-tsvwg-sctp-multipath-09.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (October 3, 2014) is 3464 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Experimental ---------------------------------------------------------------------------- == Missing Reference: 'RFCXXXX' is mentioned on line 960, but not defined ** Obsolete normative reference: RFC 4960 (Obsoleted by RFC 9260) ** Obsolete normative reference: RFC 6096 (Obsoleted by RFC 9260) == Outdated reference: A later version (-16) exists of draft-ietf-tsvwg-sctp-failover-05 == Outdated reference: A later version (-27) exists of draft-dreibholz-tsvwg-sctpsocket-multipath-06 == Outdated reference: A later version (-27) exists of draft-dreibholz-tsvwg-sctpsocket-sqinfo-06 Summary: 2 errors (**), 0 flaws (~~), 5 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group P. Amer 3 Internet-Draft University of Delaware 4 Intended status: Experimental M. Becke 5 Expires: April 6, 2015 University of Duisburg-Essen 6 T. Dreibholz 7 Simula Research Laboratory 8 N. Ekiz 9 University of Delaware 10 J. Iyengar 11 Franklin and Marshall College 12 P. Natarajan 13 Cisco Systems 14 R. Stewart 15 Adara Networks 16 M. Tuexen 17 Muenster Univ. of Appl. Sciences 18 October 3, 2014 20 Load Sharing for the Stream Control Transmission Protocol (SCTP) 21 draft-tuexen-tsvwg-sctp-multipath-09.txt 23 Abstract 25 The Stream Control Transmission Protocol (SCTP) supports multi-homing 26 for providing network fault tolerance. However, mainly one path is 27 used for data transmission. Only timer-based retransmissions are 28 carried over other paths as well. 30 This document describes how multiple paths can be used simultaneously 31 for transmitting user messages. 33 Status of This Memo 35 This Internet-Draft is submitted in full conformance with the 36 provisions of BCP 78 and BCP 79. 38 Internet-Drafts are working documents of the Internet Engineering 39 Task Force (IETF). Note that other groups may also distribute 40 working documents as Internet-Drafts. The list of current Internet- 41 Drafts is at http://datatracker.ietf.org/drafts/current/. 43 Internet-Drafts are draft documents valid for a maximum of six months 44 and may be updated, replaced, or obsoleted by other documents at any 45 time. It is inappropriate to use Internet-Drafts as reference 46 material or to cite them other than as "work in progress." 48 This Internet-Draft will expire on April 6, 2015. 50 Copyright Notice 52 Copyright (c) 2014 IETF Trust and the persons identified as the 53 document authors. All rights reserved. 55 This document is subject to BCP 78 and the IETF Trust's Legal 56 Provisions Relating to IETF Documents 57 (http://trustee.ietf.org/license-info) in effect on the date of 58 publication of this document. Please review these documents 59 carefully, as they describe your rights and restrictions with respect 60 to this document. Code Components extracted from this document must 61 include Simplified BSD License text as described in Section 4.e of 62 the Trust Legal Provisions and are provided without warranty as 63 described in the Simplified BSD License. 65 Table of Contents 67 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 68 2. Conventions . . . . . . . . . . . . . . . . . . . . . . . . . 3 69 3. Load Sharing . . . . . . . . . . . . . . . . . . . . . . . . 3 70 3.1. Split Fast Retransmissions . . . . . . . . . . . . . . . 4 71 3.2. Appropriate Congestion Window Growth . . . . . . . . . . 4 72 3.3. Appropriate Delayed Acknowledgements . . . . . . . . . . 5 73 4. Non-Renegable SACK . . . . . . . . . . . . . . . . . . . . . 6 74 4.1. Negotiation . . . . . . . . . . . . . . . . . . . . . . . 6 75 4.2. The New Chunk Type: Non-Renegable SACK (NR-SACK) . . . . 6 76 4.3. An Illustrative Example . . . . . . . . . . . . . . . . . 11 77 4.4. Procedures . . . . . . . . . . . . . . . . . . . . . . . 15 78 4.4.1. Sending an NR-SACK chunk . . . . . . . . . . . . . . 15 79 4.4.2. Receiving an NR-SACK Chunk . . . . . . . . . . . . . 17 80 5. Buffer Blocking Mitigation . . . . . . . . . . . . . . . . . 18 81 5.1. Sender Buffer Splitting . . . . . . . . . . . . . . . . . 18 82 5.2. Receiver Buffer Splitting . . . . . . . . . . . . . . . . 18 83 5.3. Chunk Rescheduling . . . . . . . . . . . . . . . . . . . 18 84 5.4. Problems during Path Failure . . . . . . . . . . . . . . 19 85 5.4.1. Problem Description . . . . . . . . . . . . . . . . . 19 86 5.4.2. Solution: Potentially-failed Destination State . . . 19 87 5.5. Non-Renegable SACK . . . . . . . . . . . . . . . . . . . 20 88 5.5.1. Problem Description . . . . . . . . . . . . . . . . . 20 89 5.5.2. Solution: Non-Renegable SACKs . . . . . . . . . . . . 20 90 6. Handling of Shared Bottlenecks . . . . . . . . . . . . . . . 21 91 6.1. Introduction . . . . . . . . . . . . . . . . . . . . . . 21 92 6.2. Initial Values . . . . . . . . . . . . . . . . . . . . . 21 93 6.3. Congestion Window Growth . . . . . . . . . . . . . . . . 21 94 6.4. Congestion Window Decrease . . . . . . . . . . . . . . . 21 95 7. Chunk Scheduling and Rescheduling . . . . . . . . . . . . . . 21 96 8. Socket API Considerations . . . . . . . . . . . . . . . . . . 21 97 9. Testbed Platform . . . . . . . . . . . . . . . . . . . . . . 22 98 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 22 99 10.1. A New Chunk Type . . . . . . . . . . . . . . . . . . . . 22 100 11. Security Considerations . . . . . . . . . . . . . . . . . . . 23 101 12. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 23 102 13. References . . . . . . . . . . . . . . . . . . . . . . . . . 23 103 13.1. Normative References . . . . . . . . . . . . . . . . . . 23 104 13.2. Informative References . . . . . . . . . . . . . . . . . 24 105 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 27 107 1. Introduction 109 One of the important features of the Stream Control Transmission 110 Protocol (SCTP), which is currently specified in [RFC4960], is 111 network fault tolerance. This feature is for example required for 112 Reliable Server Pooling (RSerPool, [RFC5351]). Therefore, 113 transmitting messages over multiple paths is supported, but only for 114 redundancy. So [RFC4960] does not specify how to use multiple paths 115 simultaneously. 117 This document overcomes this limitation by specifying how multiple 118 paths can be used simultaneously. This has several benefits: 120 o Improved bandwidth usage. 122 o Better availability check with real user messages compared to 123 HEARTBEAT-based information. 125 2. Conventions 127 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 128 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 129 document are to be interpreted as described in [RFC2119]. 131 3. Load Sharing 133 Basic requirement for applying SCTP load sharing is the Concurrent 134 Multipath Transfer (CMT) extension of SCTP, which utilises multiple 135 paths simultaneously. We denote CMT-enabled SCTP as CMT-SCTP 136 throughout this document. CMT-SCTP is introduced in [IAS06] and in 137 more detail in [I06], some illustrative examples of chunk handling 138 are provided in [OMNeTWorkshop2010-SCTP]. CMT-SCTP provides three 139 modifications to standard SCTP (split Fast Retransmissions, 140 appropriate congestion window growth and delayed SACKs), which are 141 described in the following subsections. 143 3.1. Split Fast Retransmissions 145 Paths with different latencies lead to overtaking of DATA chunks. 146 This leads to gap reports, which are handled by Fast Retransmissions. 147 However, due to the fact that multiple paths are used simultaneously, 148 these Fast Retransmissions are usually useless and furthermore lead 149 to a decreased congestion window size. 151 To avoid unnecessary Fast Retransmissions, the sender has to keep 152 track of the path each DATA chunk has been sent on and consider 153 transmission paths before performing Fast Retransmissions. That is, 154 on reception of a SACK, the sender MUST identify the highest 155 acknowledged TSN on each path. A chunk SHOULD only be considered as 156 missing if its TSN is smaller than the highest acknowledged TSN on 157 its path. Section 3.1 of [OMNeTWorkshop2010-SCTP] contains an 158 illustrated example. 160 3.2. Appropriate Congestion Window Growth 162 The congestion window adaptation algorithm for SCTP [RFC4960] allows 163 increasing the congestion window only when a new cumulative ack 164 (CumAck) is received by a sender. When SACKs with unchanged CumAcks 165 are generated (due to reordering) and later arrive at a sender, the 166 sender does not modify its congestion window. Since a CMT-SCTP 167 receiver naturally observes reordering, many SACKs are sent 168 containing new gap reports but not new CumAcks. When these gaps are 169 later acked by a new CumAck, congestion window growth occurs, but 170 only for the data newly acked in the most recent SACK. Data 171 previously acked through gap reports will not contribute to 172 congestion window growth, in order to prevent sudden increases in the 173 congestion window resulting in bursts of data being sent. 175 To overcome the problems described above, the congestion window 176 growth has to be handled as follows [IAS06]: 178 o The sender SHOULD keep track of the earliest non-retransmitted 179 outstanding TSN per path. 181 o The sender SHOULD keep track of the earliest retransmitted 182 outstanding TSN per path. 184 o The in-order delivery per path SHOULD be deduced. 186 o The congestion window of a path SHOULD be increased when the 187 earliest non-retransmitted outstanding TSN of this path is 188 advanced ('Pseudo CumAck') OR when the earliest retransmitted 189 outstanding TSN of this path is advanced ('RTX Pseudo CumAck'). 191 Section 3.2 of [OMNeTWorkshop2010-SCTP] contains an illustrated 192 example of appropriate congestion window handling for CMT-SCTP. 194 3.3. Appropriate Delayed Acknowledgements 196 Standard SCTP [RFC4960] sends a SACK as soon as an out-of-sequence 197 TSN has been received. Delayed Acknowledgements are only allowed if 198 the received TSNs are in sequence. However, due to the load 199 balancing of CMT-SCTP, DATA chunks may overtake each other. This 200 leads to a high number of out-of-sequence TSNs, which have to be 201 acknowledged immediately. Clearly, this behaviour increases the 202 overhead traffic (usually nearly one SACK chunk for each received 203 packet containing a DATA chunk). 205 Delayed Acknowledgements for CMT-SCTP are handled as follows: 207 o In addition to [RFC4960], delaying of SACKs SHOULD *also* be 208 applied for out-of-sequence TSNs. 210 o A receiver MUST maintain a counter for the number of DATA chunks 211 received before sending a SACK. The value of the counter is 212 stored into each SACK chunk (FIXME: add details; needs reservation 213 of flags bits by IANA). After transmitting a SACK, the counter 214 MUST be reset to 0. Its initial value MUST be 0. 216 o The SACK handling procedure for a missing TSN M is extended as 217 follows: 219 * If all newly acknowledged TSNs have been transmitted over the 220 same path: 222 + If there are newly acknowledged TSNs L and H so that L <= M 223 <= H, the missing count of TSN M SHOULD be incremented by 224 one (like for standard SCTP according to [RFC4960]). 226 + Else if all newly acknowledged TSNs N satisfy the condition 227 M <= N, the missing count of TSN M SHOULD be incremented by 228 the number of TSNs reported in the SACK chunk. 230 * Otherwise (that is, there are newly acknowledged TSNs on 231 different paths), the missing count of TSN M SHOULD be 232 incremented by one (like for standard SCTP according to 233 [RFC4960]). 235 Section 3.3 of [OMNeTWorkshop2010-SCTP] contains an illustrated 236 example of Delayed Acknowledgements for CMT-SCTP. 238 4. Non-Renegable SACK 240 4.1. Negotiation 242 Before sending/receiving NR-SACKs (see [YEN10]), both peer endpoints 243 MUST agree on using NR-SACKs. This agreement MUST be negotiated 244 during association establishment. NR-SACK is an extension to the 245 core SCTP, and SCTP extensions that an endpoint supports are reported 246 to the peer endpoint in Supported Extensions Parameter during 247 association establishment (see Section 4.2.7 of [RFC5061].) The 248 Supported Extensions Parameter consists of a list of non-standard 249 Chunk Types that are supported by the sender. 251 An endpoint supporting the NR-SACK extension MUST list the NR-SACK 252 chunk in the Supported Extensions Parameter carried in the INIT or 253 INIT-ACK chunk, depending on whether the endpoint initiates or 254 responds to the initiation of the association. If the NR-SACK chunk 255 type ID is listed in the Chunk Types List of the Supported Extensions 256 Parameter, then the receiving endpoint MUST assume that the NR-SACK 257 chunk is supported by the sending endpoint. 259 Both endpoints MUST support NR-SACKs for either endpoint to send an 260 NR-SACK. If an endpoint establishes an association with a remote 261 endpoint that does not list NR-SACK in the Supported Extensions 262 Parameter carried in INIT chunk, then both endpoints of the 263 association MUST NOT use NR-SACKs. After association establishment, 264 an endpoint MUST NOT renegotiate the use of NR-SACKs. 266 Once both endpoints indicate during association establishment that 267 they support the NR-SACK extension, each endpoint SHOULD acknowledge 268 received DATA chunks with NR-SACK chunks, and not SACK chunks. That 269 is, throughout an SCTP association, both endpoints SHOULD send either 270 SACK chunks or NR-SACK chunks, never a mixture of the two. 272 4.2. The New Chunk Type: Non-Renegable SACK (NR-SACK) 274 Table 1 illustrates a new chunk type that will be used to transfer 275 NR-SACK information. 277 Chunk Type Chunk Name 278 -------------------------------------------------------------- 279 0x10 Non-Renegable Selective Acknowledgment (NR-SACK) 281 Table 1: NR-SACK Chunk 283 As the NR-SACK chunk replaces the SACK chunk, many SACK chunk fields 284 are preserved in the NR-SACK chunk. These preserved fields have the 285 same semantics with the corresponding SACK chunk fields, as defined 286 in [RFC4960], Section 3.3.4. The Gap Ack fields from RFC4960 have 287 been renamed as R Gap Ack to emphasize their renegable nature. Their 288 semantics are unchanged. For completeness, we describe all fields of 289 the NR-SACK chunk, including those that are identical in the SACK 290 chunk. 292 Similar to the SACK chunk, the NR-SACK chunk is sent to a peer 293 endpoint to (1) acknowledge DATA chunks received in-order, (2) 294 acknowledge DATA chunks received out-of-order, and (3) identify DATA 295 chunks received more than once (i.e., duplicate.) In addition, the 296 NR-SACK chunk (4) informs the peer endpoint of non-renegable out-of- 297 order DATA chunks. 299 0 1 2 3 300 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 301 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 302 | Type = 0x10 | Chunk Flags | Chunk Length | 303 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 304 | Cumulative TSN Ack | 305 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 306 | Advertised Receiver Window Credit (a_rwnd) | 307 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 308 |Number of R Gap Ack Blocks = N |Number of NR Gap Ack Blocks = M| 309 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 310 | Number of Duplicate TSNs = X | Reserved | 311 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 312 | R Gap Ack Block #1 Start | R Gap Ack Block #1 End | 313 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 314 / / 315 \ ... \ 316 / / 317 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 318 | R Gap Ack Block #N Start | R Gap Ack Block #N End | 319 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 320 | NR Gap Ack Block #1 Start | NR Gap Ack Block #1 End | 321 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 322 / / 323 \ ... \ 324 / / 325 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 326 | NR Gap Ack Block #M Start | NR Gap Ack Block #M End | 327 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 328 | Duplicate TSN 1 | 329 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 330 / / 331 \ ... \ 332 / / 333 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 334 | Duplicate TSN X | 335 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 337 Type: 8 bits 339 This field holds the IANA defined chunk type for NR-SACK chunk. The 340 suggested value of this field for IANA is 0x10. 342 Chunk Flags: 8 bits 344 Currently not used. It is recommended a sender set all bits to zero 345 on transmit, and a receiver ignore this field. 347 Chunk Length: 16 bits (unsigned integer) [Same as SACK chunk] 349 This value represents the size of the chunk in bytes including the 350 Chunk Type, Chunk Flags, Chunk Length, and Chunk Value fields. 352 Cumulative TSN Ack: 32 bits (unsigned integer) [Same as SACK chunk] 354 The value of the Cumulative TSN Ack is the last TSN received before a 355 break in the sequence of received TSNs occurs. The next TSN value 356 following the Cumulative TSN Ack has not yet been received at the 357 endpoint sending the NR-SACK. 359 Advertised Receiver Window Credit (a_rwnd): 32 bits (unsigned 360 integer) [Same as SACK chunk] 362 Indicates the updated receive buffer space in bytes of the sender of 363 this NR-SACK, see Section 6.2.1 of [RFC4960] for details. 365 Number of (R)enegable Gap Ack Blocks (N): 16 bits (unsigned integer) 367 Indicates the number of Renegable Gap Ack Blocks included in this NR- 368 SACK. 370 Number of (N)on(R)enegable Gap Ack Blocks (M): 16 bits (unsigned 371 integer) 373 Indicates the number of Non-Renegable Gap Ack Blocks included in this 374 NR-SACK. 376 Number of Duplicate TSNs (X): 16 bits [Same as SACK chunk] 378 Contains the number of duplicate TSNs the endpoint has received. 379 Each duplicate TSN is listed following the NR Gap Ack Block list. 381 Reserved : 16 bits 383 Currently not used. It is recommended a sender set all bits to zero 384 on transmit, and a receiver ignore this field. 386 (R)enegable Gap Ack Blocks: 388 The NR-SACK contains zero or more R Gap Ack Blocks. Each R Gap Ack 389 Block acknowledges a subsequence of renegable out-of-order TSNs. By 390 definition, all TSNs acknowledged by R Gap Ack Blocks are "greater 391 than" the value of the Cumulative TSN Ack. 393 Because of TSN numbering wraparound, comparisons and all arithmetic 394 operations discussed in this document are based on "Serial Number 395 Arithmetic" as described in Section 1.6 of [RFC4960]. 397 R Gap Ack Blocks are repeated for each R Gap Ack Block up to 'N' 398 defined in the Number of R Gap Ack Blocks field. All DATA chunks 399 with TSNs >= (Cumulative TSN Ack + R Gap Ack Block Start) and <= 400 (Cumulative TSN Ack + R Gap Ack Block End) of each R Gap Ack Block 401 are assumed to have been received correctly, and are renegable. 403 R Gap Ack Block Start: 16 bits (unsigned integer) 405 Indicates the Start offset TSN for this R Gap Ack Block. This number 406 is set relative to the cumulative TSN number defined in Cumulative 407 TSN Ack field. To calculate the actual start TSN number, the 408 Cumulative TSN Ack is added to this offset number. The calculated 409 TSN identifies the first TSN in this R Gap Ack Block that has been 410 received. 412 R Gap Ack Block End: 16 bits (unsigned integer) 414 Indicates the End offset TSN for this R Gap Ack Block. This number 415 is set relative to the cumulative TSN number defined in the 416 Cumulative TSN Ack field. To calculate the actual TSN number, the 417 Cumulative TSN Ack is added to this offset number. The calculated 418 TSN identifies the TSN of the last DATA chunk received in this R Gap 419 Ack Block. 421 N(on)R(enegable) Gap Ack Blocks: 423 The NR-SACK contains zero or more NR Gap Ack Blocks. Each NR Gap Ack 424 Block acknowledges a continuous subsequence of non-renegable out-of- 425 order DATA chunks. If a TSN is nr-gap-acked in any NR-SACK chunk, 426 then all subsequently transmitted NR-SACKs with a smaller cum-ack 427 value than that TSN SHOULD also nr-gap-ack that TSN. 429 NR Gap Ack Blocks are repeated for each NR Gap Ack Block up to 'M' 430 defined in the Number of NR Gap Ack Blocks field. All DATA chunks 431 with TSNs >= (Cumulative TSN Ack + NR Gap Ack Block Start) and <= 432 (Cumulative TSN Ack + NR Gap Ack Block End) of each NR Gap Ack Block 433 are assumed to be received correctly, and are Non-Renegable. 435 NR Gap Ack Block Start: 16 bits (unsigned integer) 437 Indicates the Start offset TSN for this NR Gap Ack Block. This 438 number is set relative to the cumulative TSN number defined in 439 Cumulative TSN Ack field. To calculate the actual TSN number, the 440 Cumulative TSN Ack is added to this offset number. The calculated 441 TSN identifies the first TSN in this NR Gap Ack Block that has been 442 received. 444 NR Gap Ack Block End: 16 bits (unsigned integer) 446 Indicates the End offset TSN for this NR Gap Ack Block. This number 447 is set relative to the cumulative TSN number defined in Cumulative 448 TSN Ack field. To calculate the actual TSN number, the Cumulative 449 TSN Ack is added to this offset number. The calculated TSN 450 identifies the TSN of the last DATA chunk received in this NR Gap Ack 451 Block. 453 Note: 455 NR Gap Ack Blocks and R Gap Ack Blocks in an NR-SACK chunk SHOULD 456 acknowledge disjoint sets of TSNs. That is, an out-of-order TSN 457 SHOULD be listed in either an R Gap Ack Block or an NR Gap Ack Block, 458 but not the both. R Gap Ack Blocks and NR Gap Ack Blocks together 459 provide the information as do the Gap Ack Block of a SACK chunk, plus 460 additional information about non-renegability. 462 If all out-of-order data acked by an NR-SACK are renegable, then the 463 Number of NR Gap Ack Blocks MUST be set to 0. If all out-of-order 464 data acked by an NR-SACK are non-renegable, then the Number of R Gap 465 Ack Blocks SHOULD be set to 0. TSNs listed in R Gap Ack Block will 466 be referred as r-gap-acked. 468 Duplicate TSN: 32 bits (unsigned integer) [Same as SACK chunk] 470 Indicates a duplicate TSN received since the last NR-SACK was sent. 471 Exactly 'X' duplicate TSNs SHOULD be reported where 'X' was defined 472 in Number of Duplicate TSNs field. 474 Each duplicate TSN is listed in this field as many times as the TSN 475 was received since the previous NR-SACK was sent. For example, if a 476 data receiver were to get the TSN 19 three times, the data receiver 477 would list 19 twice in the outbound NR-SACK. After sending the NR- 478 SACK if the receiver received one more TSN 19, the receiver would 479 list 19 as a duplicate once in the next outgoing NR-SACK. 481 4.3. An Illustrative Example 483 Assume the following DATA chunks have arrived at the receiver. 485 -------------------------------- 486 | TSN=16| SID=2 | SSN=N/A| U=1 | 487 -------------------------------- 488 | TSN=15| SID=1 | SSN= 4 | U=0 | 489 -------------------------------- 490 | TSN=14| SID=0 | SSN= 4 | U=0 | 491 -------------------------------- 492 | TSN=13| SID=2 | SSN=N/A| U=1 | 493 -------------------------------- 494 | | 495 -------------------------------- 496 | TSN=11| SID=0 | SSN= 3 | U=0 | 497 ------------------------------- 498 | | 499 -------------------------------- 500 | | 501 -------------------------------- 502 | TSN=8 | SID=2 | SSN=N/A| U=1 | 503 -------------------------------- 504 | TSN=7 | SID=1 | SSN= 2 | U=0 | 505 -------------------------------- 506 | TSN=6 | SID=1 | SSN= 1 | U=0 | 507 -------------------------------- 508 | TSN=5 | SID=0 | SSN= 1 | U=0 | 509 -------------------------------- 510 | | 511 -------------------------------- 512 | TSN=3 | SID=1 | SSN= 0 | U=0 | 513 -------------------------------- 514 | TSN=2 | SID=0 | SSN= 0 | U=0 | 515 -------------------------------- 517 The above figure shows the list of DATA chunks at the receiver. TSN 518 denotes the transmission sequence number of the DATA chunk, SID 519 denotes the stream id to which the DATA chunk belongs, SSN denotes 520 the sequence number of the DATA chunk within its stream, and the U 521 bit denotes whether the DATA chunk requires ordered(=0) or 522 unordered(=1) delivery [RFC4960]. Note that TSNs 4,9,10, and 12 have 523 not arrived. 525 This data can be viewed as three separate streams as follows (assume 526 each stream begins with SSN=0.) Note that in this example, the 527 application uses stream 2 for unordered data transfer. By 528 definition, SSN fields of unordered DATA chunks are ignored. 530 Stream-0: 532 SSN: 0 1 2 3 4 533 TSN: | 2 | 5 | | 11 | 14 | 534 U-Bit: | 0 | 0 | | 0 | 0 | 536 Stream-1: 538 SSN: 0 1 2 3 4 539 TSN: | 3 | 6 | 7 | | 15 | 540 U-Bit: | 0 | 0 | 0 | | 0 | 542 Stream-2: 544 SSN: N/A N/A N/A 545 TSN: | 8 | 13 | 16 | 546 U-Bit: | 1 | 1 | 1 | 548 The NR-SACK to acknowledge the above data SHOULD be constructed as 549 follows for each of the three cases described below (the a_rwnd is 550 arbitrarily set to 4000): 552 CASE-1: Minimal Data Receiver Responsibility - no out-of-order 553 deliverable data yet delivered 555 None of the deliverable out-of-order DATA chunks have been delivered, 556 and the receiver of the above data does not take responsibility for 557 any of the received out-of-order DATA chunks. The receiver reserves 558 the right to renege any or all of the out-of-order DATA chunks. 560 +-----------------------------+-----------------------------+ 561 | Type = 0x10 | 00000000 | Chunk Length = 32 | 562 +-----------------------------+-----------------------------+ 563 | Cumulative TSN Ack = 3 | 564 +-----------------------------+-----------------------------+ 565 | a_rwnd = 4000 | 566 +-----------------------------+-----------------------------+ 567 | Num of R Gap Ack Blocks = 3 |Num of NR Gap Ack Blocks = 0 | 568 +-----------------------------+-----------------------------+ 569 | Num of Duplicates = 0 | 0x00 | 570 +-----------------------------+-----------------------------+ 571 |R Gap Ack Block #1 Start = 2 | R Gap Ack Block #1 End = 5 | 572 +-----------------------------+-----------------------------+ 573 |R Gap Ack Block #2 Start = 8 | R Gap Ack Block #2 End = 8 | 574 +-----------------------------+-----------------------------+ 575 |R Gap Ack Block #3 Start = 10| R Gap Ack Block #3 End = 13 | 576 +-----------------------------+-----------------------------+ 577 CASE-2: Minimal Data Receiver Responsibility - all out-of-order 578 deliverable data delivered 580 In this case, the NR-SACK chunk is being sent after the data receiver 581 has delivered all deliverable out-of-order DATA chunks to its 582 receiving application(i.e., TSNs 5,6,7,8,13, and 16.) The receiver 583 reserves the right to renege on all undelivered out-of-order DATA 584 chunks(i.e., TSNs 11,14, and 15.) 586 +------------------------------+------------------------------+ 587 | Type = 0x10 | 0x00 | Chunk Length = 40 | 588 +------------------------------+------------------------------+ 589 | Cumulative TSN Ack = 3 | 590 +------------------------------+------------------------------+ 591 | a_rwnd = 4000 | 592 +------------------------------+------------------------------+ 593 | Num of R Gap Ack Blocks = 2 | Num of NR Gap Ack Blocks = 3 | 594 +------------------------------+------------------------------+ 595 | Num of Duplicates = 0 | 0x00 | 596 +------------------------------+------------------------------+ 597 | R Gap Ack Block #1 Start = 8 | R Gap Ack Block #1 End = 8 | 598 +------------------------------+------------------------------+ 599 | R Gap Ack Block #2 Start = 11| R Gap Ack Block #2 End = 12 | 600 +------------------------------+------------------------------+ 601 |NR Gap Ack Block #1 Start = 2 | NR Gap Ack Block #1 End = 5 | 602 +------------------------------+------------------------------+ 603 |NR Gap Ack Block #2 Start = 10| NR Gap Ack Block #2 End = 10 | 604 +------------------------------+------------------------------+ 605 |NR Gap Ack Block #3 Start = 13| NR Gap Ack Block #3 End = 13 | 606 +------------------------------+------------------------------+ 608 CASE-3: Maximal Data Receiver Responsibility 610 In this special case, all out-of-order data blocks acknowledged are 611 non-renegable. This case would occur when the data receiver is 612 programmed never to renege, and takes responsibility to deliver all 613 DATA chunks that arrive out-of-order. In this case Num of R Gap Ack 614 Blocks is zero indicating all reported out-of-order TSNs are nr-gap- 615 acked. 617 +--------------------------------+-------------------------------+ 618 | Type = 0x10 | 0x00 | Chunk Length = 32 | 619 +--------------------------------+-------------------------------+ 620 | Cumulative TSN Ack = 3 | 621 +--------------------------------+-------------------------------+ 622 | a_rwnd = 4000 | 623 +--------------------------------+-------------------------------+ 624 | Num of R Gap Ack Blocks = 0 | Num of NR Gap Ack Blocks = 3 | 625 +--------------------------------+-------------------------------+ 626 | Num of Duplicates = 0 | 0x00 | 627 +--------------------------------+-------------------------------+ 628 | NR Gap Ack Block #1 Start = 2 | NR Gap Ack Block #1 End = 5 | 629 +--------------------------------+-------------------------------+ 630 | NR Gap Ack Block #2 Start = 8 | NR Gap Ack Block #2 End = 8 | 631 +--------------------------------+-------------------------------+ 632 | NR Gap Ack Block #3 Start = 10 | NR Gap Ack Block #3 End = 13 | 633 +--------------------------------+-------------------------------+ 635 4.4. Procedures 637 The procedures regarding "when" to send an NR-SACK chunk are 638 identical to the procedures regarding when to send a SACK chunk, as 639 outlined in Section 6.2 of [RFC4960]. 641 4.4.1. Sending an NR-SACK chunk 643 All of the NR-SACK chunk fields identical to the SACK chunk MUST be 644 formed as described in Section 6.2 of [RFC4960]. 646 It is up to the data receiver whether or not to take responsibility 647 for delivery of each out-of-order DATA chunk. An out-of-order DATA 648 chunk that has already been delivered, or that the receiver takes 649 responsibility to deliver (i.e., guarantees not to renege) is Non 650 Renegable(NR), and SHOULD be included in an NR Gap Ack Block field of 651 the outgoing NR-SACK. All other out-of-order data is (R)enegable, 652 and SHOULD be included in R Gap Ack Block field of the outgoing NR- 653 SACK. 655 Consider three types of data receiver: 657 CASE-1: Data receiver takes no responsibility for delivery of any 658 out-of-order DATA chunks 660 CASE-2: Data receiver takes responsibility for all out-of-order DATA 661 chunks that are "deliverable" (i.e., DATA chunks in-sequence 662 within the stream they belong to, or DATA chunks whose (U)nordered 663 bit is 1) 665 CASE-3: Data receiver takes responsibility for delivery of all out- 666 of-order DATA chunks, whether deliverable or not deliverable 668 The data receiver SHOULD follow the procedures outlined below for 669 building the NR-SACK. 671 CASE-1: 673 1A) Identify the TSNs received out-of-order. 675 1B) For these out-of-order TSNs, identify the R Gap Ack Blocks. 676 Fill the Number of R Gap Ack Blocks (N) field, R Gap Ack Block #i 677 Start, and R Gap Ack Block #i End where i goes from 1 to N. 679 1C) Set the Number of NR Gap Ack Blocks (M) field to 0. 681 CASE-2: 683 2A) Identify the TSNs received out-of-order. 685 2B) For the received out-of-order TSNs, check the (U)nordered bit of 686 each TSN. Tag unordered TSNs as NR. 688 2C) For each stream, also identify the TSNs received out-of-order 689 but are in-sequence within that stream. Tag those in-sequence 690 TSNs as NR. 692 2D) Tag all out-of-order data that is not NR as (R)enegable. 694 2E) For those TSNs tagged as (R)enegable, identify the (R)enegable 695 Blocks. Fill the Number of R Gap Ack Blocks(N) field, R Gap Ack 696 Block #i Start, and R Gap Ack Block #i End where i goes from 1 to 697 N. 699 2F) For those TSNs tagged as NR, identify the NR Blocks. Fill the 700 Number of NR Gap Ack Blocks(M) field, NR Gap Ack Block #i Start, 701 and NR Gap Ack Block #i End where i goes from 1 to M. 703 CASE-3: 705 3A) Identify the TSNs received out-of-order. All of these TSNs 706 SHOULD be nr-gap-acked. 708 3B) Set the Number of R Gap Ack Blocks (N) field to 0. 710 3C) For these out-of-order TSNs, identify the NR Gap Ack Blocks. 711 Fill the Number of NR Gap Ack Blocks (M) field, NR Gap Ack Block 712 #i Start, and NR Gap Ack Block #i End where i goes from 1 to M. 714 RFC4960 states that the SCTP endpoint MUST report as many Gap Ack 715 Blocks as can fit in a single SACK chunk limited by the current path 716 MTU. When using NR-SACKs, the SCTP endpoint SHOULD fill as many R 717 Gap Ack Blocks and NR Gap Ack Blocks starting from the Cumulative TSN 718 Ack value as can fit in a single NR-SACK chunk limited by the current 719 path MTU. If space remains, the SCTP endpoint SHOULD fill as many 720 Duplicate TSNs as possible starting from Cumulative TSN Ack value. 722 4.4.2. Receiving an NR-SACK Chunk 724 When an NR-SACK chunk is received, all of the NR-SACK fields 725 identical to a SACK chunk SHOULD be processed and handled as in SACK 726 chunk handling outlined in Section 6.2.1 of [RFC4960]. 728 The NR Gap Ack Block Start(s) and NR Gap Ack Block End(s) are offsets 729 relative to the cum-ack. To calculate the actual range of nr-gap- 730 acked TSNs, the cum-ack MUST be added to the Start and End. 732 For example, assume an incoming NR-SACK chunk's cum-ack is 12 and an 733 NR Gap Ack Block defines the NR Gap Ack Block Start=5, and the NR Gap 734 Ack Block End=7. This NR Gap Ack block nr-gap-acks TSNs 17 through 735 19 inclusive. 737 Upon reception of an NR-SACK chunk, all TSNs listed in either R Gap 738 Ack Block(s) or NR Gap Ack Block(s) SHOULD be processed as would be 739 TSNs included in Gap Ack Block(s) of a SACK chunk. All TSNs in all 740 NR Gap Ack Blocks SHOULD be removed from the data sender's 741 retransmission queue as their delivery to the receiving application 742 has either already occurred, or is guaranteed by the data receiver. 744 Although R Gap Ack Blocks and NR Gap Ack Blocks SHOULD be disjoint 745 sets, NR-SACK processing SHOULD work if an NR-SACK chunk has a TSN 746 listed in both an R Gap Ack Block and an NR Gap Ack Block. In this 747 case, the TSN SHOULD be treated as Non-Renegable. 749 Implementation Note: 751 Most of NR-SACK processing at the data sender can be implemented by 752 using the same routines as in SACK that process the cum ack and the 753 gap ack(s), followed by removal of nr-gap-acked DATA chunks from the 754 retransmission queue. However, with NR-SACKs, as out-of-order DATA 755 is sometimes removed from the retransmission queue, the gap ack 756 processing routine should recognize that the data sender's 757 retransmission queue has some transmitted data removed. For example, 758 while calculating missing reports, the gap ack processing routine 759 cannot assume that the highest TSN transmitted is always at the tail 760 (right edge) of the retransmission queue. 762 5. Buffer Blocking Mitigation 764 TBD. See [Dre2012], [PAMS2011], [Globecom2010]. 766 5.1. Sender Buffer Splitting 768 TBD. See [Dre2012], [PAMS2011], [Globecom2010]. 770 5.2. Receiver Buffer Splitting 772 TBD. See [Dre2012], [PAMS2011], [Globecom2010]. 774 5.3. Chunk Rescheduling 776 This algorithm ensures quick blocking resolution for ordered data. 777 TBD. See [Dre2012], [Globecom2010]. 779 5.4. Problems during Path Failure 781 This section discusses CMT's receive buffer related problems during 782 path failure, and proposes a solution for the same. 784 5.4.1. Problem Description 786 Link failures arise when a router or a link connecting two routers 787 fails due to link disconnection, hardware malfunction, or software 788 error. Overloaded links caused by flash crowds and denial-of-service 789 (DoS) attacks also degrade end-to-end communication between peer 790 hosts. Ideally, the routing system detects link failures, and in 791 response, reconfigures the routing tables and avoids routing traffic 792 via the failed link. However, existing research highlights problems 793 with Internet backbone routing that result in long route convergence 794 times. The pervasiveness of path failures motivated us to study 795 their impact on CMT, since CMT achieves better throughput via 796 simultaneous data transmission over multiple end-to-end paths. 798 CMT is an extension to SCTP, and therefore retains SCTP's failure 799 detection process. A CMT sender uses a tunable failure detection 800 threshold called Path.Max.Retrans (PMR). When a sender experiences 801 more than PMR consecutive timeouts while trying to reach an active 802 destination, the destination is marked as failed. With PMR=5, the 803 failure detection takes 6 consecutive timeouts or 63s. After every 804 timeout, the CMT sender continues to transmit new data on the failed 805 path increasing the chances of receive buffer (rbuf) blocking and 806 degrading CMT performance during permanent and short-term path 807 failures [NEA08]. 809 5.4.2. Solution: Potentially-failed Destination State 811 To mitigate the rbuf blocking, we introduce a new destination state 812 called 'potentially-failed' state in SCTP (and CMT's) failure 813 detection process [I-D.ietf-tsvwg-sctp-failover]. This solution is 814 based on the rationale that loss detected by a timeout implies either 815 severe congestion or failure en route. After a single timeout on a 816 path, a sender is unsure, and marks the corresponding destination as 817 'potentially-failed' (PF). A PF destination is not used for data 818 transmission or retransmission. CMT's retransmission policies are 819 augmented to include the PF state. Performance evaluations prove 820 that the PF state significantly reduces rbuf blocking during failure 821 detection [NEA08]. 823 5.5. Non-Renegable SACK 825 This section discusses problems with SCTP's SACK mechanism and how it 826 affects the send buffer and CMT performance. 828 5.5.1. Problem Description 830 Gap-acks acknowledge DATA chunks that arrive out-of-order to a 831 transport layer data receiver. A gap-ack in SCTP is advisory, in 832 that, while it notifies a data sender about the reception of 833 indicated DATA chunks, the data receiver is permitted to later 834 discard DATA chunks that it previously had gap-acked. Discarding a 835 previously gap-acked DATA chunk is known as 'reneging'. Because of 836 the possibility of reneging in SCTP, any gap-acked DATA chunk MUST 837 NOT be removed from the data sender's retransmission queue until the 838 DATA chunk is later CumAcked. 840 Situations exist when a data receiver knows that reneging on a 841 particular out-of-order DATA chunk will never take place, such as 842 (but not limited to) after an out-of-order DATA chunk is delivered to 843 the receiving application. With current SACKs in SCTP, it is not 844 possible for a data receiver to inform a data sender if or when a 845 particular out-of-order 'deliverable' DATA chunk has been 'delivered' 846 to the receiving application. Thus the data sender MUST keep a copy 847 of every gap-acked out-of-order DATA chunk(s) in the data sender's 848 retransmission queue until the DATA chunk is CumAcked. This use of 849 the data sender's retransmission queue is wasteful. The wasted 850 buffer often degrades CMT performance; the degradation increases when 851 a CMT flow traverses via paths with disparate end-to-end properties 852 [NEY08]. 854 5.5.2. Solution: Non-Renegable SACKs 856 Non-Renegable Selective Acknowledgments (NR-SACKs) Section 4 are a 857 new kind of acknowledgements, extending SCTP's SACK chunk 858 functionalities. The NR-SACK chunk is an extension of the existing 859 SACK chunk. Several fields are identical, including the Cumulative 860 TSN Ack, the Advertised Receiver Window Credit (a_rwnd), and 861 Duplicate TSNs. These fields have the same semantics as described in 862 [RFC4960]. 864 NR-SACKs also identify out-of-order DATA chunks that a receiver 865 either: (1) has delivered to its receiving application, or (2) takes 866 full responsibility to eventually deliver to its receiving 867 application. These out-of-order DATA chunks are 'non-renegable.' 868 Non-Renegable data are reported in the NR Gap Ack Block field of the 869 NR-SACK chunk as described Section 4. We refer to non-renegable 870 selective acknowledgements as 'nr-gap-acks.' 871 When an out-of-order DATA chunk is nr-gap-acked, the data sender no 872 longer needs to keep that particular DATA chunk in its retransmission 873 queue, thus allowing the data sender to free up its buffer space 874 sooner than if the DATA chunk were only gap-acked. NR-SACKs improve 875 send buffer utilization and throughput for CMT flows [NEY08]. 877 6. Handling of Shared Bottlenecks 879 6.1. Introduction 881 CMT-SCTP assumes all paths to be disjoint. Since each path 882 independently uses a TCP-like congestion control, an SCTP association 883 using N paths over the same bottleneck acquires N times the bandwidth 884 of a concurrent TCP flow. This is clearly unfair. A reliable 885 detection of shared bottlenecks is impossible in arbitrary networks 886 like the Internet. Therefore, [ICC2012] [ConTEL2011], [AINA2010] 887 apply the idea of Resource Pooling to CMT-SCTP. Resource Pooling 888 (RP) denotes 'making a collection of resources behave like a single 889 pooled resource' [WHB09]. The modifications of RP-enabled CMT-SCTP, 890 further denoted as CMT/RP-SCTP, are described in the following 891 subsections. A detailed description of CMT/RP-SCTP, including 892 congestion control examples, can be found in [ICC2012], [ConTEL2011], 893 [AINA2010]. 895 6.2. Initial Values 897 TDB. 899 6.3. Congestion Window Growth 901 TDB. See [Dre2012], [ICC2012], [ConTEL2011]. 903 6.4. Congestion Window Decrease 905 TDB. See [Dre2012], [ICC2012], [ConTEL2011]. 907 7. Chunk Scheduling and Rescheduling 909 TDB. See [Dre2012], [PFLDNeT2010]. 911 8. Socket API Considerations 913 See [I-D.dreibholz-tsvwg-sctpsocket-multipath] and 914 [I-D.dreibholz-tsvwg-sctpsocket-sqinfo]. 916 9. Testbed Platform 918 A large-scale and realistic Internet testbed platform with support 919 for the multi-homing feature of the underlying SCTP protocol is 920 NorNet. Particularly, it is also a platform for multi-path transport 921 experiments with CMT-SCTP. A description of and introduction to 922 NorNet is provided in [ComNets2013-Core], [PAMS2013-NorNet], 923 [NNUW2-Dreibholz-NorNetCore-Introduction], 924 [NNUW2-Dreibholz-NorNetCore-Tutorial]. Further information can be 925 found on the project website [NorNet-Website] at https://www.nntb.no. 927 An Open Source simulation model for CMT-SCTP is available for OMNeT++ 928 within the INET Framework. See [INET-Framework] for the Git 929 repository. For documentation on the model, see [Dre2012]. 931 10. IANA Considerations 933 NOTE to RFC-Editor: 935 "RFCXXXX" is to be replaced by the RFC number you assign this 936 document. 938 NOTE to RFC-Editor: 940 The suggested values for the chunk type and the chunk parameter 941 types are tentative and to be confirmed by IANA. 943 This document (RFCXXXX) is the reference for all registrations 944 described in this section. The suggested changes are described 945 below. 947 10.1. A New Chunk Type 949 A chunk type has to be assigned by IANA. It is suggested to use the 950 values given in Section 4. IANA should assign this value from the 951 pool of chunks with the upper two bits set to '00'. 953 This requires an additional line in the "Chunk Types" registry for 954 SCTP: 956 Chunk Types 958 ID Value Chunk Type Reference 959 ----- ---------- --------- 960 16 Non-Renegable SACK (NR-SACK) [RFCXXXX] 962 The registration table as defined in [RFC6096] for the chunk flags of 963 this chunk type is empty. 965 11. Security Considerations 967 This document does not add any additional security considerations in 968 addition to the ones given in [RFC4960]. 970 12. Acknowledgments 972 The authors wish to thank Hakim Adhari, Phillip Conrad, Jonathan 973 Leighton, Ertugrul Yilmaz and Xing Zhou for their invaluable comments 974 and support. 976 13. References 978 13.1. Normative References 980 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 981 Requirement Levels", BCP 14, RFC 2119, March 1997. 983 [RFC4960] Stewart, R., "Stream Control Transmission Protocol", RFC 984 4960, September 2007. 986 [RFC5061] Stewart, R., Xie, Q., Tuexen, M., Maruyama, S., and M. 987 Kozuka, "Stream Control Transmission Protocol (SCTP) 988 Dynamic Address Reconfiguration", RFC 5061, September 989 2007. 991 [RFC5351] Lei, P., Ong, L., Tuexen, M., and T. Dreibholz, "An 992 Overview of Reliable Server Pooling Protocols", RFC 5351, 993 September 2008. 995 [RFC6096] Tuexen, M. and R. Stewart, "Stream Control Transmission 996 Protocol (SCTP) Chunk Flags Registration", RFC 6096, 997 January 2011. 999 [I-D.ietf-tsvwg-sctp-failover] 1000 Nishida, Y., Natarajan, P., Caro, A., Amer, P., and K. 1001 Nielsen, "Quick Failover Algorithm in SCTP", draft-ietf- 1002 tsvwg-sctp-failover-05 (work in progress), July 2014. 1004 [I-D.dreibholz-tsvwg-sctpsocket-multipath] 1005 Dreibholz, T., Becke, M., and H. Adhari, "SCTP Socket API 1006 Extensions for Concurrent Multipath Transfer", draft- 1007 dreibholz-tsvwg-sctpsocket-multipath-06 (work in 1008 progress), July 2013. 1010 [I-D.dreibholz-tsvwg-sctpsocket-sqinfo] 1011 Dreibholz, T., Seggelmann, R., and M. Becke, "Sender Queue 1012 Info Option for the SCTP Socket API", draft-dreibholz- 1013 tsvwg-sctpsocket-sqinfo-06 (work in progress), July 2013. 1015 13.2. Informative References 1017 [I06] Iyengar, J., "End-to-End Concurrent Multipath Transfer 1018 Using Transport Layer Multihoming", PhD Dissertation 1019 Computer Science Dept., University of Delaware, April 1020 2006, . 1023 [IAS06] Iyengar, J., Amer, P., and R. Stewart, "Concurrent 1024 Multipath Transfer Using SCTP Multihoming Over Independent 1025 End-to-End Paths", Journal IEEE/ACM Transactions on 1026 Networking, October 2006, 1027 . 1029 [NEA08] Natarajan, P., Ekiz, N., Iyengar, J., Amer, P., and R. 1030 Stewart, "Concurrent Multipath Transfer Using Transport 1031 Layer Multihoming: Introducing the Potentially-failed 1032 Destination State", Proceedings of the IFIP Networking, 1033 May 2008, . 1036 [NEY08] Natarajan, P., Ekiz, N., Yilmaz, E., Amer, P., Iyengar, 1037 J., and R. Stewart, "Non-Renegable Selective 1038 Acknowledgments (NR-SACKs) for SCTP", Proceedings of the 1039 16th IEEE International Conference on Network Protocols 1040 (ICNP) , October 2008, 1041 . 1043 [WHB09] Wischik, D., Handley, M., and M. Braun, "The Resource 1044 Pooling Principle", Journal ACM SIGCOMM Computer 1045 Communication Review, October 2009, 1046 . 1049 [OMNeTWorkshop2010-SCTP] 1050 Dreibholz, T., Becke, M., Pulinthanath, J., and E. 1051 Rathgeb, "Implementation and Evaluation of Concurrent 1052 Multipath Transfer for SCTP in the INET Framework", 1053 Proceedings of the 3rd ACM/ICST International Workshop on 1054 OMNeT++, ISBN 978-963-9799-87-5, DOI 10.4108/ 1055 ICST.SIMUTOOLS2010.8673, March 2010, . 1059 [AINA2010] 1060 Dreibholz, T., Becke, M., Pulinthanath, J., and E. 1061 Rathgeb, "Applying TCP-Friendly Congestion Control to 1062 Concurrent Multipath Transfer", Proceedings of the 24th 1063 IEEE International Conference on Advanced Information 1064 Networking and Applications (AINA), Pages 312-319, 1065 ISBN 978-0-7695-4018-4, DOI 10.1109/AINA.2010.117, April 1066 2010, . 1069 [YEN10] Yilmaz, E., Ekiz, N., Natarajan, P., Amer, P., Leighton, 1070 J., Baker, F., and R. Stewart, "Throughput Analysis of 1071 Non-Renegable Selective Acknowledgments (NR-SACKs) for 1072 SCTP", Computer Communications, doi:10.1016/ 1073 j.comcom.2010.06.028, 2010. 1075 [PFLDNeT2010] 1076 Dreibholz, T., Seggelmann, R., Tuexen, M., and E. Rathgeb, 1077 "Transmission Scheduling Optimizations for Concurrent 1078 Multipath Transfer", Proceedings of the 8th International 1079 Workshop on Protocols for Future, Large-Scale and Diverse 1080 Network Transports (PFLDNeT), Volume 8, ISSN 2074-5168, 1081 November 2010, . 1084 [Globecom2010] 1085 Dreibholz, T., Becke, M., Rathgeb, E., and M. Tuexen, "On 1086 the Use of Concurrent Multipath Transfer over Asymmetric 1087 Paths", Proceedings of the IEEE Global Communications 1088 Conference (GLOBECOM), ISBN 978-1-4244-5637-6, DOI 10.1109 1089 /GLOCOM.2010.5683579, December 2010, . 1093 [PAMS2011] 1094 Adhari, H., Dreibholz, T., Becke, M., Rathgeb, E., and M. 1095 Tuexen, "Evaluation of Concurrent Multipath Transfer over 1096 Dissimilar Paths", Proceedings of the 1st International 1097 Workshop on Protocols and Applications with Multi-Homing 1098 Support (PAMS), Pages 708-714, ISBN 978-0-7695-4338-3, 1099 DOI 10.1109/WAINA.2011.92, March 2011, . 1103 [ConTEL2011] 1104 Dreibholz, T., Becke, M., Adhari, H., and E. Rathgeb, "On 1105 the Impact of Congestion Control for Concurrent Multipath 1106 Transfer on the Transport Layer", Proceedings of the 11th 1107 IEEE International Conference on 1108 Telecommunications (ConTEL), Pages 397-404, 1109 ISBN 978-953-184-152-8, June 2011, . 1113 [ICC2012] Becke, M., Dreibholz, T., Adhari, H., and E. Rathgeb, "On 1114 the Fairness of Transport Protocols in a Multi-Path 1115 Environment", Proceedings of the IEEE International 1116 Conference on Communications (ICC), Pages 2666-2672, 1117 ISBN 978-1-4577-2052-9, DOI 10.1109/ICC.2012.6363695, June 1118 2012, . 1121 [Dre2012] Dreibholz, T., "Evaluation and Optimisation of Multi-Path 1122 Transport using the Stream Control Transmission Protocol", 1123 March 2012, . 1127 [NorNet-Website] 1128 Dreibholz, T., "NorNet -- A Real-World, Large-Scale Multi- 1129 Homing Testbed", Online: https://www.nntb.no/, 2014, 1130 . 1132 [PAMS2013-NorNet] 1133 Dreibholz, T. and E. Gran, "Design and Implementation of 1134 the NorNet Core Research Testbed for Multi-Homed Systems", 1135 Proceedings of the 3nd International Workshop on Protocols 1136 and Applications with Multi-Homing Support (PAMS), Pages 1137 1094-1100, ISBN 978-0-7695-4952-1, DOI 10.1109/ 1138 WAINA.2013.71, March 2013, . 1142 [ComNets2013-Core] 1143 Gran, E., Dreibholz, T., and A. Kvalbein, "NorNet Core - A 1144 Multi-Homed Research Testbed", Computer Networks, Special 1145 Issue on Future Internet Testbeds, ISSN 1389-1286, 1146 DOI 10.1016/j.bjp.2013.12.035, January 2014, . 1150 [INET-Framework] 1151 Hornig, R. and A. Varga, "INET Framework Git Repository", 1152 Online: https://github.com/inet-framework/inet, 2014, 1153 . 1155 [NNUW2-Dreibholz-NorNetCore-Introduction] 1156 Dreibholz, T., "The NorNet Core Testbed - Introduction and 1157 Status in August 2014", Proceedings of the 2nd 1158 International NorNet Users Workshop (NNUW-2), August 2014, 1159 . 1162 [NNUW2-Dreibholz-NorNetCore-Tutorial] 1163 Dreibholz, T., "An Experiment Tutorial for the NorNet Core 1164 Testbed", Proceedings of the 2nd International NorNet 1165 Users Workshop (NNUW-2), August 2014, . 1169 Authors' Addresses 1171 Paul D. Amer 1172 University of Delaware, Computer and Information Sciences Department 1173 Newark, Delaware 19716 1174 U.S.A. 1176 Phone: +1-302-831-1944 1177 Email: amer@cis.udel.edu 1178 URI: http://www.eecis.udel.edu/~amer/ 1180 Martin Becke 1181 University of Duisburg-Essen, Institute for Experimental Mathematics 1182 Ellernstrasse 29 1183 45326 Essen, Nordrhein-Westfalen 1184 Germany 1186 Phone: +49-201-183-7667 1187 Fax: +49-201-183-7673 1188 Email: martin.becke@uni-due.de 1189 URI: http://www.scimbe.de/ 1190 Thomas Dreibholz 1191 Simula Research Laboratory, Network Systems Group 1192 Martin Linges vei 17 1193 1364 Fornebu, Akershus 1194 Norway 1196 Phone: +47-6782-8200 1197 Fax: +47-6782-8201 1198 Email: dreibh@simula.no 1199 URI: http://www.iem.uni-due.de/~dreibh/ 1201 Nasif Ekiz 1202 University of Delaware, Computer and Information Sciences Department 1203 Newark, Delaware 19716 1204 U.S.A. 1206 Email: nekiz@udel.edu 1207 URI: http://www.eecis.udel.edu/~nekiz/ 1209 Janardhan Iyengar 1210 Franklin and Marshall College, Mathematics and Computer Science 1211 PO Box 3003 1212 Lancaster, Pennsylvania 17604-3003 1213 U.S.A. 1215 Phone: +1-717-358-4774 1216 Email: jiyengar@fandm.edu 1217 URI: http://www.fandm.edu/jiyengar/ 1219 Preethi Natarajan 1220 Cisco Systems 1221 425 East Tasman Drive 1222 San Jose, California 95134 1223 U.S.A. 1225 Email: prenatar@cisco.com 1227 Randall R. Stewart 1228 Adara Networks 1229 Chapin, South Carolina 29036 1230 U.S.A. 1232 Email: randall@lakerest.net 1233 Michael Tuexen 1234 Muenster University of Applied Sciences 1235 Stegerwaldstrasse 39 1236 48565 Steinfurt, Nordrhein-Westfalen 1237 Germany 1239 Email: tuexen@fh-muenster.de 1240 URI: https://www.fh-muenster.de/fb2/personen/professoren/tuexen/