idnits 2.17.1 draft-ietf-rddp-sctp-06.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** It looks like you're using RFC 3978 boilerplate. You should update this to the boilerplate described in the IETF Trust License Policy document (see https://trustee.ietf.org/license-info), which is required now. -- Found old boilerplate from RFC 3978, Section 5.1 on line 16. -- Found old boilerplate from RFC 3978, Section 5.5 on line 762. -- Found old boilerplate from RFC 3979, Section 5, paragraph 1 on line 739. -- Found old boilerplate from RFC 3979, Section 5, paragraph 2 on line 746. -- Found old boilerplate from RFC 3979, Section 5, paragraph 3 on line 752. ** This document has an original RFC 3978 Section 5.4 Copyright Line, instead of the newer IETF Trust Copyright according to RFC 4748. ** This document has an original RFC 3978 Section 5.5 Disclaimer, instead of the newer disclaimer which includes the IETF Trust according to RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (June 27, 2006) is 6506 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'ADDIP-Draft' is mentioned on line 223, but not defined == Unused Reference: 'I-D.ietf-rddp-rdmap' is defined on line 692, but no explicit reference was found in the text ** Obsolete normative reference: RFC 2960 (Obsoleted by RFC 4960) == Outdated reference: A later version (-07) exists of draft-ietf-rddp-ddp-05 == Outdated reference: A later version (-07) exists of draft-ietf-rddp-rdmap-06 == Outdated reference: A later version (-22) exists of draft-ietf-tsvwg-addip-sctp-15 Summary: 4 errors (**), 0 flaws (~~), 7 warnings (==), 7 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Remote Direct Data Placement C. Bestler 3 Working Group R. Stewart 4 Internet-Draft Editor 5 Expires: December 29, 2006 June 27, 2006 7 Stream Control Transmission Protocol (SCTP) Direct Data Placement (DDP) 8 Adaptation 9 draft-ietf-rddp-sctp-06.txt 11 Status of this Memo 13 By submitting this Internet-Draft, each author represents that any 14 applicable patent or other IPR claims of which he or she is aware 15 have been or will be disclosed, and any of which he or she becomes 16 aware will be disclosed, in accordance with Section 6 of BCP 79. 18 Internet-Drafts are working documents of the Internet Engineering 19 Task Force (IETF), its areas, and its working groups. Note that 20 other groups may also distribute working documents as Internet- 21 Drafts. 23 Internet-Drafts are draft documents valid for a maximum of six months 24 and may be updated, replaced, or obsoleted by other documents at any 25 time. It is inappropriate to use Internet-Drafts as reference 26 material or to cite them other than as "work in progress." 28 The list of current Internet-Drafts can be accessed at 29 http://www.ietf.org/ietf/1id-abstracts.txt. 31 The list of Internet-Draft Shadow Directories can be accessed at 32 http://www.ietf.org/shadow.html. 34 This Internet-Draft will expire on December 29, 2006. 36 Copyright Notice 38 Copyright (C) The Internet Society (2006). 40 Abstract 42 This document specifies an Adaptation Layer to provide a Lower Layer 43 Protocol (LLP) service for Direct Data Placement (DDP) using the 44 Stream Control Transmission Protocol (SCTP). 46 Table of Contents 48 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 49 2. Definitions . . . . . . . . . . . . . . . . . . . . . . . . . 4 50 3. Conventions . . . . . . . . . . . . . . . . . . . . . . . . . 6 51 4. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 7 52 4.1. Motivation . . . . . . . . . . . . . . . . . . . . . . . . 7 53 4.2. Overview . . . . . . . . . . . . . . . . . . . . . . . . . 7 54 5. Data Formats . . . . . . . . . . . . . . . . . . . . . . . . . 8 55 5.1. Adaptation Layer Indicator . . . . . . . . . . . . . . . . 8 56 5.2. Payload Data Chunks . . . . . . . . . . . . . . . . . . . 8 57 5.2.1. DDP Source Sequence Number (DDP-SSN) . . . . . . . . . 9 58 5.2.2. DDP Segment Chunk . . . . . . . . . . . . . . . . . . 9 59 5.2.3. DDP Stream Session Control . . . . . . . . . . . . . . 10 60 6. DDP Stream Sessions . . . . . . . . . . . . . . . . . . . . . 11 61 6.1. Sequencing . . . . . . . . . . . . . . . . . . . . . . . . 11 62 6.2. Legal Sequence: Active/Passive Session Accepted . . . . . 11 63 6.3. Legal Sequence: Active/Passive Session Rejected . . . . . 12 64 6.4. Legal Sequence: Active/Passive Session Non-ULP Rejected . 12 65 6.5. ULP Specific Sequencing . . . . . . . . . . . . . . . . . 13 66 6.6. Other Sequencing Rules . . . . . . . . . . . . . . . . . . 13 67 7. SCTP Endpoints . . . . . . . . . . . . . . . . . . . . . . . . 14 68 7.1. Adaptation Layer Indication Restriction . . . . . . . . . 14 69 7.2. Multihoming Implications . . . . . . . . . . . . . . . . . 14 70 8. Number of Streams . . . . . . . . . . . . . . . . . . . . . . 15 71 9. Fragmentation . . . . . . . . . . . . . . . . . . . . . . . . 16 72 10. Sequenced Unordered Operation . . . . . . . . . . . . . . . . 17 73 11. Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . 18 74 11.1. Association Initialization . . . . . . . . . . . . . . . . 18 75 11.2. Chunk Bundling . . . . . . . . . . . . . . . . . . . . . . 18 76 11.3. Association Termination . . . . . . . . . . . . . . . . . 19 77 12. IANA considerations . . . . . . . . . . . . . . . . . . . . . 20 78 13. Security Considerations . . . . . . . . . . . . . . . . . . . 21 79 14. Contributors . . . . . . . . . . . . . . . . . . . . . . . . . 22 80 15. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 23 81 16. Normative references . . . . . . . . . . . . . . . . . . . . . 23 82 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 24 83 Intellectual Property and Copyright Statements . . . . . . . . . . 25 85 1. Introduction 87 This document describes a method to adapt Direct Data Placement 88 [I-D.ietf-rddp-ddp] to Stream Control Transmission Protocol (SCTP) 89 [RFC2960]. 91 Some implementations may include this adaptation layer within their 92 SCTP implementations to obtain maximum performance but the behavior 93 of SCTP will be unaffected. An SCTP Layer used solely by this 94 adaptation layer is able to take certain optimizations based on the 95 limited subset of SCTP capabilities used. In order to allow 96 optimization for these implementations we specify the use of the new 97 adaptation layer indication as defined in [I-D.ietf-tsvwg-addip-sctp] 99 2. Definitions 101 DDP - See Direct Data Placement Protocol. 103 DDP Endpoint - The logical sender/receiver of DDP Segments. An SCTP 104 Stream pair is not assumed to have a DDP Endpoint. DDP Segments 105 may only be sent once a DDP Endpoint has been assigned to an SCTP 106 Stream pair by a local interface. 108 DDP Source Stream Sequence (DDP-SSN) - A stream specific sequence 109 number assigned by the Adaptation Layer for each SCTP Data Chunk 110 sent. This is the order that chunks were submitted to SCTP, no 111 matter what order they are actually sent or received in. 113 DDP Segment - The smallest unit of data transfer for the DDP 114 protocol. It includes a DDP Header and ULP Payload (if present). 115 A DDP Segment should be sized to fit within the Lower Layer 116 Protocol MULPDU. 118 DDP Segment Chunk - An SCTP Payload Data Chunk that encapsulates the 119 DDP Stream Sequence (DDP-SSN) and a DDP Segment. 121 DDP Stream - a sequence of DDP Segments whose ordering is defined by 122 the LLP. For SCTP, a DDP Stream maps directly to a bi-directional 123 pair of SCTP streams with the same Stream IDs. Note that DDP has 124 no ordering guarantees between DDP Streams. 126 DDP Stream Session - A single pairing of DDP Endpoints over a DDP 127 Stream that lasts from a Initiation message through the 128 Termination message(s). 130 DDP Stream Session Control Message - DDP Stream Session Control 131 messages are used to control the association of the DDP Endpoint 132 with the DDP Stream. 134 Direct Data Placement Protocol (DDP) - A wire protocol that supports 135 Direct Data Placement by associating explicit memory buffer 136 placement information with the LLP payload units. 138 Protection Domain - A common local interface convention to control 139 which Steering Tags (STags) are valid with which DDP Endpoints. 140 Under this convention both the Steering Tag and DDP Endpoint are 141 created within the context of a Protection Domain, and the 142 Steering Tag may only be enabled for DDP Endpoints created under 143 the same Protection Domain. 145 RDMA - Remote Direct Memory Access. 147 RNIC - RDMA Network Interface Card. 149 SCTP association - A protocol relationship between two SCTP 150 endpoints. An SCTP association supports multiple SCTP streams. 152 SCTP Data Chunk - An SCTP Chunk used to convey Payload Data. There 153 can be multiple Chunks within each SCTP packet. Other Chunks are 154 used to control the SCTP Association. 156 SCTP endpoint - The logical sender/receiver of SCTP packets. On a 157 multi-homed host, an SCTP endpoint is represented to its peers as 158 a combination of a SCTP port number and a set of eligible 159 destination transport addresses to which SCTP packets can be sent. 161 SCTP Stream - A uni-directional logical channel established from one 162 to another associated SCTP endpoint. There can be multiple SCTP 163 Streams within each SCTP association. An SCTP Stream is used to 164 form one direction of a DDP stream. 166 Transmission Sequence Number (TSN) - A 32-bit sequence number used 167 internally by SCTP. One TSN is attached to each chunk containing 168 user data to permit the receiving SCTP endpoint to acknowledge its 169 receipt and detect duplicate deliveries. 171 3. Conventions 173 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 174 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 175 document are to be interpreted as described in RFC2119 [RFC2119]. 177 4. Introduction 179 4.1. Motivation 181 This document specifies an Adaptation Layer which fulfills the 182 requirements of a Lower Layer Protocol (LLP) for DDP using a specific 183 subset of SCTP capabilities. 185 The defined protocol is intended to be implementable over existing 186 SCTP stacks, while clearly defining what portions of SCTP are 187 required to enable an implementation to be optimized specifically to 188 support DDP. 190 4.2. Overview 192 The Adaptation Layer uses a pair of like-numbered SCTP Streams within 193 an SCTP Association to provide a reliable DDP Stream between two DDP 194 Endpoints. Except as specifically noted, each DDP Segment submitted 195 by the DDP Layer is encoded as a single unordered SCTP Data Chunk. 196 In addition to the DDP Segment the Data Chunk also contains a 197 sequence number (DDP-SSN) that reflects the order that DDP submitted 198 the segments in for that stream. 200 A DDP Stream Session is defined by DDP Stream Session Control Chunks 201 that manage the state of the DDP Stream Session. These Chunks 202 dynamically bind DDP Endpoints to the DDP Stream Session, and DDP 203 Segment Chunks are used to reliably deliver DDP Segments with the 204 session. 206 5. Data Formats 208 5.1. Adaptation Layer Indicator 210 The DDP/SCTP Adaptation Layer uses all streams within an SCTP 211 association. An SCTP Association that has had the DDP Adaptation 212 Indication negotiated will carry only SCTP Data Chunks as defined in 213 this document. 215 It is presumed that the handling of incoming data chunks for DDP 216 enabled associations is sufficiently different than for routine SCTP 217 associations that it is undesirable to require support for mixing DDP 218 and non-DDP streams in a single association. More than a single 219 association is required if an application desires to utilize both DDP 220 and non-DDP traffic with the same remote host. 222 We define a adaptation indication which MUST appear in the INIT or 223 INIT-ACK with the following format as defined in [ADDIP-Draft] 224 [I-D.ietf-tsvwg-addip-sctp] 226 0 1 2 3 227 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 228 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 229 | Type =0xC006 | Length = Variable | 230 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 231 | Adaptation Indication | 232 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 234 Adaptation Indication: 236 The following values are suggested for DDP in this document, but 237 the final value will be assigned by IANA: 239 DDP - 0x00000001 241 5.2. Payload Data Chunks 243 The DDP SCTP adaptation uses two types of SCTP Payload Data Chunks, 244 differentiated by the Payload Protocol Identifier: 246 DDP Segment Chunks are used to reliably deliver DDP Segments sent 247 between DDP Endpoints. 249 DDP Stream Session Control Messages are used to establish and 250 teardown DDP Stream Sessions, specifically by controlling the 251 binding of DDP endpoints with SCTP streams. 253 Payload Protocol Identifier: 255 The following value are defined for DDP in this document, 256 but the final values will be assigned by IANA: 258 DDP Segment Chunk - 0x00000010 259 DDP Stream Session Control - 0x00000011 261 5.2.1. DDP Source Sequence Number (DDP-SSN) 263 All SCTP Payload Data Chunks used by this Adaptation layer include a 264 DDP Source Sequence Number (DDP-SSN). The DDP-SSN tracks the 265 sequence the messages were submitted to the SCTP layer for the SCTP 266 stream in use. The DDP-SSN MUST have the same value that the SCTP 267 Stream Sequence Number (SSN) would have been assigned had ordered 268 SCTP Payload Data Chunks been used rather than unordered. 270 The rationale for specifying the DDP-SSN is as follows: 272 The SCTP Stream Sequence Number (SSN) is not suitable for this 273 purpose, because all messages defined by this document use 274 unordered Payload Data Chunks to ensure prompt delivery from the 275 receiving SCTP layer. 277 The SCTP Transmission Sequence Number (TSN) is not suitable for 278 determine the original order of Data Chunks within a stream. The 279 sending SCTP layer is allowed to optimize the transmission 280 sequence of unordered Data Chunks to encourage Chunk Bundling, or 281 other purposes. 283 5.2.2. DDP Segment Chunk 285 0 1 2 3 286 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 287 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 288 | DDP-SSN | DDP Segment | 289 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 290 | | 291 | ... | 292 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 294 DDP Segments are as defined in [I-D.ietf-rddp-ddp]. The DDP Segment 295 Chunk serves the same purpose as the MPA Upper Layer PDU (MULPDU) in 296 that it carries DDP Segments over a reliable protocol with added 297 sequencing information. 299 5.2.3. DDP Stream Session Control 301 0 1 2 3 302 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 303 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 304 | DDP-SSN | Function Code | 305 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 306 | Private Data (Dependent on Function Code) | 307 | ... | 308 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 310 The following function code values are defined for DDP in 311 this document: 313 DDP Stream Session Initiate - 0x001 314 DDP Stream Session Accept - 0x002 315 DDP Stream Session Reject - 0x003 316 DDP Stream Session Terminate - 0x004 318 ULP supplied Private Data MUST be included for DDP Stream Session 319 Initiate DDP Stream Session Accept and DDP Stream Session Reject 320 messages. However, the ULP supplied Private DATA MAY be of zero 321 length. 323 Private Data length MUST NOT exceed 512 bytes in any message. 325 Private Data MUST NOT be included in the DDP Stream Session Terminate 326 message. 328 Received DDP Stream Session Control messages SHOULD be reported to 329 the ULP. If reported, any supplied Private Data MUST be available 330 for the ULP to examine. 332 The DDP/SCTP adaptation layer MAY limit the number of Session 333 Initiate requests that it has submitted to the ULP. When a DDP 334 Stream Session Initiate cannot be forwarded to the ULP due to such a 335 limit the adaptation layer MUST respond with a DDP Stream Session 336 Terminate message. 338 6. DDP Stream Sessions 340 A DDP Endpoint is the logical sender/receiver of DDP Segments. A DDP 341 Stream connects two DDP Endpoints using a matched pair of SCTP 342 Streams having the same SCTP Stream Identifiers. 344 A DDP Stream Session defines the sequence of Data Chunks exchanged 345 between two DDP Endpoints over a DDP Stream that has a distinct 346 beginning and end as defined in the following section. Data Chunks 347 from one DDP Stream Session are never carried over to the next 348 session. Each Data Chunk unambiguously belongs to exactly one 349 session. The DDP Source Sequence Numbers assigned to the Data Chunks 350 for a session MUST NOT have any gaps. 352 The local interface MAY dynamically associate a DDP Endpoint with the 353 DDP Stream based upon the initial exchanges of a DDP Session, and 354 dynamically terminate that association at the session's end. 355 Alternately a specialized local interface could simply statically map 356 DDP Endpoints to DDP Streams. 358 Conventionally local interfaces for RDMA have deferred the selection 359 of the DDP Endpoint until after the ULP decides to accept an RDMA 360 connection request. But that is a local interface choice and not a 361 wire protocol requirement. 363 A DDP Stream is associated with at most one Protection Domain during 364 a single DDP Stream Session. On the passive side the association is 365 typically deferred until the DDP Stream Session Accept message. 367 6.1. Sequencing 369 The DDP Source Sequence Number(DDP-SSN) is reset to zero at the 370 beginning of each DDP Stream Session. 372 The normative sequence for considering Payload Data Chunks within a 373 given session is based upon each Data Chunk's DDP-SSN. When 374 considered in this normative sequence, all sessions MUST conform to 375 one the patterns defined in this section. 377 If the adaptation layer receives a Payload Data Chunk that conforms 378 to none of the enumerated legal patterns the DDP Stream Session MUST 379 be terminated. 381 6.2. Legal Sequence: Active/Passive Session Accepted 383 In this DDP Stream Session sequence one DDP Endpoint assumes the 384 active role in requesting a DDP Stream Session, which the other side 385 accepts. 387 Active Side sends a DDP Stream Session Initiate message. 389 Passive Side sends a DDP Stream Session Accept message. 391 Each side may then send zero or more DDP Segments with increasing 392 DDP-SSNs, subject to any flow control imposed by other protocol 393 layers. 395 The final User Data Chunk for each side MAY be a DDP Stream 396 Terminate. At least one side MUST send a DDP Stream Terminate. 397 Note that this would follow any RDMAP Terminate message, which to 398 the Adaptation layer is simply another DDP Segment. 400 6.3. Legal Sequence: Active/Passive Session Rejected 402 DDP Stream Sessions allow each party to send a single non-payload 403 message before the other end commits specific resources to the 404 session. This allow each end to determine which resources are to be 405 used, and how they are to be configured, or even if the session 406 should be granted. 408 These decision MAY be influenced by the need to assign a specific 409 Protection Domain, to determine how many RDMA Read Credits are 410 required, or to determine now many receive operations the ULP should 411 enable. 413 Because of these, or other, factors the Passive side MAY choose to 414 reject a DDP Stream Session Request. This results in the following 415 legal sequence: 417 Active Side sends a DDP Stream Session Initiate message. 419 Passive Side sends a DDP Stream Session Reject message. 421 An DDP Stream Session Reject message MUST NOT be sent unless the 422 rejection is at the direction of the ULP. 424 6.4. Legal Sequence: Active/Passive Session Non-ULP Rejected 426 Acceptance or rejection of DDP Stream Session Initiate messages 427 SHOULD be under the control of the ULP. This MAY require passing an 428 event to the ULP. There MUST be a finite limit on the number of such 429 requests that are pending a ULP decision. When more session requests 430 are received the passive side MUST respond to the Initiate message 431 with a DDP Stream Terminate Message. 433 6.5. ULP Specific Sequencing 435 An implementation MAY choose to support additional ULP specific 436 sequences, but MUST NOT do so unless requested to do so by the ULP. 438 A defined ULP MUST be able to operate using only the defined 439 mandatory session sequences. Any additional sequences must be used 440 only for optional optimizations. 442 6.6. Other Sequencing Rules 444 A DDP Stream Session Control message MUST NOT be sent if it may be 445 received before a prior DDP Stream Session Control message within the 446 same DDP Stream Session. 448 An active side of a DDP Stream Session MUST NOT send a DDP Segment 449 that might be received before the DDP Stream Session Initiate 450 message. 452 This MAY be determined by SCTP acking of the Data Chunk used to carry 453 the DDP Stream Session Initiate message, or by receipt of a 454 responsive DDP Stream Session Control message. 456 A DDP Stream MUST NOT be re-used for another DDP Stream Session while 457 any Data Chunk from a prior session might be outstanding. 459 7. SCTP Endpoints 461 7.1. Adaptation Layer Indication Restriction 463 The local interface MUST allow the ULP to specify an SCTP endpoint to 464 use a specific Adaptation Indication. It MAY require the ULP to do 465 so. 467 Once an endpoint decides on its acceptable Adaptation Indication(s), 468 it SHOULD terminate all requests to establish an association with any 469 different Adaptation Indication. 471 An SCTP implementation MAY choose to accept association requests for 472 a given SCTP endpoint only until one association for the endpoint has 473 been established. At that point it MAY choose to restrict all 474 further associations for the same endpoint to use the same Adaptation 475 Indication. 477 7.2. Multihoming Implications 479 SCTP allows an SCTP endpoint to be associated with multiple IP 480 addresses, potentially representing different interface devices. 481 Distribution of the logic for a single DDP stream across multiple 482 input devices can be very undesirable, resulting in complex cache 483 coherency challenges. Therefore the local interface MAY restrict 484 DDP-enabled SCTP endpoints to a single IP address, or to a set of IP 485 addresses that are all assigned to the same input device ("RNIC"). 487 The default binding of a DDP enabled SCTP endpoint SHOULD NOT cover 488 more than a single IP address unless doing so results in no 489 additional bus traffic or duplication of memory registration 490 resources. This will frequently result in a different default than 491 for SCTP endpoints that are not DDP enabled. 493 Even when multi-homing is supported, ULPs are cautioned that they 494 SHOULD NOT use ULP control of the source address in attempt to load- 495 balance a stream across multiple paths. A receiving DDP/SCTP 496 implementation that chooses to support multi-homing SHOULD optimize 497 its design on the assumption that multi-homing will be used for 498 network fault tolerance, and not to load-balance between paths. This 499 is consistent with recommended SCTP practices. 501 8. Number of Streams 503 DDP Streams are bidirectional. They are always composed by pairing 504 the inbound and outbound SCTP streams with the same SCTP Stream 505 Identifier. 507 The adaptation layer should request the maximum number of SCTP stream 508 it will wish to use over the lifetime of the association. SCTP 509 streams must still be bound to DDP Endpoints, and a DDP enabled SCTP 510 association does not support ordered Data Chunks. Therefore the mere 511 existence of an SCTP stream is unlikely to require significant 512 supporting resources. 514 This mapping uses an SCTP association to carry one or more DDP 515 Steams. Each DDP Stream will be mapped to a pair of SCTP streams 516 with the same SCTP stream number. The adaptation MUST initialize all 517 of its SCTP associations with the same number of inbound and outbound 518 streams. 520 9. Fragmentation 522 A DDP/SCTP Receiver already deals with fragmentation at both the IP 523 and DDP Layers. Therefore use of SCTP layer segmenting will be 524 avoided for most cases. 526 As a Lower Layer Protocol (LLP) for DDP, the SCTP adaptation layer 527 MUST inform the DDP layer of the maximum DDP Segment size that will 528 be supported. This should be the largest value that can be supported 529 without use of IP or SCTP fragmentation, or 516 bytes, whichever is 530 larger. 532 A minimum of 516 bytes is required to allow a DDP Stream Session 533 Control Message with 512 bytes of Private Data. 535 SCTP data chunk fragmentation MUST NOT be used unless the alternative 536 is IP fragmentation. 538 The SCTP adaptation layer SHOULD set the maximum DDP Segment size 539 below the theoretical maximum in order to allow bundling of Control 540 Chunks in the same SCTP packet. 542 The SCTP adaptation layer MUST reject DDP Segments that are larger 543 than the maximum size specified. 545 10. Sequenced Unordered Operation 547 The Adaptation layer MUST use the Unordered option on all Data Chunks 548 (U Flag set to one). The SCTP Layer is expected to deliver unordered 549 Data Chunks without delay. 551 Because DDP employs unordered SCTP delivery, the receiver MUST NOT 552 rely upon the SCTP Transmission Sequence Number (TSN) to imply 553 ordering of DDP Segments. The fact that the SCTP Data Chunk for a 554 DDP Segment is prior the cumulative ack point does not guarantee that 555 all prior DDP segments have been placed. The SCTP sender is not 556 obligated to transmit unordered Data Chunks in the order presented. 558 The DDP-SSN can be used without special logic to determine the 559 submission sequence when the maximum number of in-flight messages is 560 less than 32768. This also applies if the sending SCTP accepts no 561 more than 32767 Data Chunks for a single stream without assigning 562 TSNs. 564 If SCTP does accept more than 32768 Data chunks for a single stream 565 without assigning TSNs, the sending DDP must simply refrain from 566 sending more than 32767 Data Chunks for a single stream without 567 acknowledgment. Note that it MUST NOT rely upon ULP flow control for 568 this purpose. Typical ULP flow control will deal exclusively with 569 untagged messages, not with DDP segments. 571 The receiver MAY perform a validity check on received DDP-SSNs to 572 ensure that any gap could be accounted for by unreceived Data Chunks. 573 Implementations SHOULD NOT allocate resources on the assumption that 574 DDP-SSNs are valid without first performing such a validity check. 575 An invalid DDP-SSN MAY result in termination of the DDP Stream. 577 11. Procedures 579 11.1. Association Initialization 581 At the startup of an association, a DDP/SCTP Adaptation Layer MUST 582 include an adaptation layer indication in its INIT or INIT-ACK (as 583 defined in Section 5.1. After the exchange of the initial first two 584 SCTP chunks (INIT and INIT-ACK), an endpoint MUST verify and inspect 585 the adaptation indication and compare it to the following table to 586 determine proper action. 588 Indication | Action 589 type | 591 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 592 | This indicates that the peer DOES NOT 593 NONE | support ANY DDP or RDMA adaptation and thus 594 | RDMA and DDP procedures MUST NOT be 595 | performed upon this association. 596 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 597 | This indicates that the peer DOES support 598 DDP | the DDP/SCTP Adaptation Layer defined here. 599 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 600 | This indicates that the peer DOES NOT 601 ANY-OTHER | support the DDP adaptation and thus 602 Indication | DDP procedures MUST NOT be performed 603 | upon this association. 605 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 607 An implementation MAY require that all associations for a given SCTP 608 endpoint be placed in the same mode. 610 The local interface MAY allow the ULP to accept only requests to 611 establish an association in a specified mode. 613 11.2. Chunk Bundling 615 SCTP allows multiple Data Chunks to be bundled in a single SCTP 616 packet. Data chunks containing DDP Segments with untagged messages 617 SHOULD NOT be delayed to facilitate bundling. Data chunks containing 618 DDP Segments with tagged messages will generally be full sized, and 619 hence not subject to bundling. However partial size tagged messages 620 MAY be delayed, as that they are frequently followed by a short 621 untagged message. 623 11.3. Association Termination 625 Termination of an SCTP Association due to errors should be handled at 626 the SCTP layer. The RDMAP defined RDMAP Terminate Message SHOULD NOT 627 be sent on each DDP Stream when a determination has been made to 628 terminate an SCTP association. Sending that message on each SCTP 629 stream could severely delay the termination of the association. 631 The local interface SHOULD notify all consumers of DDP streams when 632 the underlying SCTP stream has been terminated. 634 Other RDMAP defined Terminate Messages MUST be generated as specified 635 when a DDP Stream is terminated. Note that with the SCTP mapping, 636 termination of a DDP Stream does not mandate termination of the 637 Association. 639 12. IANA considerations 641 This document defines a new SCTP Adaptation Layer Indication 642 codepoint. [I-D.ietf-tsvwg-addip-sctp] creates the registry from 643 which this codepoint is to be assigned. Any unallocated codepoint 644 may be assigned. The value of one is suggested. 646 This document also defines two new SCTP Payload Protocol Identifier 647 (PPIDs). RFC2960 [RFC2960] creates the registry from which these 648 identifiers are to be assigned. The Payload Protocol Identifiers 649 allocated need to be unique, but have no other requirements. The 650 following values are suggested: 652 DDP Segment Chunk - 0x00000010 653 DDP Stream Session Control - 0x00000011 655 13. Security Considerations 657 Any direct placement of memory could pose a significant security risk 658 if adequate local controls are not provided. These threats are 659 addressed in the appropriate DDP [I-D.ietf-rddp-ddp], RDMA [I-D.ietf- 660 rddp-rdmap] or Security [I-D.ietf-rddp-security] drafts. This 661 document does not add any additional security risks over those found 662 in RFC2960 [RFC2960]. 664 14. Contributors 666 Many thanks to our contributors who have spent many hours reading and 667 reviewing keeping us straight: Sukanta Ganguly an independent 668 consultant, Vivek Kashyap of IBM, Jim Pinkerton of Microsoft, and 669 Hemal Shah of Broadcom. Thanks for all your hard work. 671 15. Acknowledgments 673 The authors would like to thank the following people that have 674 provided comments and input: Stephen Bailey, David Black, Douglas 675 Otis, Allyn Romanow and Jim Williams. 677 16. Normative references 679 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 680 Requirement Levels", BCP 14, RFC 2119, March 1997. 682 [RFC2960] Stewart, R., Xie, Q., Morneault, K., Sharp, C., 683 Schwarzbauer, H., Taylor, T., Rytina, I., Kalla, M., 684 Zhang, L., and V. Paxson, "Stream Control Transmission 685 Protocol", RFC 2960, October 2000. 687 [I-D.ietf-rddp-ddp] 688 Shah, H., "Direct Data Placement over Reliable 689 Transports", draft-ietf-rddp-ddp-05 (work in progress), 690 July 2005. 692 [I-D.ietf-rddp-rdmap] 693 Recio, R., "A Remote Direct Memory Access Protocol 694 Specification", draft-ietf-rddp-rdmap-06 (work in 695 progress), June 2006. 697 [I-D.ietf-rddp-security] 698 Pinkerton, J., "DDP/RDMAP Security", 699 draft-ietf-rddp-security-10 (work in progress), June 2006. 701 [I-D.ietf-tsvwg-addip-sctp] 702 Stewart, R., "Stream Control Transmission Protocol (SCTP) 703 Dynamic Address Reconfiguration", 704 draft-ietf-tsvwg-addip-sctp-15 (work in progress), 705 June 2006. 707 Authors' Addresses 709 Caitlin Bestler 710 Editor 711 Broadcom Corporation 712 16215 Alton Parkway 713 P.O. Box 57013 714 Irvine, CA 92619-7013 715 USA 717 Phone: 949-926-6383 718 Email: caitlinb@broadcom.com 720 Randall R. Stewart 721 Editor 722 Cisco Systems, Inc. 723 Forest Drive 724 Columbia, SC 29036 725 USA 727 Phone: +1-815-342-5222 728 Email: rrs@cisco.com 730 Intellectual Property Statement 732 The IETF takes no position regarding the validity or scope of any 733 Intellectual Property Rights or other rights that might be claimed to 734 pertain to the implementation or use of the technology described in 735 this document or the extent to which any license under such rights 736 might or might not be available; nor does it represent that it has 737 made any independent effort to identify any such rights. Information 738 on the procedures with respect to rights in RFC documents can be 739 found in BCP 78 and BCP 79. 741 Copies of IPR disclosures made to the IETF Secretariat and any 742 assurances of licenses to be made available, or the result of an 743 attempt made to obtain a general license or permission for the use of 744 such proprietary rights by implementers or users of this 745 specification can be obtained from the IETF on-line IPR repository at 746 http://www.ietf.org/ipr. 748 The IETF invites any interested party to bring to its attention any 749 copyrights, patents or patent applications, or other proprietary 750 rights that may cover technology that may be required to implement 751 this standard. Please address the information to the IETF at 752 ietf-ipr@ietf.org. 754 Disclaimer of Validity 756 This document and the information contained herein are provided on an 757 "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS 758 OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET 759 ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, 760 INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE 761 INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED 762 WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 764 Copyright Statement 766 Copyright (C) The Internet Society (2006). This document is subject 767 to the rights, licenses and restrictions contained in BCP 78, and 768 except as set forth therein, the authors retain all their rights. 770 Acknowledgment 772 Funding for the RFC Editor function is currently provided by the 773 Internet Society.