idnits 2.17.1 draft-ietf-rddp-sctp-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** It looks like you're using RFC 3978 boilerplate. You should update this to the boilerplate described in the IETF Trust License Policy document (see https://trustee.ietf.org/license-info), which is required now. -- Found old boilerplate from RFC 3978, Section 5.1 on line 23. -- Found old boilerplate from RFC 3978, Section 5.5 on line 715. -- Found old boilerplate from RFC 3979, Section 5, paragraph 1 on line 692. -- Found old boilerplate from RFC 3979, Section 5, paragraph 2 on line 699. -- Found old boilerplate from RFC 3979, Section 5, paragraph 3 on line 705. ** This document has an original RFC 3978 Section 5.4 Copyright Line, instead of the newer IETF Trust Copyright according to RFC 4748. ** This document has an original RFC 3978 Section 5.5 Disclaimer, instead of the newer disclaimer which includes the IETF Trust according to RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The abstract seems to contain references ([2], [3], [4], [RDMA-Draft], [DDP-Draft]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (August 14, 2005) is 6830 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: 'RDMA-Draft' on line 603 -- Looks like a reference, but probably isn't: 'DDP-Draft' on line 603 -- Looks like a reference, but probably isn't: 'ADDIP-Draft' on line 177 == Unused Reference: '5' is defined on line 631, but no explicit reference was found in the text ** Obsolete normative reference: RFC 2960 (ref. '2') (Obsoleted by RFC 4960) == Outdated reference: A later version (-07) exists of draft-ietf-rddp-ddp-05 == Outdated reference: A later version (-07) exists of draft-ietf-rddp-rdmap-05 == Outdated reference: A later version (-32) exists of draft-ietf-tsvwg-sctpsocket-10 ** Downref: Normative reference to an Informational draft: draft-ietf-tsvwg-sctpsocket (ref. '5') == Outdated reference: A later version (-22) exists of draft-ietf-tsvwg-addip-sctp-12 Summary: 6 errors (**), 0 flaws (~~), 8 warnings (==), 10 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Remote Direct Data Placement R. Stewart 3 Working Group Cisco Systems, Inc. 4 Internet-Draft C. Bestler 5 Expires: February 15, 2006 Broadcom 6 H. Shah 7 Intel Corporation 8 V. Kashyap 9 IBM 10 S. Ganguly 11 Consultant 12 August 14, 2005 14 Stream Control Transmission Protocol (SCTP) Remote Direct Memory Access 15 (RDMA) Direct Data Placement (DDP) Adaptation 16 draft-ietf-rddp-sctp-02.txt 18 Status of this Memo 20 By submitting this Internet-Draft, each author represents that any 21 applicable patent or other IPR claims of which he or she is aware 22 have been or will be disclosed, and any of which he or she becomes 23 aware will be disclosed, in accordance with Section 6 of BCP 79. 25 Internet-Drafts are working documents of the Internet Engineering 26 Task Force (IETF), its areas, and its working groups. Note that 27 other groups may also distribute working documents as Internet- 28 Drafts. 30 Internet-Drafts are draft documents valid for a maximum of six months 31 and may be updated, replaced, or obsoleted by other documents at any 32 time. It is inappropriate to use Internet-Drafts as reference 33 material or to cite them other than as "work in progress." 35 The list of current Internet-Drafts can be accessed at 36 http://www.ietf.org/ietf/1id-abstracts.txt. 38 The list of Internet-Draft Shadow Directories can be accessed at 39 http://www.ietf.org/shadow.html. 41 This Internet-Draft will expire on February 15, 2006. 43 Copyright Notice 45 Copyright (C) The Internet Society (2005). 47 Abstract 48 This document describes a method to adapt Direct Data Placement (DDP) 49 and Remote Direct Memory Access (RDMA) to Stream Control Transmission 50 Protocol (SCTP) RFC2960 [2] using a generic description found in 51 [RDMA-Draft] [4] and [DDP-Draft] [3]. 53 Table of Contents 55 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 56 1.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . 3 57 1.2 Conventions . . . . . . . . . . . . . . . . . . . . . . . 4 58 2. Data Formats . . . . . . . . . . . . . . . . . . . . . . . . . 5 59 2.1 Adaptation Layer Indicator . . . . . . . . . . . . . . . . 5 60 2.2 Payload Data Chunks . . . . . . . . . . . . . . . . . . . 5 61 2.2.1 DDP Source Sequence Number (DDP-SSN) . . . . . . . . . 6 62 2.2.2 DDP Segment Payload Data Chunk . . . . . . . . . . . . 6 63 2.2.3 DDP Stream Session Control Data Chunk . . . . . . . . 7 64 3. DDP Stream Sessions . . . . . . . . . . . . . . . . . . . . . 8 65 3.1 Sequencing . . . . . . . . . . . . . . . . . . . . . . . . 8 66 3.2 Legal Sequence: Active/Passive Session Accepted . . . . . 8 67 3.3 Legal Sequence: Active/Passive Session Rejected . . . . . 9 68 3.4 Legal Sequence: Active/Passive Session Non-ULP Rejected . 9 69 3.5 ULP Specific Sequencing . . . . . . . . . . . . . . . . . 9 70 3.6 Other Sequencing Rules . . . . . . . . . . . . . . . . . . 9 71 4. SCTP Endpoints . . . . . . . . . . . . . . . . . . . . . . . . 11 72 4.1 Adaptation Layer Indication Restriction . . . . . . . . . 11 73 4.2 Multihoming Implications . . . . . . . . . . . . . . . . . 11 74 5. Number of Streams . . . . . . . . . . . . . . . . . . . . . . 12 75 6. Fragmentation . . . . . . . . . . . . . . . . . . . . . . . . 13 76 7. Sequenced Unordered Operation . . . . . . . . . . . . . . . . 14 77 8. Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . 15 78 8.1 Association Initialization . . . . . . . . . . . . . . . . 15 79 8.2 Chunk Bundling . . . . . . . . . . . . . . . . . . . . . . 15 80 8.3 Association Termination . . . . . . . . . . . . . . . . . 16 81 9. IANA considerations . . . . . . . . . . . . . . . . . . . . . 17 82 10. Security Considerations . . . . . . . . . . . . . . . . . . 18 83 11. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . 19 84 12. Normative References . . . . . . . . . . . . . . . . . . . . 19 85 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . 19 86 Intellectual Property and Copyright Statements . . . . . . . . 21 88 1. Introduction 90 This document describes a method to adapt Direct Data Placement (DDP) 91 and Remote Direct Memory Access (RDMA) to Stream Control Transmission 92 Protocol (SCTP) RFC2960 [2] using a generic description found in 93 [RDMA-Draft] [4] and [DDP-Draft] [3] This adaption provides a method 94 for two peers to know that each side is performing DDP or RDMA thus 95 enabling hardware acceleration if available. 97 Some implementations may include this adaptation layer within their 98 SCTP implementations to obtain maximum performance but the behavior 99 of SCTP will be unaffected. In order to accomplish this we specify 100 the use of the new adaptation layer indication as defined in [ADDIP- 101 Draft] [6] 103 1.1 Definitions 105 DDP Endpoint - The logical sender/receiver of DDP Segments. An SCTP 106 Stream pair is not assumed to have a DDP Endpoint. DDP Segments 107 may only be sent once a DDP Endpoint has been assigned to an SCTP 108 Stream pair by a local interface. 110 DDP Source Stream Sequence (DDP-SSN) - A stream specific sequence 111 number assigned by the DDP layer for each SCTP Data Chunk sent. 112 Use of the SCTP Stream Sequence Number (SSN) could result in 113 ordered delivery at the receiving end. Use of unordered Data 114 Chunks indicates that the receiving SCTP layer is to deliver them 115 without delay. The DDP-SSN retains the original order the Data 116 Chunks were generated in, no matter what order they were actually 117 sent or received. 119 DDP Stream - A bi-directional pair of SCTP streams which have the 120 same SCTP stream identifier. 122 DDP Stream Session - A single pairing of DDP Endpoints over a DDP 123 Stream that lasts from a Initiation message through the 124 Termination message(s). 126 DDP Stream Session Control - DDP Stream Session Control messages are 127 used to control the association of the DDP Endpoint with the DDP 128 Stream. 130 RDMA - Remote Direct Memory Access. 132 RNIC - RDMA Network Interface Card. 134 SCTP association - A protocol relationship between two SCTP 135 endpoints. An SCTP association supports multiple SCTP streams. 137 SCTP Data Chunk - An SCTP Chunk used to convey Payload Data. There 138 can be multiple Chunks within each SCTP packet. Other Chunks are 139 used to control the SCTP Association. 141 SCTP endpoint - The logical sender/receiver of SCTP packets. On a 142 multi-homed host, an SCTP endpoint is represented to its peers as 143 a combination of a SCTP port number and a set of eligible 144 destination transport addresses to which SCTP packets can be sent. 146 SCTP Stream - A uni-directional logical channel established from one 147 to another associated SCTP endpoint. There can be multiple SCTP 148 Streams within each SCTP association. An SCTP Stream is used to 149 form one direction of a DDP stream. 151 Transmission Sequence Number (TSN) - A 32-bit sequence number used 152 internally by SCTP. One TSN is attached to each chunk containing 153 user data to permit the receiving SCTP endpoint to acknowledge its 154 receipt and detect duplicate deliveries. 156 1.2 Conventions 158 The keywords MUST, MUST NOT, REQUIRED, SHALL, SHALL NOT, SHOULD, 159 SHOULD NOT, RECOMMENDED, NOT RECOMMENDED, MAY, and OPTIONAL, when 160 they appear in this document, are to be interpreted as described in 161 RFC2119 [1]. 163 2. Data Formats 165 2.1 Adaptation Layer Indicator 167 This mapping places an entire SCTP association into a specific DDP 168 mode: DDP or DDP+RDMA. It is presumed that the handling of incoming 169 data chunks for DDP enabled associations is sufficiently different 170 than for routine SCTP associations that it is undesirable to mix DDP 171 and non-DDP streams in a single association. An application that 172 needs to mix DDP and non-DDP traffic must use use different 173 associations with different adaptation indications for the DDP 174 traffic and non-DDP traffic. 176 We define a adaption indication which MUST appear in the INIT or 177 INIT-ACK with the following format as defined in [ADDIP-Draft] [6] 179 0 1 2 3 180 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 181 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 182 | Type =0xC006 | Length = Variable | 183 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 184 | Adaptation Indication | 185 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 187 Adaptation Indication: 189 The following values are defined for DDP in this document: 191 DDP - 0x00000001 192 DDP+RDMA - 0x00000002 194 2.2 Payload Data Chunks 196 After the SCTP association has been established, all DDP relevant 197 messages are encoded as Payload Data Chunks. Each includes a SCTP 198 Stream identifier, a Transmissions Sequence Number (TSN), a Payload 199 Protocol Identifier, the chunk length and the payload data bytes. 201 The DDP SCTP adaptation uses two types of Payload Data Chunks, 202 differentiated by the Payload Protocol Identifier: 204 DDP Segments are use to for messages sent between DDP Endpoints. 205 Each DDP Segment is exactly contained in one SCTP payload data 206 chunk with the payload protocol identifier 0x00000001 207 DDP Stream Session messages are used to control the binding of DDP 208 endpoints with SCTP streams. 210 Payload Protocol Identifier: 212 The following value are defined for DDP in this document: 214 DDP Segment - 0x00000001 215 DDP Stream Session Control - 0x00000002 217 2.2.1 DDP Source Sequence Number (DDP-SSN) 219 All Payload Data Chunks include a DDP Source Sequence Number (DDP- 220 SSN) that tracks the sequence the messages were submitted to the SCTP 221 layer. This field MUST be maintained by the adaptation layer. It is 222 initialized to 1 for each stream at the beginning of each DDP Stream 223 Session. DDP-SSN is increased by one (modulo 65536) for each DDP 224 segment submitted to the SCTP layer 226 The SCTP Stream Sequence Number (SSN) is not suitable for this 227 purpose, because all messages defined by this document use unordered 228 Payload Data Chunks to ensure prompt delivery from the receiving SCTP 229 layer. 231 The SCTP Transmission Sequence Number (TSN) is not suitable for 232 determine the original order of Data Chunks within a stream. The 233 sending SCTP layer is allowed to optimize the transmission sequence 234 of unordered Data Chunks to encourage Chunk Bundling, or other 235 purposes. 237 DDP requires that an LLP deliver ordering information with each DDP 238 Segment. The SCTP Adaptation presents the DDP-SSN for this purpose. 240 2.2.2 DDP Segment Payload Data Chunk 242 0 1 2 3 243 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 244 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 245 | DDP-SSN | DDP Segment | 246 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 247 | | 248 | ... | 249 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 251 DDP Segments are as defined in [DDP-Draft]. 253 2.2.3 DDP Stream Session Control Data Chunk 255 0 1 2 3 256 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 257 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 258 | DDP-SSN | Function Code | 259 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 260 | Private Data (Dependent on Function Code) | 261 | ... | 262 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 264 The following function code values are defined for DDP in 265 this document: 267 DDP Stream Session Initiate - 0x001 268 DDP Stream Session Accept - 0x002 269 DDP Stream Session Reject - 0x003 270 DDP Stream Session Terminate - 0x004 272 ULP supplied Private Data MUST be included for DDP Stream Session 273 Initiate DDP Stream Session Accept and DDP Stream Session Reject 274 messages. However, the ULP supplied Private DATA MAY be of zero 275 length. Private Data length MUST NOT exceed 512 bytes in any 276 message. Private Data MUST NOT be included for the DDP Stream 277 Session Terminate message. 279 The length of private data is derived from the length of the Data 280 Chunk 282 Received DDP Stream Session Control messages SHOULD be reported to 283 the ULP. If reported, any supplied Private Data MUST be available 284 for the ULP to examine. 286 There MAY be a limit on the rate at which Stream Session Control 287 message can be reported to the ULP. When this rate is exceeded, or 288 when other factors prevent the message from being reported to the 289 ULP, the session MUST be terminated. 291 3. DDP Stream Sessions 293 A DDP Endpoint is the logical sender/receiver of DDP Segments. A DDP 294 Stream connects two DDP Endpoints using a matched pair of SCTP 295 Streams. 297 A DDP Stream Session defines the sequence of Data Chunks exchanged 298 between two DDP Endpoints over a DDP Stream that has a distinct 299 beginning and end. Data Chunks from one DDP Stream Session are never 300 carried over to the next session. 302 The local interface MAY associate a DDP Endpoint with the DDP Stream 303 based upon the initial exchanges of a DDP Session, and terminate that 304 association at the session's end. 306 A DDP Stream is associated with at most one Protection Domain during 307 a single DDP Stream Session. 309 3.1 Sequencing 311 The DDP Source Sequence Number(DDP-SSN) is reset to one at the 312 beginning of each DDP Stream Session. 314 The Payload Data Chunks for a given session, when sequenced by their 315 DDP-SSN, MUST follow one of the patterns defined in this section. 317 If the adaptation layer receives a Payload Data Chunk that conforms 318 to none of the enumerated legal patterns the DDP Stream Session MUST 319 be terminated. 321 3.2 Legal Sequence: Active/Passive Session Accepted 323 In this DDP Stream Session sequence one DDP Endpoint assumes the 324 active role in requesting a DDP Stream Session, which the other side 325 accepts. 327 Active Side sends a DDP Stream Session Initiate message. 329 Passive Side sends a DDP Stream Session Accept mesage. 331 Each side may then send zero or more DDP Segments with increasing 332 DDP-SSNs, subject to various layers of flow control. 334 The final User Data Chunk for each side MAY be a DDP Stream 335 Terminate. At least one side MUST send a DDP Stream Terminate. 336 Note that this would follow any RDMAP Terminate message, which to 337 this layer is simply another Payload Data Chunk. 339 3.3 Legal Sequence: Active/Passive Session Rejected 341 DDP Stream Sessions allow each party to send a single non-payload 342 message before the other end commits specific resources to the 343 session. This allow each end to determine which resources are to be 344 used, and how they are to be configured, or even if the session 345 should be granted. 347 These decision MAY be influenced by the need to assign a specific 348 Protection Domain, to determine how many RDMA Read Credits are 349 required, or to determine now many receive operations the ULP should 350 enable. 352 Because of these, or other, factors the Passive side MAY choose to 353 reject a DDP Stream Session Request. This results in the following 354 legal sequence: 356 Active Side sends a DDP Stream Session Initiate message. 358 Passive Side sends a DDP Stream Session Reject mesage. 360 An DDP Stream Session Reject message MUST NOT be sent unless the 361 rejection is at the direction of the ULP. 363 3.4 Legal Sequence: Active/Passive Session Non-ULP Rejected 365 Acceptance or rejection of DDP Stream Session Initiate messages 366 SHOULD be under the control of the ULP. This MAY require passing an 367 event to the ULP. There MUST be a finite limit on the number of such 368 requests that are pending a ULP decision. When more session requests 369 are received the passive side MUST respond to the Initiate message 370 with a DDP Stream Terminate Message. 372 3.5 ULP Specific Sequencing 374 An implementation MAY choose to support additional ULP specific 375 sequences, but MUST NOT do so unless requested to do so by the ULP. 377 A defined ULP MUST be able to operate using only the defined 378 mandatory session sequences. Any additional sequences must be used 379 only for optional optimizations. 381 3.6 Other Sequencing Rules 383 A DDP Stream Session Control message MUST NOT be sent if it may be 384 received before a prior DDP Stream Session Control message within the 385 same DDP Stream Session. 387 An active side of a DDP Stream Session MUST NOT send a DDP Segment 388 that might be received before the DDP Stream Session Initiate 389 message. 391 This MAY be determined by SCTP acking of the Data Chunk used to carry 392 the DDP Stream Session Initiate message, or by receipt of a 393 responsive DDP Stream Session Control message. 395 A DDP Stream MUST NOT be re-used for another DDP Stream Session while 396 any Data Chunk from a prior session might be outstanding. 398 4. SCTP Endpoints 400 4.1 Adaptation Layer Indication Restriction 402 The local interface MUST allow the ULP to specify an SCTP endpoint to 403 use a specific Adaptation Indication. It MAY require the ULP to do 404 so. 406 Once an endpoint decides on its acceptable Adaptation Indication(s), 407 it SHOULD terminate all requests to establish an association with any 408 different Adaptation Indication. 410 An SCTP implementation MAY choose to accept association requests for 411 a given SCTP endpoint only until one association for the endpoint has 412 been established. At that point it MAY choose to restrict all 413 further associations for the same endpoint to use the same Adaptation 414 Indication. 416 4.2 Multihoming Implications 418 SCTP allows an SCTP endpoint to be associated with multiple IP 419 addresses, potentially representing different interface devices. 420 Distribution of the logic for a single DDP stream across multiple 421 input devices can be very undesirable, resulting in complex cache 422 coherency challenges. Therefore the local interface MAY restrict 423 DDP-enabled SCTP endpoints to a single IP address, or to a set of IP 424 addresses that are all assigned to the same input device ("RNIC"). 426 The default binding of a DDP enabled SCTP endpoint SHOULD NOT cover 427 more than a single IP address unless doing so results in no 428 additional bus traffic or duplication of memory registration 429 resources. This will frequently result in a different default than 430 for SCTP endpoints that are not DDP enabled. 432 Even when multi-homing is supported, ULPs are cautioned that they 433 SHOULD NOT use ULP control of the source address in attempt to load- 434 balance a stream across multiple paths. A receiving DDP/SCTP 435 implementation that chooses to support multi-homing SHOULD optimize 436 its design on the assumption that multi-homing will be used for 437 network fault tolerance, and not to load-balance between paths. This 438 is consistent with recommended SCTP practices. 440 5. Number of Streams 442 DDP Streams are bidirectional. They are always composed by pairing 443 the inbound and outbound SCTP streams with the same SCTP Stream 444 Identifier. 446 DDP should request the maximum number of SCTP stream it will wish to 447 use over the lifetime of the association. SCTP streams must still be 448 bound to DDP Endpoints, and a DDP or DDP+RDMA enabled SCTP 449 association does not support ordered Data Chunks. Therefore the mere 450 existence of an SCTP stream is unlikely to require signifigant 451 supporting resources. 453 This mapping uses an SCTP association to carry one or more DDP 454 Steams. Each DDP Stream will be mapped to a pair of SCTP streams 455 with the same SCTP stream number. DDP MUST initialize all of its 456 SCTP associations with the same number of inbound and outbound 457 streams. 459 6. Fragmentation 461 A DDP/SCTP Receiver already must deal with fragementation at both the 462 IP and DDP Layers. Therefore use of SCTP layer segmenting will be 463 avoided for most cases. 465 As a Lower Layer Protocol (LLP) for DDP, the SCTP adaptation layer 466 MUST inform the DDP layer of the DDP Segment size that will be 467 supported. This should be the largest value that can be supported 468 without use of IP or SCTP fragmention, or 516 bytes, whichever is 469 larger. 471 SCTP Data Chunk fragmentation MUST NOT be used for the cases where IP 472 fragmentation is not required. SCTP data chunk fragmentation MAY be 473 used to avoid IP fragmentation 475 The SCTP adaptation layer SHOULD set the maximum DDP Segment size 476 below the theoretical maximum in order to allow bundling of Control 477 Chunks in the same SCTP packet. 479 The SCTP adaptation layer MUST reject user messages that are larger 480 than the maximum specified. 482 7. Sequenced Unordered Operation 484 DDP MUST use the Unordered option on all Data Chunks (U Flag set to 485 one). The SCTP Layer is expected to deliver Data Chunks to the DDP 486 layer without delay. 488 Because DDP employs unordered SCTP delivery, the receiver MUST NOT 489 rely upon the SCTP Transmission Sequence Number (TSN) to imply 490 ordering of DDP Segments. The fact that the SCTP Data Chunk for a 491 DDP Segment is prior the cumulative ack point does not guarantee that 492 all prior DDP segments have been placed. The SCTP sender is not 493 obligated to transmit unordered Data Chunks in the order presented. 495 The DDP-SSN can be used without special logic to determine the 496 submission sequence when the maximum number of in-flight messages is 497 less than 32768. This also applies if the sending SCTP accepts no 498 more than 32767 Data Chunks for a single stream without assigning 499 TSNs. 501 If SCTP does accept more than 32768 Data chunks for a single stream 502 without assigning TSNs, the sending DDP must simply refrain from 503 sending more than 32767 Data Chunks for a single stream without 504 acknowledgement. Note that it MUST NOT rely upon ULP flow control 505 for this purpose. Typical ULP flow control will deal exclusively 506 with untagged DDP Messages, not with DDP segments. 508 The receiving DDP implementation MAY perform a validity check on 509 received DDP-SSNs to ensure that any gap could be accounted for by 510 unreceived Data Chunks. Implementations are advised against 511 allocating resources on the assumption that DDP-SSNs are valid 512 without first performing such a validtity check. An invalid DDP-SSN 513 MAY result in termination of the DDP Stream. 515 8. Procedures 517 8.1 Association Initialization 519 At the startup of an association, an endpoint wishing to perform DDP, 520 RDMA, or DDP+RDMA placement MUST include an adaptation layer 521 indication in its INIT or INIT-ACK (as defined in Section 2.1. After 522 the exchange of the initial first two SCTP chunks (INIT and INIT- 523 ACK), an endpoint MUST verify and inspect the adaptation indication 524 and compare it to the following table to determine proper action. 526 Indication | Action 527 type | 529 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 530 | This indicates that the peer DOES NOT 531 NONE | support ANY DDP or RDMA adaption and thus 532 | RDMA and DDP procedures MUST NOT be 533 | performed upon this association. 534 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 535 | This indicates that the peer DOES support 536 DDP | DDP (but not RDMA). Procedures outlined in 537 | [DDP-Draft] MUST be followed. 538 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 539 | This indicates that the peer supports BOTH 540 DDP+RDMA | RDMA and DDP. If the receiving endpoint 541 | indicated the same, then the procedures in 542 | both [RDMA-Draft] and [DDP-Draft] 543 | MUST be followed. If the local endpoint 544 | only indicated DDP, then ONLY the 545 | procedures in [DDP-Draft] MUST be followed. 546 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 547 | This indicates that the peer DOES NOT 548 ANY-OTHER | support ANY DDP or RDMA adaption and thus 549 Indication | RDMA and DDP procedures MUST NOT be 550 | performed upon this association. 552 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 554 An implementation MAY require that all associations for a given SCTP 555 endpoint be placed in the same mode. 557 The local interface MAY allow the ULP to accept only requests to 558 establish an association in a specified mode. 560 8.2 Chunk Bundling 562 SCTP allows multiple Data Chunks to be bundled in a single SCTP 563 packet. Data chunks containing DDP Segments with untagged messages 564 SHOULD NOT be delayed to facilitate bundling. Data chunks containing 565 DDP Segments with tagged messages will generally be full sized, and 566 hence not subject to bundling. However partial size tagged messages 567 MAY be delayed, as that they are frequently followed by a short 568 untagged message. 570 8.3 Association Termination 572 Termination of an SCTP Association due to errors should be handled at 573 the SCTP layer. The RDMAP defined RDMAP Terminate Message SHOULD NOT 574 be sent on each DDP Stream when a determination has been made to 575 terminate an SCTP association. Sending that message on each SCTP 576 stream could severely delay the termination of the association. 578 The local interface SHOULD notify all consumers of DDP streams when 579 the underlying SCTP stream has been terminated. 581 Other RDMAP defined Terminate Messages MUST be generated as specified 582 when a DDP Stream is terminated. Note that with the SCTP mapping, 583 termination of a DDP Stream does not mandate termination of the 584 Association. 586 9. IANA considerations 588 This document defines two new Adaptation Layer Indication codepoints: 590 DDP - 0x00000001 591 DDP+RDMA - 0x00000002 593 This document also defines two new Payload Protocol Identifier 594 (PPIDs): 596 DDP Segment - 0x00000001 597 DDP Stream Session Control - 0x00000002 599 10. Security Considerations 601 Any direct placement of memory could pose a significant security risk 602 if adequate local controls are not provided. These threats should be 603 addressed in the appropriate DDP [DDP-Draft] [3] or RDMA [RDMA-Draft] 604 [4] drafts. This document does not add any additional security risks 605 over those found in RFC2960 [2]. 607 11. Acknowledgments 609 Special Acknowledgment to Sukanta Ganguly for his extra efforts in 610 reading and reviewing this document. 612 The authors would like to thank the following people that have 613 provided comments and input: Stephen Bailey, David Black, Douglas 614 Otis, Allyn Romanow and Jim Williams. 616 12. Normative References 618 [1] Bradner, S., "Key words for use in RFCs to Indicate Requirement 619 Levels", BCP 14, RFC 2119, March 1997. 621 [2] Stewart, R., Xie, Q., Morneault, K., Sharp, C., Schwarzbauer, 622 H., Taylor, T., Rytina, I., Kalla, M., Zhang, L., and V. Paxson, 623 "Stream Control Transmission Protocol", RFC 2960, October 2000. 625 [3] Shah, H., "Direct Data Placement over Reliable Transports", 626 draft-ietf-rddp-ddp-05 (work in progress), July 2005. 628 [4] Recio, R., "An RDMA Protocol Specification", 629 draft-ietf-rddp-rdmap-05 (work in progress), July 2005. 631 [5] Stewart, R., "Sockets API Extensions for Stream Control 632 Transmission Protocol (SCTP)", draft-ietf-tsvwg-sctpsocket-10 633 (work in progress), February 2005. 635 [6] Stewart, R., "Stream Control Transmission Protocol (SCTP) 636 Dynamic Address Reconfiguration", 637 draft-ietf-tsvwg-addip-sctp-12 (work in progress), June 2005. 639 Authors' Addresses 641 Randall R. Stewart 642 Cisco Systems, Inc. 643 Forest Drive 644 Columbia, SC 29036 645 USA 647 Phone: 648 Email: rrs@cisco.com 649 Caitlin Bestler 650 Broadcom 651 49 Discovery 652 Irvine, CA 92618 653 USA 655 Phone: 656 Email: caitlinb@siliquent.com 658 Hemal V. Shah 659 Intel Corporation 660 Mailstop: PTL1 661 1501 S. Mopac Expressway, #400 662 Austin, TX 78746 663 USA 665 Phone: 666 Email: hemal.shah@intel.com 668 Vivek Kashyap 669 IBM 670 15450 SW Koll Parkway 671 Beaverton, OR 57006 672 USA 674 Phone: 675 Email: vivk@us.ibm.com 677 Sukanta Ganguly 678 Consultant 680 Phone: 681 Email: sganguly@yahoo.com 683 Intellectual Property Statement 685 The IETF takes no position regarding the validity or scope of any 686 Intellectual Property Rights or other rights that might be claimed to 687 pertain to the implementation or use of the technology described in 688 this document or the extent to which any license under such rights 689 might or might not be available; nor does it represent that it has 690 made any independent effort to identify any such rights. Information 691 on the procedures with respect to rights in RFC documents can be 692 found in BCP 78 and BCP 79. 694 Copies of IPR disclosures made to the IETF Secretariat and any 695 assurances of licenses to be made available, or the result of an 696 attempt made to obtain a general license or permission for the use of 697 such proprietary rights by implementers or users of this 698 specification can be obtained from the IETF on-line IPR repository at 699 http://www.ietf.org/ipr. 701 The IETF invites any interested party to bring to its attention any 702 copyrights, patents or patent applications, or other proprietary 703 rights that may cover technology that may be required to implement 704 this standard. Please address the information to the IETF at 705 ietf-ipr@ietf.org. 707 Disclaimer of Validity 709 This document and the information contained herein are provided on an 710 "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS 711 OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET 712 ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, 713 INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE 714 INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED 715 WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 717 Copyright Statement 719 Copyright (C) The Internet Society (2005). This document is subject 720 to the rights, licenses and restrictions contained in BCP 78, and 721 except as set forth therein, the authors retain all their rights. 723 Acknowledgment 725 Funding for the RFC Editor function is currently provided by the 726 Internet Society.