idnits 2.17.1 draft-bailey-roi-ddp-rdma-arch-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- Couldn't find a document date in the document -- date freshness check skipped. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: 'DAFS' is defined on line 679, but no explicit reference was found in the text == Unused Reference: 'FIBRE' is defined on line 683, but no explicit reference was found in the text == Unused Reference: 'IB' is defined on line 687, but no explicit reference was found in the text == Unused Reference: 'MYR' is defined on line 690, but no explicit reference was found in the text == Unused Reference: 'SDP' is defined on line 712, but no explicit reference was found in the text == Unused Reference: 'SRVNET' is defined on line 715, but no explicit reference was found in the text == Unused Reference: 'VI' is defined on line 719, but no explicit reference was found in the text -- Possible downref: Non-RFC (?) normative reference: ref. 'DAFS' -- Possible downref: Non-RFC (?) normative reference: ref. 'FIBRE' -- Possible downref: Non-RFC (?) normative reference: ref. 'IB' -- Possible downref: Non-RFC (?) normative reference: ref. 'MYR' -- Possible downref: Non-RFC (?) normative reference: ref. 'RDDP' -- Possible downref: Non-RFC (?) normative reference: ref. 'RDMACON' -- Possible downref: Non-RFC (?) normative reference: ref. 'ROM' ** Obsolete normative reference: RFC 2960 (ref. 'SCTP') (Obsoleted by RFC 4960) -- Possible downref: Non-RFC (?) normative reference: ref. 'SDP' -- Possible downref: Non-RFC (?) normative reference: ref. 'SRVNET' -- Possible downref: Non-RFC (?) normative reference: ref. 'VI' Summary: 3 errors (**), 0 flaws (~~), 9 warnings (==), 12 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet-Draft Stephen Bailey (Sandburst) 3 Expires: May 2003 Tom Talpey (NetApp) 5 The Architecture of Direct Data Placement (DDP) 6 And Remote Direct Memory Access (RDMA) 7 On Internet Protocols 8 draft-bailey-roi-ddp-rdma-arch-01 10 Status of this Memo 12 This document is an Internet-Draft and is in full conformance with 13 all provisions of Section 10 of RFC2026. 15 Internet-Drafts are working documents of the Internet Engineering 16 Task Force (IETF), its areas, and its working groups. Note that 17 other groups may also distribute working documents as Internet- 18 Drafts. 20 Internet-Drafts are draft documents valid for a maximum of six 21 months and may be updated, replaced, or obsoleted by other 22 documents at any time. It is inappropriate to use Internet-Drafts 23 as reference material or to cite them other than as "work in 24 progress." 26 The list of current Internet-Drafts can be accessed at 27 http://www.ietf.org/ietf/1id-abstracts.txt 29 The list of Internet-Draft Shadow Directories can be accessed at 30 http://www.ietf.org/shadow.html. 32 Copyright Notice 34 Copyright (C) The Internet Society (2002). All Rights Reserved. 36 Abstract 38 This document defines an abstract architecture for Direct Data 39 Placement (DDP) and Remote Direct Memory Access (RDMA) protocols to 40 run on Internet Protocol-suite transports. This architecture does 41 not necessarily reflect the proper way to implement such protocols, 42 but is, rather, a descriptive tool for defining and understanding 43 the protocols. 45 Table Of Contents 47 1. Introduction . . . . . . . . . . . . . . . . . . . . . . 2 48 2. Architecture . . . . . . . . . . . . . . . . . . . . . . 3 49 2.1. Direct Data Placement (DDP) Protocol Architecture . . . 3 50 2.1.1. Transport Operations . . . . . . . . . . . . . . . . . . 5 51 2.1.2. DDP Operations . . . . . . . . . . . . . . . . . . . . . 6 52 2.1.3. Transport Characteristics in DDP . . . . . . . . . . . . 9 53 2.2. Remote Direct Memory Access Protocol Architecture . . . 10 54 2.2.1. RDMA Operations . . . . . . . . . . . . . . . . . . . . 11 55 2.2.2. Transport Characteristics in RDMA . . . . . . . . . . . 14 56 3. Security Considerations . . . . . . . . . . . . . . . . 14 57 4. IANA Considerations . . . . . . . . . . . . . . . . . . 15 58 5. Acknowledgements . . . . . . . . . . . . . . . . . . . . 15 59 References . . . . . . . . . . . . . . . . . . . . . . . 15 60 Authors' Addresses . . . . . . . . . . . . . . . . . . . 16 61 Full Copyright Statement . . . . . . . . . . . . . . . . 17 63 1. Introduction 65 This document defines an abstract architecture for Direct Data 66 Placement (DDP) and Remote Direct Memory Access (RDMA) protocols to 67 run on Internet Protocol-suite transports [RDDP, ROM]. This 68 architecture does not necessarily reflect the proper way to 69 implement such protocols, but is, rather, a descriptive tool for 70 defining and understanding the protocols. 72 The first part of the document describes the architecture of DDP 73 protocols, including what assumptions are made about the transports 74 on which DDP is built. The second part describes the architecture 75 of RDMA protocols layered on top of DDP. 77 Before introducing the protocols, three definitions will be useful 78 to guide discussion: 80 o Placement - writing to a data buffer. 82 o Delivery - informing the Upper Layer Protocol (ULP) (e.g. 83 RDMA) that a particular message is available for use. 84 Delivery therefore may be viewed as the "control" signal 85 associated with a unit of data. Note that the order of 86 delivery is defined more strictly than it is for placement. 88 o Completion - informing the ULP or application that a 89 particular RDMA operation has finished. A completion, for 90 instance, may require the delivery of several messages, or it 91 may also reflect that some local processing has finished. 93 The goal of the DDP protocol is to allow the efficient placement of 94 data into buffers designated by Upper Layer Protocols (e.g. RDMA). 95 This is described in detail in [ROM]. Efficiency may be 96 characterized by the minimization of the number of transfers of the 97 data over the receiver's system buses. 99 The goal of the RDMA protocol is to provide the semantics to enable 100 Remote Direct Memory Access between peers in a way consistent with 101 application requirements. The RDMA protocol provides facilities 102 immediately useful to existing and future networking, storage, and 103 other application protocols. [DAFS, FIBRE, IB, MYR, SDP, SRVNET, 104 VI] 106 The DDP and RDMA protocols work together to achieve their 107 respective goals. RDMA provides facilities to a ULP for 108 identifying buffers, controlling the transfer of data between ULP 109 peers, and providing completion notifications to the ULP. RDMA 110 uses the features of DDP to steer payloads to specific buffers at 111 the Data Sink. ULPs that do not require the features of RDMA may 112 be layered directly on top of DDP. 114 The DDP and RDMA protocols are transport independent. The 115 following figure shows the relationship between RDMA, DDP, Upper 116 Layer Protocols and Transport. 118 +---------------------------------------------------+ 119 | ULP | 120 +---------+------------+----------------------------+ 121 | | | RDMA | 122 | | +----------------------------+ 123 | | DDP | 124 | +-----------------------------------------+ 125 | Transport | 126 +---------------------------------------------------+ 128 2. Architecture 130 The Architecture section is presented in two parts: Direct Data 131 Placement Protocol architecture and Remote Direct Memory Access 132 Protocol architecture. 134 2.1. Direct Data Placement (DDP) Protocol Architecture 136 The central idea of general-purpose DDP is that a data sender will 137 supplement the data it sends with placement information that allows 138 the receiver's network interface to place the data directly at its 139 final destination without any copying. DDP can be used to steer 140 received data to its final destination, without requiring layer- 141 specific behavior for each different layer. Data sent with such 142 DDP information is said to be `tagged'. 144 The central component of the DDP architecture is the `buffer', 145 which is an object with beginning and ending addresses, and a 146 method (set()) to set the value of an octet at an address. In many 147 cases, a buffer corresponds directly to a portion of host user 148 memory. However, DDP does not depend on this---a buffer could be a 149 disk file, or anything else that can be viewed as an addressable 150 collection of octets. Abstractly, a buffer provides the interface: 152 typedef struct { 153 const address_t start; 154 const address_t end; 155 void set(address_t a, data_t v); 156 } ddp_buffer_t; 158 address_t 160 a reference to local memory 162 data_t 164 an octet data value. 166 The protocol layering and in-line data flow of DDP is: 168 Client Protocol 169 (e.g. ULP or RDMA) 170 | ^ 171 untagged messages | | untagged message delivery 172 tagged messages | | tagged message delivery 173 v | 174 DDP+---> data placement 175 ^ 176 | transport messages 177 v 178 Transport 179 (e.g. SCTP, DCP, framed TCP) 180 ^ 181 | IP datagrams 182 v 183 . . . 185 In addition to in-line data flow, the client protocol registers 186 buffers with DDP, and DDP performs buffer update (set()) operations 187 as a result of receiving tagged messages. 189 DDP messages may be split into multiple, smaller DDP messages, each 190 in a separate transport message. However, if the transport is 191 unreliable or unordered, messages split across transport messages 192 may or may not provide useful behavior, in the same way as 193 splitting arbitrary upper layer messages across unreliable or 194 unordered transport messages may or may not provide useful 195 behavior. In other words, the same considerations apply to 196 building client protocols on different types of transports with or 197 without the use of DDP. 199 A DDP message split across transport messages looks like: 201 DDP message: Transport messages: 203 stag=s, offset=o, message 1: 204 notify=y, id=i |type=ddp | 205 message= |stag=s | 206 |aabbccddee|-------. |offset=o | 207 ~ ... ~----. \ |notify=n | 208 |vvwwxxyyzz|-. \ \ |id=? | 209 | \ `--->|aabbccddee| 210 | \ ~ ... ~ 211 | +----->|iijjkkllmm| 212 | | 213 + | message 2: 214 \ | |type=ddp | 215 \ | |stag=s | 216 \ + |offset=o+n| 217 \ \ |notify=y | 218 \ \ |id=i | 219 \ `-->|nnooppqqrr| 220 \ ~ ... ~ 221 `---->|vvwwxxyyzz| 223 Although this picture suggests that DDP information is carried in- 224 line with the message payload, components of the DDP information 225 may also be in transport-specific fields, or derived from 226 transport-specific control information if the transport permits. 228 2.1.1. Transport Operations 230 For the purposes of this architecture, the transport provides: 232 void xpt_send(socket_t s, message_t m); 233 message_t xpt_recv(socket_t s); 234 msize_t xpt_max_msize(socket_t s); 236 socket_t 238 a transport address, including IP addresses, ports and other 239 transport-specific identifiers. 241 message_t 243 a string of octets. 245 msize_t (scalar) 247 a message size. 249 xpt_send(socket_t s, message_t m) 251 send a transport message. 253 xpt_recv(socket_t s) 255 receive a transport message. 257 xpt_max_msize(socket_t s) 259 get the current maximum transport message size. Corresponds, 260 roughly, to the current path Maximum Transfer Unit (PMTU), 261 adjusted by underlying protocol overheads. 263 Real implementations of xpt_send() and xpt_recv() typically return 264 error indications, but that is not relevant to this architecture. 266 2.1.2. DDP Operations 268 The DDP layer provides: 270 void ddp_send(socket_t s, message_t m); 271 void ddp_send_ddp(socket_t s, message_t m, ddp_addr_t d, 272 ddp_notify_t n); 273 ddp_recv_t ddp_recv(socket_t s); 274 bdesc_t ddp_register(socket_t s, ddp_buffer_t b); 275 void ddp_deregister(bhand_t bh); 276 msizes_t ddp_max_msizes(socket_t s); 278 ddp_addr_t 280 the buffer address portion of a tagged message: 282 typedef struct { 283 stag_t stag; 284 address_t offset; 285 } ddp_addr_t; 287 stag_t (scalar) 289 a Steering Tag. A stag_t identifies the destination buffer 290 for tagged messages. stag_ts are generated when the buffer is 291 registered, communicated to the sender by some client protocol 292 convention and inserted in DDP messages. stag_t values in 293 this DDP architecture are assumed to be completely opaque to 294 the client protocol, and implementation-dependent. However, 295 particular implementations, such as DDP on a multicast 296 transport (see below), may provide the buffer holder some 297 control in selecting stag_ts. 299 ddp_notify_t 301 the notification portion of a DDP message, used to signal that 302 the message represents the final fragment of a multi-segmented 303 DDP message: 305 typedef struct { 306 boolean_t notify; 307 ddp_msg_id_t i; 308 } ddp_notify_t; 310 ddp_msg_id_t (scalar) 312 a DDP message identifier. msg_id_ts are chosen by the DDP 313 message receiver (buffer holder), communicated to the sender 314 by some client protocol convention and inserted in DDP 315 messages. Whether a message reception indication is requested 316 for a DDP message is a matter of client protocol convention. 317 Unlike stag_ts, the structure of msg_id_ts is opaque to DDP, 318 and therefore, completely in the hands of the client protocol. 320 bdesc_t 322 a description of a registered buffer: 324 typedef struct { 325 bhand_t bh; 326 ddp_addr_t a; 327 } bdesc_t; 329 `a.offset' is the starting offset of the registered buffer, 330 which may have no relationship to the `start' or `end' 331 addresses of that buffer. However, particular 332 implementations, such as DDP on a multicast transport (see 333 below), may allow some client protocol control over the 334 starting offset. 336 bhand_t 338 an opaque buffer handle used to deregister a buffer. 340 ddp_recv_t 342 an untagged message, a tagged message reception indication, or 343 a tagged message reception error: 345 typedef union { 346 message_t m; 347 ddp_msg_id_t i; 348 ddp_err_t e; 349 } ddp_recv_t; 351 ddp_err_t 353 indicates an error while receiving a tagged message, typically 354 `offset' out of bounds, or `stag' is not registered to the 355 socket. 357 msizes_t 359 The maximum untagged and tagged messages that fit in a single 360 transport message: 362 typedef struct { 363 msize_t max_untagged; 364 msize_t max_tagged; 365 } msizes_t; 367 ddp_send(socket_t s, message_t m) 368 send an untagged message. 370 ddp_send_ddp(socket_t s, message_t m, ddp_addr_t d, ddp_notify_t n) 372 send a tagged message. 374 ddp_recv(socket_t s) 376 get the next received untagged message, tagged message 377 reception indication, or tagged message error. 379 ddp_register(socket_t s, ddp_buffer_t b) 381 register a buffer for DDP on a socket. The same buffer may be 382 registered multiple times on the same or different sockets. 383 Different buffers may also refer to portions of the same 384 underlying addressable object (buffer aliasing). 386 ddp_deregister(bhand_t bh) 388 remove a registration from a buffer. 390 ddp_max_msizes(socket_t s) 392 get the current maximum untagged and tagged message sizes that 393 will fit in a single transport message. 395 2.1.3. Transport Characteristics In DDP 397 Certain characteristics of the transport on which DDP is mapped 398 determine the nature of the service provided to client protocols. 399 Specifically, transports are: 401 o reliable or unreliable, 403 o ordered or unordered, 405 o single source or multisource, 407 o single destination or multidestination (multicast or anycast). 409 Some transports support several combinations of these 410 characteristics. For example, SCTP [SCTP] is reliable, single 411 source, single destination (point-to-point) and supports both 412 ordered and unordered modes. 414 In general, these transport characteristics equally affect 415 transport and DDP message delivery. However, there are several 416 issues specific to DDP messages. 418 A key component of DDP is how the following operations on the 419 receiving side are ordered among themselves, and how they relate to 420 corresponding operations on the sending side: 422 o set()s, 424 o untagged message reception indications, and 426 o tagged message reception indications. 428 These relationships depend upon the characteristics of the 429 underlying transport in a way which is defined by the DDP protocol. 430 For example, if the transport is unreliable and unordered, the DDP 431 protocol might specify that the client protocol is subject to the 432 consequences of transport messages being lost or duplicated, rather 433 requiring different characteristics be presented to the client 434 protocol. 436 Multidestination data delivery is the other transport 437 characteristic which may require specific consideration in a DDP 438 protocol. As mentioned above, the basic DDP model assumes that 439 buffer address values returned by ddp_register() are opaque to the 440 client protocol, and can be implementation dependent. The most 441 natural way to map DDP to a multidestination transport is to 442 require all receivers produce the same buffer address when 443 registering a multidestination destination buffer. Restriction of 444 the DDP model to accommodate multiple destinations involves 445 engineering tradeoffs comparable to those of providing non-DDP 446 multidestination transport capability. 448 2.2. Remote Direct Memory Access (RDMA) Protocol Architecture 450 Remote Direct Memory Access (RDMA) extends the capabilities of DDP 451 with the ability to read from buffers registered to a socket (RDMA 452 Read). This allows a client protocol to perform arbitrary, 453 bidirectional data movement without involving the remote client. 454 When RDMA is implemented in hardware, arbitrary data movement can 455 be performed without involving the remote host CPU at all. 457 In addition, RDMA protocols usually specify a transport-independent 458 untagged message service (Send) with characteristics which are both 459 very efficient to implement in hardware, and convenient for client 460 protocols. 462 The RDMA architecture is patterned after the traditional model for 463 device programming, where the client requests an operation using 464 Send-like actions (programmed I/O), the server performs the 465 necessary data transfers for the operation (DMA reads and writes), 466 and notifies the client of completion. The programmed I/O+DMA 467 model efficiently supports a high degree of concurrency and 468 flexibility for both the client and server, even when operations 469 have a wide range of intrinsic latencies. 471 RDMA is layered as a client protocol on top of DDP: 473 Client Protocol 474 | ^ 475 Sends | | Send reception indications 476 RDMA Read Requests | | RDMA Read Completion indications 477 RDMA Writes | | RDMA Write Completion indications 478 v | 479 RDMA 480 | ^ 481 untagged messages | | untagged message delivery 482 tagged messages | | tagged message delivery 483 v | 484 DDP+---> data placement 485 ^ 486 | transport messages 487 v 488 . . . 490 In addition to in-line data flow, read (get()) and update (set()) 491 operations are performed on buffers registered with RDMA as a 492 result of RDMA Read Requests and RDMA Writes, respectively. 494 An RDMA `buffer' extends a DDP buffer with a get() operation that 495 retrieves the value of the octet at address `a': 497 typedef struct { 498 const address_t start; 499 const address_t end; 500 void set(address_t a, data_t v); 501 data_t get(address_t a); 502 } rdma_buffer_t; 504 2.2.1. RDMA Operations 506 The RDMA layer provides: 508 void rdma_send(socket_t s, message_t m); 509 void rdma_write(socket_t s, message_t m, ddp_addr_t d, 510 rdma_notify_t n); 511 void rdma_read(socket_t s, ddp_addr_t s, ddp_addr_t d); 512 rdma_recv_t rdma_recv(socket_t s); 513 bdesc_t rdma_register(socket_t s, rdma_buffer_t b, 514 bmode_t mode); 515 void rdma_deregister(bhand_t bh); 516 msizes_t rdma_max_msizes(socket_t s); 518 Although, for clarity, these data transfer interfaces are 519 synchronous, rdma_read() and possibly rdma_send() (in the presence 520 of Send flow control), can require an arbitrary amount of time to 521 complete. To express the full concurrency and interleaving of RDMA 522 data transfer, these interfaces are also defined to be 523 multithreaded. For example, a client protocol may perform an 524 rdma_send(), while an rdma_read() operation is in progress. 526 rdma_notify_t 528 RDMA Write notification information, used to signal that the 529 message represents the final fragment of a multi-segmented 530 RDMA message: 532 typedef struct { 533 boolean_t notify; 534 rdma_write_id_t i; 535 } rdma_notify_t; 537 identical in function to ddp_notify_t, except that the type 538 rdma_write_id_t may not be equivalent to ddp_msg_id_t. 540 rdma_write_id_t (scalar) 542 an RDMA Write identifier. 544 rdma_recv_t 546 a Send message, an RDMA Write completion identifier, or an 547 RDMA error: 549 typedef union { 550 message_t m; 551 rdma_write_id_t i; 552 rdma_err_t e; 553 } rdma_recv_t; 555 rdma_err_t 557 an RDMA protocol error indication. RDMA errors include buffer 558 addressing errors corresponding to ddp_err_ts, and buffer 559 protection violations (e.g. RDMA Writing a buffer only 560 registered for reading). 562 bmode_t 564 buffer registration mode (permissions). Any combination of 565 permitting RDMA Read (BMODE_READ) and RDMA Write (BMODE_WRITE) 566 operations. 568 rdma_send(socket_t s, message_t m) 570 send a message, delivering it to the next untagged RDMA buffer 571 at the remote peer. 573 rdma_write(socket_t s, message_t m, ddp_addr_t d, rdma_notify_t n) 575 RDMA Write to remote buffer address d. 577 rdma_read(socket_t s, ddp_addr_t s, ddp_addr_t d) 579 RDMA Read from remote buffer address s to local buffer address 580 d. 582 rdma_recv(socket_t s); 584 get the next received Send message, RDMA Write completion 585 identifier, or RDMA error. 587 rdma_register(socket_t s, rdma_buffer_t b, bmode_t mode) 589 register a buffer for RDMA on a socket (for read access, write 590 access or both). As with DDP, the same buffer may be 591 registered multiple times on the same or different sockets, 592 and different buffers may refer to portions of the same 593 underlying addressable object. 595 rdma_deregister(bhand_t bh) 597 remove a registration from a buffer. 599 rdma_max_msizes(socket_t s) 601 get the current maximum Send (max_untagged) and RDMA Read or 602 Write (max_tagged) operations that will fit in a single 603 transport message. The values returned by rdma_max_msizes() 604 are closely related to the values returned by 605 ddp_max_msizes(), but may not be equal. 607 2.2.2. Transport Characteristics In RDMA 609 As with DDP, RDMA can be used on transports with a variety of 610 different characteristics that manifest themselves directly in the 611 service provided by RDMA. 613 Like DDP, an RDMA protocol must specify how: 615 o set()s, 617 o get()s, 619 o Send messages, and 621 o RDMA Read completions 623 are ordered among themselves and how they relate to corresponding 624 operations on the remote peer(s). These relationships are likely 625 to be a function of the underlying transport characteristics. 627 There are some additional characteristics of RDMA which may 628 translate poorly to unreliable or multipoint transports due to 629 attendant complexities in managing endpoint state: 631 o Send flow control 633 o RDMA Read 635 These difficulties can be overcome by placing restrictions on the 636 service provided by RDMA. However, many RDMA clients, especially 637 those that separate data transfer and application logic concerns, 638 are likely to depend upon capabilities only provided by RDMA on a 639 point-to-point, reliable transport. 641 3. Security Considerations 643 System integrity must be maintained in any RDMA solution. 644 Mechanisms must be specified to prevent RDMA or DDP operations from 645 impairing system integrity. For example, the threat caused by 646 potential buffer overflow needs full examination, and prevention 647 mechanisms must be spelled out. 649 Because a Steering Tag exports access to a memory region, one 650 critical aspect of security is the scope of this access. It must 651 be possible to individually control specific attributes of the 652 access provided by a Steering Tag, including remote read access, 653 remote write access, and others that might be identified. A 654 specification must provide both implementation requirements 655 relevant to this issue, and guidelines to assist implementors in 656 making the appropriate design decisions. 658 A number of other potential attacks have been envisioned and must 659 be addressed. Some such examples are outlined in [RDMACON]. 660 Resource issues leading to denial-of-service attacks, overwrites 661 and other concurrent operations, the ordering of completions as 662 required by the RDMA protocol, and the granularity of transfer are 663 all within the required scope of any security analysis of RDMA and 664 DDP. 666 4. IANA Considerations 668 IANA considerations are not addressed in by this document. Any 669 IANA considerations resulting from the use of DDP or RDMA must be 670 addressed in the relevant standards. 672 5. Acknowledgements 674 The authors wish to acknowledge the valuable contributions of David 675 Black, Jeff Mogul and Allyn Romanow. 677 6. References 679 [DAFS] 680 Direct Access File System http://www.dafscollaborative.org 681 http://www.ietf.org/internet-drafts/draft-wittle-dafs-00.txt 683 [FIBRE] 684 Fibre Channel Standard 685 http://www.fibrechannel.com/technology/index.master.html 687 [IB] InfiniBand Architecture Specification, Volumes 1 and 2, 688 Release 1.0.a. http://www.infinibandta.org 690 [MYR] 691 Myrinet, http://www.myricom.com 693 [RDDP] 694 Remote Direct Data Placement Working Group charter, 695 http://www.ietf.org/html.charters/rddp-charter.html 697 [RDMACON] 698 D. Black, M. Speer, J. Wroclawski, "DDP and RDMA Concerns", 699 http://www.ietf.org/internet-drafts/draft-black-rdma- 700 concerns-00.txt, Work in Progress, June 2002 702 [ROM] 703 A. Romanow, J. Mogul, T. Talpey, S. Bailey, "RDMA over IP 704 Problem Statement", http://www.ietf.org/internet-drafts/draft- 705 romanow-rdma-over-ip-problem-statement-01.txt, Work in 706 Progress, November 2002 708 [SCTP] 709 R. Stewart et al., "Stream Transmission Control Protocol", 710 Standards Track RFC, http://www.ietf.org/rfc/rfc2960 712 [SDP] 713 Sockets Direct Protocol v1.0 715 [SRVNET] 716 Compaq Servernet, 717 http://nonstop.compaq.com/view.asp?PAGE=ServerNet 719 [VI] Virtual Interface Architecture Specification Version 1.0. 720 http://www.viarch.org/html/collateral/san_10.pdf 722 Authors' Addresses 724 Stephen Bailey 725 Sandburst Corporation 726 600 Federal Street 727 Andover, MA 01810 USA 728 USA 730 Phone: +1 978 689 1614 731 Email: steph@sandburst.com 733 Tom Talpey 734 Network Appliance 735 375 Totten Pond Road 736 Waltham, MA 02451 USA 738 Phone: +1 781 768 5329 739 Email: thomas.talpey@netapp.com 741 Full Copyright Statement 743 Copyright (C) The Internet Society (2002). All Rights Reserved. 745 This document and translations of it may be copied and furnished to 746 others, and derivative works that comment on or otherwise explain 747 it or assist in its implementation may be prepared, copied, 748 published and distributed, in whole or in part, without restriction 749 of any kind, provided that the above copyright notice and this 750 paragraph are included on all such copies and derivative works. 751 However, this document itself may not be modified in any way, such 752 as by removing the copyright notice or references to the Internet 753 Society or other Internet organizations, except as needed for the 754 purpose of developing Internet standards in which case the 755 procedures for copyrights defined in the Internet Standards process 756 must be followed, or as required to translate it into languages 757 other than English. 759 The limited permissions granted above are perpetual and will not be 760 revoked by the Internet Society or its successors or assigns. 762 This document and the information contained herein is provided on 763 an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET 764 ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR 765 IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF 766 THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED 767 WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.