2.8.15 Remote Direct Data Placement (rddp)

NOTE: This charter is a snapshot of the 56th IETF Meeting in San Francisco, California USA. It may now be out-of-date.

Last Modified: 2003-02-26

David Black <black_david@emc.com>
Transport Area Director(s):
Scott Bradner <sob@harvard.edu>
Allison Mankin <mankin@psg.com>
Transport Area Advisor:
Scott Bradner <sob@harvard.edu>
Technical Advisor(s):
Allyn Romanow <allyn@cisco.com>
Stephen Bailey <steph@cs.uchicago.edu>
Mailing Lists:
General Discussion: rddp@ietf.org
To Subscribe: rddp-request@ietf.org
In Body: subscribe
Archive: www.ietf.org/mail-archive/working-groups/rddp/current/maillist.html
Description of Working Group:
The purpose of this WG is to enable Remote Direct Data Placement (rddp) capabilities with IP transport protocols, in particular with SCTP. RDDP capabilities refer to the ability to place data directly from the NIC into application buffers, without intensive CPU usage. This strategy avoids the costs of data copying and enables using IP as a method for high speed buffer to buffer transfer, allowing IP to replace special purpose networks currently in use. Remote Direct Memory Access (RDMA) is an example of this concept.

Conceptually, RDDP functionality can be viewed as consisting of two layers. First the direct data placement capability, which is accomplished through a tag and a lookup table on the NIC. Above this core functionality, an RDDP control protocol is needed to specify how the direct data placement can be used, for example READ and WRITE commands.

The work of the WG is to accomplish four items:

1) A (transport independent) protocol core to support direct data placement from the NIC into specified memory, usually application buffers.

2) A (transport independent) protocol core layered on top of the direct data placement protocol that specifies control of RDDP.

3) A mapping of the direct data placement protocol onto SCTP, for standards track, including a clear applicability statement of the expected service from the mapping.

4) A mapping of the direct data placement protocol onto TCP, for informational, because TCP's service is a less good match to RDDP, including an applicability statement of the issues regarding the service available from the mapping.

The working group will ensure that the resulting technology will be secure and will not enable new attacks on systems supporting RDDP. The WG will not modify existing Internet transport protocols, but may forward issues it discovers in such transport protocols that are not full Internet Standards to the appropriate IETF WGs for their consideration.

Goals and Milestones:
Done  Submit Internet-Draft including problem statement and architecture
Done  Submit Initial draft of Remote Direct Data Placement protocol (RDDP)
Done  Submit Initial draft of RDMA control protocol, to be named.
Done  Initial draft mapping the RDDP core and control onto SCTP including A/S
MAR 03  Submit problem statement/architecture to IESG for consideration as an informational publication
MAR 03  Area Director review of charter and progress, with possible rechartering or closure.
APR 03  Initial draft informationally mapping the RDDP core and control onto TCP, including A/S
JUL 03  Submit Remote Data Placement Protocol (RDDP) to IESG for consideration as a Proposed Standard
JUL 03  Submit RDMA control protocol (named TBD) to IESG for consideration as a Proposed Standard
OCT 03  Submit mapping the RDDP core and RDMA control onto SCTP including detailed applicability statement to IESG for consideration as a Proposed Standard
OCT 03  Submit mapping the RDDP core and RDMA control onto TCP including detailed applicability statement to IESG for consideration as informational RFC
  • - draft-stewart-rddp-sctp-02.txt
  • - draft-ietf-rddp-rdma-concerns-00.txt
  • - draft-ietf-rddp-arch-01.txt
  • - draft-ietf-rddp-problem-statement-01.txt
  • - draft-ietf-rddp-rdmap-00.txt
  • - draft-bestler-rddp-applicability-00.txt
  • - draft-ietf-rddp-ddp-00.txt
  • No Request For Comments

    Current Meeting Report

    RDDP WG Meeting Minutes
    56th IETF
    March 17 & 20 2003
    Thanks to Allyn Romanow for taking notes.
    -- TCP Mapping Requirements (David Black)
    This is a reminder from the WG chair in combination with the Area 
    Directors of what is expected of the RDDP WG's TCP mapping work.  
    (1) The "proof" requirement that two communicating stacks can't get 
    confused about the format of communication applies to the entire RDDP 
    protocol stack running over TCP, not just the issue of whether a 
    modified vs. unmodified TCP is being used.  32 bits of zeroes (first 
    marker) to distinguish what is going on is not enough, 128 bits 
    randomly generated for the connection is enough (sufficient, but may be 
    more than needed).  Relying on ULP to do this is considerably weaker than 
    solving the problem in RDDP.  Negotiation is not required - a 
    declaration mechanism like the SCTP adaption indication exchange is 
    sufficient (no additional round trip is required).
    (2) The nfsv4 WG is starting work on application of RDDP to NFSv4.  
    NFSv4's current level of data integrity does not require a CRC; if this 
    remains the case for NFSv4 over RDDP, the RDDP CRC will need to be 
    optional to use.
    (3) Fixed interval markers are a potential problem for software 
    implementations (see below, as a great deal of time was spent on this 
    issue later).
    -- Administrivia
    Reminder that the problem statement and architecture drafts are in WG Last 
    Call.  Please read and comment on the list.
    -- SCTP Mapping Draft (Randy Stewart)
    See slides (will be posted).  Draft is relatively far along.
    Note that SCTP uses (completely) unordered delivery when carrying RDDP 
    traffic.  This requires DDP to track its operation ordering in order to 
    enforce any delivery order requirements.
    -- Applicability Statement Draft (Randy Stewart for Caitlin Bestler)
    Plan has changed to a 3 draft approach - (1) TCP mapping, (2) SCTP 
    mapping, and (3) an applicability statement draft covering both 
    mappings.  Lode Coene (editor of SCTP's applicability statement, which set a 
    good example) has agreed to help out with the RDDP applicability 
    statement (many thanks).
    Advice from the WG chairs and ADs: A good applicability statement is 
    guidance on utility, not advertising to promote usage.  An important 
    target audience is ULP designers who are looking advice on whether to use 
    RDDP and with which transport.
    Both the SCTP mapping draft and applicability statement draft were 
    approved to be official RDDP WG drafts - next versions will have 
    draft-ietf-rddp-* names.
    -- Allyn Romanow - NICELI Workshop reminder and security update
    For NICELI, see:
    Security update - folks working on this issue are making good progress with 
    Catherine Meadows (security advisor).  Initial draft will be posted for 
    discussion in the next 2-3 weeks.
    -- DDP Update -  Hemal Shah
    couple of small changes.  
    -- RDMAP Update - Paul Culley (for Renato Recio)
    a couple of small changes (no presentation slides)
    -- TCP mapping
    - MPA Update - Paul Culley
    Packing of multiple ULPDUS into single TCP segment is now allowed.  MPA is 
    now specified as an "adaptation layer" for RDDP that must be 
    explicitly enabled by higher-level protocol (i.e., DDP/RDMA or ULP 
    further up the stack).
    - IFT update - Jim Williams
    Added adaption indication, based to some extent on the one used with SCTP.  
    CRC is optional and negotiable
    Markers are negotiable, with some details to be fleshed out.
    Speculative placement proposed for use when markers not provided - this 
    received a rather negative reaction in the meeting.
    -- TCP mapping issues
    David Black worked through several major outstanding issues for the WG:
    (1) Alignment and TCP Mapping
    This is about aligning transmitted ULPDUS (w/RDDP headers) to TCP 
    segment boundaries.  
    David Black summarized from the mailing list discussion and made a 
    proposal Alignment is not required for interoperability, the RDDP 
    receiver cannot rely on it, but it is a potentially useful 
    Therefore, the proposal is to treat alignment as an 
    implementation optimization only, and hence not use any MUST or SHOULD (in 
    the RFC 2119 sense) for alignment, but instead use (at most) lower case 
    "should" along with a description of the circumstances in which 
    alignment is a useful optimization.
    This represents rough consensus of the Working Group participants in the 
    Additional note from WG chair: While nothing is certain, this approach is 
    very likely to be acceptable to the tsvwg (Transport Area) WG, where past 
    alignment work has generated controversy.
    (2) Markers - this generated a *lot* of discussion, mostly focused on 
    marker interval.
    Markers are motivated by a  need to regain synchronization for 
    placement after a packet drop or reordering without having to waiting for 
    retransmit.  Markers allow finding where the header is in byte stream.  MPA 
    currently requires markers at 512 byte fixed intervals
    A potential problem with markers is the overhead for existing stacks For 
    example, for an 8KB transfer, 1 send operation becomes 16 sends (Some 
    people suggest fewer than 16 because can use gather technique.)  This 
    overhead is a potential problem for a software implementation.
    Markers have been motivated by flow through designs that may be overly 
    optimized, especially given above rough consensus on alignment.  The 
    discussion focused on buffering requirements, and whether a link frame per 
    connection is required - it clearly is in the worst case, but there are 
    enough shorter packets/frames (e.g., about 500 bytes on Ethernet) that on 
    average across multiple connections, one can use less, and hence a marker 
    interval shorter than the max link frame size (e.g., about 1500 bytes for 
    Ethernet without jumbo frames) does save implementation resources.
    There was at least one comment that Markers should be optional.
    There is no rough consensus, but David notes that there is a lot of 
    support for markers and the 512 byte interval in particular.
    (3) NFSV4 and CRCs
    The nfsv4 WG charter now includes NFSV4 over RDDP problem statement and 
    NFS does not require stronger data integrity than TCP/IP checksums, so a 
    mandatory RDDP CRC may be excessive although optional RDDP CRC could be 
    The argument in the other direction is that RDDP is new and wants to 
    provide a better level of service (data integrity), so demanding that all 
    users of RDDP step up to that level of service is within reason.
    After much discussion the conclusion is to defer the issue of whether to 
    require CRC usage for further input from the nfsv4 WG's 
    requirements work.
    -David concludes:
    alignment - there is rough consensus in the room, will recheck on list.
    marker interval - there is clear support for a 512B interval CRC - will 
    defer "MUST use" issue until nfsv4 WG is further along in 
    considering requirements for RDDP with NFSv4.
    Hope to close remaining issues in this space by summer IETF (Vienna)


    IFT Proposal
    Direct Data Placement (DDP) over Reliable Transports
    RDDP: TCP Alignment
    TCP Mapping: Markers