2.8.13 Remote Direct Data Placement (rddp)

NOTE: This charter is a snapshot of the 60th IETF Meeting in San Diego, CA USA. It may now be out-of-date.

Last Modified: 2004-06-15

David Black <black_david@emc.com>
Transport Area Director(s):
Allison Mankin <mankin@psg.com>
Jon Peterson <jon.peterson@neustar.biz>
Transport Area Advisor:
Jon Peterson <jon.peterson@neustar.biz>
Technical Advisor(s):
Allyn Romanow <allyn@cisco.com>
Stephen Bailey <steph@cs.uchicago.edu>
Mailing Lists:
General Discussion: rddp@ietf.org
To Subscribe: rddp-request@ietf.org
In Body: subscribe
Archive: http://www.ietf.org/mail-archive/web/rddp/index.html
Description of Working Group:
The purpose of this WG is to enable Remote Direct Data Placement (rddp) capabilities with IP transport protocols, in particular with SCTP. RDDP capabilities refer to the ability to place data directly from the NIC into application buffers, without intensive CPU usage. This strategy avoids the costs of data copying and enables using IP as a method for high speed buffer to buffer transfer, allowing IP to replace special purpose networks currently in use. Remote Direct Memory Access (RDMA) is an example of this concept.

Conceptually, RDDP functionality can be viewed as consisting of two layers. First the direct data placement capability, which is accomplished through a tag and a lookup table on the NIC. Above this core functionality, an RDDP control protocol is needed to specify how the direct data placement can be used, for example READ and WRITE commands.

The work of the WG is to accomplish four items:

1) A (transport independent) protocol core to support direct data placement from the NIC into specified memory, usually application buffers.

2) A (transport independent) protocol core layered on top of the direct data placement protocol that specifies control of RDDP.

3) A mapping of the direct data placement protocol onto SCTP, for standards track, including a clear applicability statement of the expected service from the mapping.

4) A mapping of the direct data placement protocol onto TCP, for informational, because TCP's service is a less good match to RDDP, including an applicability statement of the issues regarding the service available from the mapping.

The working group will ensure that the resulting technology will be secure and will not enable new attacks on systems supporting RDDP. The WG will not modify existing Internet transport protocols, but may forward issues it discovers in such transport protocols that are not full Internet Standards to the appropriate IETF WGs for their consideration.

Goals and Milestones:
Done  Submit Internet-Draft including problem statement and architecture
Done  Submit Initial draft of Remote Direct Data Placement protocol (RDDP)
Done  Submit Initial draft of RDMA control protocol, to be named.
Done  Initial draft mapping the RDDP core and control onto SCTP including A/S
Done  Submit problem statement and architecture drafts to IESG for consideration as informational publications
Done  Initial draft of security considerations for RDDP
Done  Initial draft of informational mapping of the RDDP core and control onto TCP
Done  Initial draft of applicability statement covering both the SCTP and TCP mappings
Mar 04  Submit RDMA control protocol (named TBD) to IESG for consideration as a Proposed Standard
Mar 04  Submit Remote Data Placement Protocol (RDDP) to IESG for consideration as a Proposed Standard
Mar 04  Submit RDDP security considerations draft to IESG
Sep 04  Submit mapping of the RDDP core and RDMA control protocols onto SCTP to IESG for consideration as proposed standard
Sep 04  Submit mapping of the RDDP core and RDMA control protocols onto TCP for consideration as an informational publication
Sep 04  Submit applicability statement for RDDP core and RDMA control protocols over both SCTP and TCP for consideration as an informational publication
Oct 04  Consult Area Directors about any additional work the WG should undertake
  • - draft-ietf-rddp-rdma-concerns-01.txt
  • - draft-ietf-rddp-arch-05.txt
  • - draft-ietf-rddp-problem-statement-04.txt
  • - draft-ietf-rddp-ddp-02.txt
  • - draft-ietf-rddp-mpa-01.txt
  • - draft-ietf-rddp-security-02.txt
  • No Request For Comments

    Current Meeting Report

    RDDP WG Minutes - FINAL
    60th IETF Meeting, San Diego
    Wednesday, August 4, 0900-1130

    Many thanks to Hemal Shah for taking the raw minutes. Letter in square brackets (e.g., [A]) is first letter in file name containing slides for that portion of the meeting.

    -- Agenda Bashing and Administrivia - David Black, WG Chair [A]
    - Blue Sheets

    The agenda was bashed prior to the meeting to include the MPA CRC issue that came up in the IPS meeting on Monday.

    -- Status of drafts - David Black, WG Chair [no slides]

    Problem statement draft needs some minor revisions, and then should be ready for IESG approval.
    Architecture draft is held up by a security issue related to determining the mandatory to implement security mechanism(s).
    RDMAP draft was sent in for posting but didn't make it. Both it and the DDP draft will have boilerplate and security updates applied and be resubmitted.
    Applicability statement and SCTP mapping are missing from the Internet-Draft servers. They will be resubmitted with updated boilerplate.

    Broadcom needs to update their IPR notice to comply with RFC 3668 soon in order to avoid it causing delays.

    MPA draft status (informational vs. standards track) will be determined by WG chair in consultation with ADs after WG last call. WG chair understands that WG would prefer it to be standards track.

    -- MPA CRC issue (David Black, WG Chair, slide extracted from Mike Ko's iSER presentation to the IPS WG on Monday) [B]

    iSER is a protocol that uses MPA (RDDP mapping to TCP) for iSCSI. The iSER draft authors have found MPA's current CRC requirements text to be insufficient for iSER usage of MPA, and hence currently impose stronger requirements. iSER requires that RDDP provide at least CRC32-class integrity, but the current MPA draft text allows CRC32C to be disabled in situations where that would not be the result.

    The RDDP WG's rough consensus has been that ULPs can assume at least CRC-class integrity from RDDP. Hence the sense of the room is that the MPA draft text needs to be tightened to do this and remove the need for iSER to add requirements to MPA.

    ACTION: MPA draft authors to propose specific new text on reflector.

    -- Security Draft (draft-ietf-rddp-security-02.txt) - Jim Pinkerton [C]

    Two Important issues:
    - Mandatory to implement security requirements.
    - How to strongly discourage use of data while write-accessible from network.

    -02 draft is not significantly different from -01, mostly a boilerplate update

    There are 34 RECOMMENDED clauses and 8 MUSTs in the current draft split among RNIC protocol implementation and application implementation requirements. It's not clear that the "RECOMMENDED"s and "MUST"s are all correct. Jim will post a new security draft to the reflector and schedule a conference call to provide a recommendation to the WG on which requirement level should be used for which clause. See RFC 2119 for the definitions of MUST and RECOMMENDED. SHOULD is a synonym for RECOMMENDED, but lower-case should has it's ordinary meaning which is weaker than an RFC 2119 SHOULD.

    --> Important Security Issue #1: IPSec - required or optional?

    A lengthy discussion ensued on this topic. Important points:
    - For optional: IPsec can involve significant hardware complexity.
    - For mandatory: Some attacks can only be mitigated by IPsec (WG has little interest in designing its own security mechanism), and they will be of concern in some deployment environments.

    IP Storage (RFC 3723) enables IPsec to be implemented separately from the protected protocol - when this is done correctly for end-to-end security, the resulting architecture is "bump in the stack (BITS)" as opposed to "bump in the wire (BITW)" - see RFC 2401 for more on this.

    The security draft needs to be clear that the security requirements are intended to enable end-to-end security.

    Making IPsec mandatory does *not* force it to be implemented in hardware. IETF RFCs impose functional requirements for security, not performance, so a potentially slow software implementation can be used to meet those requirements.

    Since iSER is using RDDP with iSCSI, reusing the IP Storage IPsec requirements (RFC 3723) for iSCSI will avoid adding complexity in that situation, as the iSCSI requirements will not be removed by iSER.

    The data rates expected for some uses of RDDP require that a key exchange protocol be mandated (i.e., only manual keying is unacceptable).

    The issue at hand has two parts:

    (1) Should IPsec be required or optional?

    Sense of room: it should be required, but without any restrictions on how it is implemented.

    (2) Should the IPsec requirements be based on the existing RFCs (e.g., use IKEv1) or the new drafts coming from the ipsec WG (e.g., use IKEv2).

    Sense of room: Base them on the existing RFCs by referencing the IP Storage IPsec requirements in RFC 3723.

    Two UPDATEs from after the meeting:
    - Making IPsec required clears the IESG security issue blocking the rddp architecture draft.
    - Based on the saag meeting on Thursday, the IETF Security Area has no problem with mandating use of IKEv1 in a new RFC, even though IKEv2 will be an RFC in the near future.

    --> Additional issues from Jim's slides.

    - Abortive termination on error
    Current protocol drafts drop connection if bad STag is received
    Proposal: No change (ok with room)

    - Should STags be random?
    Small security advantage to doing so, IPsec is the real solution. HW implementations prefer to control tags for linear lookup Recommendation: No change (ok with room).
    UPDATE: Also ok with Security AD whose DISCUSS on this issue is blocking the rddp architecture draft at the IESG - MUST implement IPsec is more than sufficient.

    - Shared Receive Queue attacks
    This issue involved some detailed text editing - do it on the reflector.

    --> Important Security Issue #2: How to strongly discourage allowing network write access to data in use by a ULP or application. This includes slides from John Carrier [D] (many thanks to John for preparing these detailed slides on *very* short notice).

    This issue has misleadingly been called "One-shot STags".

    Issue is around having data sink buffer contents getting changed before the buffer is invalidated. If protocol or application logic depends on the contents not changing, this may open up attacks on the protocol. IPsec doesn't help because one has to protect against a less than fully trusted peer who can authenticate and use IPsec if required.

    One-shot STags are a "heavy hammer" approach to this problem - completion of an RDMA write always invalidates the associated STag. The problem is that this is too heavy - both NFS/RDMA and iSER need multi-shot STags for resource management and optimization reasons, and cannot live with a restriction that all STags be single-shot. One consequence is that the receiver MUST check for STag invalidation before using the data, even when Send with Invalidate was used (have to make sure right STag was invalidated).

    There are a couple of additional possibilities:
    a) Receive and Invalidate. If one knows which untagged receive buffer will be used for the ULP operation that signals completion of the data transfer, one can pre-mark that receive buffer to force invalidation of the STag when the transfer completes.

    Problem: This won't work when data transfers can complete in non-deterministic order, because the exact sequence of completion has to be known in advance. Both iSCSI and NFS exhibit this non-deterministic completion order behavior.

    Sense of room: Not worth pursuing further due to limited applicability.

    b) RDMA Write & Invalidate: Define an RDMA Write primitive that also invalidates the STag that it uses. Having every RDMA Write invalidate the associated STag is an implementation of One-shot STags, and hence won't work. Defining RDMA Writes that do and don't invalidate the appropriate STag still requires a receiver check for invalidation and hence provides no leverage.

    Sense of room: Not worth pursuing.

    Conclusion of issue: Need to make sure security and protocol drafts contain stiff warnings about the need for any ULP using RDDP to invalidate receive STags prior to using data they cover. Architecture draft will incorporate this warning also. Need to specifically call attention to situations in which protocol logic path depends on contents of received buffer.

    --> Things to be done to security draft:
    - Guidance for application protocols like NFS which implement security, including informative reference to channel bindings draft (Nicolas Williams to provide this ref, David Black to send him email reminding him to provide it. UPDATE: Done.)
    - Finish summary table of attacks/trust models
    - Finish Tom Talpey's comments posted to the reflector

    -- Next steps

    - Revise security draft.
    - Add appropriate security requirements in DDP, RDMAP, and architecture draft.
    - Good chance to move DDP, RDMAP, and security drafts to WG Last Call by mid-September.
    - WG Last Call on transport mappings drafts (MPA, SCTP mapping) and applicability statement will be later.


    CRC32C Protection in the Layer Below iSER
    Draft-ietf-rddp-security-02 Summary of outstanding issues
    One-Shot STags