Current Meeting Report
Jabber Logs

2.8.15 Remote Direct Data Placement (rddp)

NOTE: This charter is a snapshot of the 55th IETF Meeting in Altanta, Georgia USA. It may now be out-of-date.

Last Modifield: 07/05/2002

David Black <>
Transport Area Director(s):
Scott Bradner <>
A. Mankin <>
Transport Area Advisor:
Scott Bradner <>
Technical Advisor(s):
Allyn Romanow <>
Stephen Bailey <>
Mailing Lists:
General Discussion:
To Subscribe:
In Body: subscribe
Description of Working Group:
The purpose of this WG is to enable Remote Direct Data Placement (rddp) capabilities with IP transport protocols, in particular with SCTP. RDDP capabilities refer to the ability to place data directly from the NIC into application buffers, without intensive CPU usage. This strategy avoids the costs of data copying and enables using IP as a method for high speed buffer to buffer transfer, allowing IP to replace special purpose networks currently in use. Remote Direct Memory Access (RDMA) is an example of this concept.

Conceptually, RDDP functionality can be viewed as consisting of two layers. First the direct data placement capability, which is accomplished through a tag and a lookup table on the NIC. Above this core functionality, an RDDP control protocol is needed to specify how the direct data placement can be used, for example READ and WRITE commands.

The work of the WG is to accomplish four items:

1) A (transport independent) protocol core to support direct data placement from the NIC into specified memory, usually application buffers.

2) A (transport independent) protocol core layered on top of the direct data placement protocol that specifies control of RDDP.

3) A mapping of the direct data placement protocol onto SCTP, for standards track, including a clear applicability statement of the expected service from the mapping.

4) A mapping of the direct data placement protocol onto TCP, for informational, because TCP's service is a less good match to RDDP, including an applicability statement of the issues regarding the service available from the mapping.

The working group will ensure that the resulting technology will be secure and will not enable new attacks on systems supporting RDDP. The WG will not modify existing Internet transport protocols, but may forward issues it discovers in such transport protocols that are not full Internet Standards to the appropriate IETF WGs for their consideration.

Goals and Milestones:
OCT 02  Submit Internet-Draft including problem statement and achitecture
OCT 02  Submit Initial draft of Remote Direct Data Placement protocol (RDDP)
OCT 02  Submit Initial draft of RDMA control protocol, to be named.
DEC 02  Initial draft mapping the RDDP core and control onto SCTP including A/S
FEB 03  Initial draft informationally mapping the RDDP core and control onto TCP, including A/S
MAR 03  Submit problem statement/architecture to IESG for consideration as an informational publication
MAR 03  Area Director review of charter and progress, with possible rechartering or closure.
JUL 03  Submit Remote Data Placement Protocol (RDDP) to IESG for consideration as a Proposed Standard
JUL 03  Submit RDMA control protocol (named TBD) to IESG for consideration as a Proposed Standard
OCT 03  Submit mapping the RDDP core and RDMA control onto SCTP including detailed applicability statement to IESG for consideration as a Proposed Standard
OCT 03  Submit mapping the RDDP core and RDMA control onto TCP including detailed applicability statement to IESG for consideration as informational RFC
No Current Internet-Drafts
No Request For Comments

Current Meeting Report

11/20/02 RDDP WG
David Black, Chair
Notes by Allyn Romanow

-- Agenda bashing
-- Work and goals
     draft-black-rdma-concerns-00.txt will be submitted as an RDDP draft to 
make it easier to find it, but the authors are not planning to submit for 
RFC status.  It's purpose it to cause people to consider issues.

-- WORKSHOP announcement (Allyn Romanow)
	Network-I/O Convergence: Experience, Lessons, Implications (NICELI)
	- SIGCOMM 2003 Workshop August 25-29, Karlsruhe Germany
	- Call for Papers - technical paper or position paper
 	- Will be at 
	- Or contact 

-- Review of Problem Statement and Architecture drafts (Tom Talpey)


Two new versions

Problem statement done
Architecture issues
     ordering of placement and delivery
     steering tag details
     security considerations and threat analysis
See David Black if you want to work on security section of the doc

These two drafts will become RDDP WG drafts.

--  SCTP Mapping draft-stewart-rddp-sctp-00.txt (Randall Stewart)

Short doc, provides SCTP mechanism for knowing what each side expects to do
Details of ways to use SCTP features not filled in yet how use SCTP most 
effectively how optimize transfer with un-ordered packets other details
Soliciting help, work with Randy. Get in touch with David Black or Randy

--  Overview of DDP and RDMA (Renato Recio)

See overview slides - these explain high level functionality and 
relationship of these two drafts.

-- DDP draft (Hemal Shah)

See slides for details.

Overview of the DDP mechanism
Distinction between placement and delivery is extremely important
	Placement - self describing segments, can place in any order
	Delivery - in-order delivery, DDP notifies ULP only when all messages 
		Only interested in LLP that provides reliable in-order delivery

STag Validation semantics - 4 models 
	After discussion, the rough consensus of the room was to carry two of the 
four models, STag bound to a single stream and access group 
(protection domain model forward.  A functional reference model will be 
needed to specify the required access control behavior; 
implementations must exhibit the externally visible behavior of the model 
(this is similar to the specification of the IPsec Security Policy 
Database and Security Association Database - see RFC 2401).

	A performance concern was raised about the consequences of access 
control.  Combining the above functional model with a functional model of a 
NIC (hardware) should help illuminate the issues here.
Clarifications on DDP Segmentation
	- Send segments in increasing order, for both untagged and tagged 
	- No overlapping of payload is allowed among DDP segments in the same 
	- Interleaving DDP segments of different DDP messages not allowed at the 
data source.  Use multiple streams for concurrency, not multiplex on the 
same stream

Requirements for Unreliable Transport
	Only considering reliable transports currently.  Anyone interested in 
unreliable transports should "Send Draft" .. says WG chair.

Local interface requirement for buffers
	Need to specify? yes, need to specify access control and 
protection, but not from the perspective of specifying a full API 

Reserved ULP field in DDP Header

Remote invalidate will need to be taken up, can be dealt with online

Q&A: DDP should cleanly layer on unordered SCTP.  Should layer on 
ordered SCTP with a little care and attention.  SCTP can support 
immediate placement and in-order delivery, provide that access to 
cumulative TSN is available to DDP.

This will become and official WG draft (next version containing access 
control text)

 -- RDMAP (Renato Recio)

There have been 3 issues on mailing list

(1) RDMAP Initialization - How to transition from non-RDMA to RDMA
	TCP mapping needs to look at SCTP, understand what it did (DDP and RDMA 
support announced during connection setup) and either do something 
similar, or explain why not.

(2) Security

Have begun working with Catherine Meadows, security expert, on the 
Security Directorate, lots more work needed on security.

Security Concerns 
     -what layer is the association made at?
     -how can IPsec be used to protect an DDP RDMA stream?
     -Enumerate the threat model (e.g. DOS attack )

(3) Multi-homed hosts for SCTP
	Proposal - this issue doesn't belong here, it belongs in SCTP WG
(TSVWG)-a problem with multi-homed hosts.

	WG Chair - SCTP mapping draft should deal with this and say how it 
should be handled or avoided.

This will become an official WG draft (next version including access 
control text)

David Black (WG Chair)- What might be the draft mapping RDMA to TCP look 

Reviews what happened in TSVWG meeting the previous day.  It will take a 
while to work out any proposed changes to TCP. It is premature to adopt the 
Williams draft as WG doc, although the WG chair thinks it has good stuff in 
it, and the draft suggests how to do the mapping without changes to TCP

Scott Bradner (Area Director) - reports on a discussion that took place 
after the TSVWG meeting.

ADs (tsvwg chairs) are very reluctant to approve changes to TCP.
It would be necessary to put together an I-D that proves that no 
possible combination of tweaked and non-tweaked TCPs could possibly get 
confused, meaning that it is necessary to prove that there is no way that 
different TCPS can get confused between each other. Agrees that it is 
premature to accept the Williams draft as a WG item. 

Q: Can we use the rddp WG as where the discussion takes place?
Bradner - the initial discussion was in TSVWG for the broader 
	Detailed work can be done here, in RDDP, then go to TSVWG, for review

-- Mapping DDP/RDMA to TCP, IFT (Jim Williams)
See slides

Open issues 
CRC -  slide proposed two CRCs if assume non-aligned PDUs as in IFT/TCP
	One CRC should be sufficient for assume aligned PDUs as in SCTP or 

	- A long inconclusive discussion ensued - the issue was redirected to the 
mailing list.

Issue - markers
Can't really discuss till figured out in TSVWG

WG Chair (David Black): MPA uses an aggressive form of markers.
Comment: Reason for markers is that they allow the ULP to progress to find 
the next PDU, after an error is found as opposed to just dropping the 

Other IFT details - not worth dealing with now.

Q: Does TCP need to be "FIXED"? does TCP placement as in SCTP cover the 
needs of RDDP?

David Black - MPA and IFT appear to be at opposite ends of the design 
space, thinks there might be a design strategy in between.
Will need to follow Scott's suggestion for any TCP changes.

Comment: Wants support for bufferless NICS as well as buffered NICs. 

David Black - there's no such thing as a bufferless NIC, as dealing with a 
DMA controller requires buffering - should talk in terms of the amount of 

-- Process going forward (David Black)

Take discussion to the list
Focus on specific features rather than one approach or another
Drafts are extremes in the design space, there should be a way in 
Too early and too much unknown to charter a design team now

Question: Should RDDP write a document on what a ULP needs to do to use 
	A reference model was mentioned- will some of this come out of a 
reference model?

David Black - ref model in two places, NIC functionality and security.

	- Another issue is the memory model. Joe Touch in TSVWG, asked, who owns 
what memory, when? It needs a state machine.  A related issue is what 
happens when there are concurrent operations on different streams to the 
same memory?  That needs to be specified minimally in order to avoid 
running afoul of multiprocessor memory coherence issues. 
	Application designer wants description of what NOT to do in both cases.


iWARP Framing for TCP
Workshop - NICELI
Problem Statement and Architecture Document Updates
State of RDMA
Direct Data Placement (DDP) over Reliable Transports
RDMAP and DDP Overview