Current Meeting Report
Slides
2.7.23 RDMA over the Internet Protocol Suite (roi) Bof
Current Meeting Report
Meeting Minutes For RDMA Over The Internet Protocol Suite BOF, at IETF 52, Salt Lake City.
Reported by: Stephen Bailey (steph@cs.uchicago.edu)
Notes taken by: Jeff Hilland
1. Agenda bashing - by Allyn Romanow
- Overview of existing solutions and Requirements items may be skipped if discussion of other items is extensive.
2. Description of the problem being addressed domain, issues, justification, motivation: draft-garcia-direct-access-problem-00.txt - by Stephen Bailey
Points raised by, or as a result of questions from the participants:
- ROI is targeted to help protocols that are run on top of congestion avoiding, IP-based transports, specifically TCP & SCTP. Problem is not specifically with IP, but with these transports.
- should be clearer about whether problem being solved is receive copy avoidance, or protocol overhead offload, or both.
- should be clearer about which, if either of these problems, SCTP can solve directly.
3. Discussion of various alternative approaches, including direct data placement - by Jeff Chase
Presented various alternatives to RDMA, specifically with TCP, and why they do not solve the problem. See the slides.
Points raised by, or as a result of questions from the participants:
- RDMA targets data transfer within the data center, specifically among fully cooperative & RDMA-aware endpoints. It is definitely not intended to accelerate, for example, HTTP transfers to edge devices.
- Is ROI is a relatively `work preserving' solution (minimum modification to existing components)?
It was pointed out that it is possible to accelerate completely unmodified applications using an RDMA-based SOCK_STREAM protocol like Sockets Direct Protocol (SDP), which is highly work preserving. For other applications that want more acceleration, more modification is required. Some APIs may be preserved, but may involve either extension or restriction of existing APIs, especially sockets, or kernel sockets for best effect.
- If the desire is to make a low cost deployment, why must the sender participate?
The sender must participate when the receiver can not predict the order or static structure of data sent by the source AND it is undesirable to provide ULP-specific acceleration hardware for every ULP to be accelerated.
- Why don't you invent a new transport?
It is not considered necessary. IP transports solve the problem of reliably transporting data from one endpoint to another, and RDMA solves the problem of steering that data directly to the ultimate application buffer.
- Is the plan to directly adopt an existing non-IP RDMA model?
No. The plan is to consider, and possibly adapt, existing solutions, but not necessarily to adopt any particular one wholesale.
- Some are still unconvinced that a protocol solution is necessary.
4. Overview of current industry solutions including RDMA - by Jim Pinkerton
This section was skipped
5. Requirements - draft-wendt-direct-access-reqs-00.txt - by Jim Wendt
This section was skipped
6. Discussion of goals and proposed charter for a WG - by Allyn Romanow
Points raised by, or as a result of questions from the participants:
- Does SCTP support RDMA or Direct Data Placement without any protocol work?
It was argued vigorously by one individual that SCTP does provide a complete solution without any protocol work. The general sense of other participants was that while the amount of protocol work is potentially small, particularly, if the goal is only receive-side copy avoidance, it was not zero.
- Can TCP support RDMA or Direct Data Placement without any protocol work?
It was argued vigorously by one individual that TCP could be, and in fact, had, been made to allow direct data placement of flexibly structured, data streams. The solution was not detailed, but it appeared to involve the sender being provided receive sequence number ranges into which various responses should be delivered, and then the sender would issue TCP segments in the order in which it wished to respond to requests, rather than in TCP sequence-number order. It was pointed out that this approach would invalidate TCP's current fast-retransmit algorithm. There also seemed to be some concern about the generality of the TCP sequence number solution, for example, if a particular response is shorter than the receiver-provided size, the TCP sequence space must be padded with filler.
- Does RDMA or Direct Data Placement protocol work belong in the IETF?
No particular conclusion was reached, but neither was this objection a rallying point for much further discussion.
- It was pointed out that the problem statement, and BOF did not really clarify the question about whether ROI is solving the receiver copy problem, or attempting to reduce other protocol overhead, or what. It was suggested that either this be refined, and any charter be written in such a way that the group not start out with confusion about this mission.
A 3-way hand vote was taken to see whether those in the room favored progressing the BOF to an IETF working group:
Drop this topic and do not pursue it further within IETF: 1 hand
Have another ROI BOF: ~40% of voting hands
Try to proceed to charter ROI: ~60% of voting hands
At least 75% of all present voted one way or the other.
Slides
ROI Problem Statement Discussion
Fast-Path Data Movement for IP Transports
WG Goals and Charter