2.3.8 IP over InfiniBand (ipoib)

NOTE: This charter is a snapshot of the 61st IETF Meeting in Washington, DC USA. It may now be out-of-date.

Last Modified: 2004-10-01


H.K. Jerry Chu <jerry.chu@sun.com>
Bill Strahm <bill@strahm.net>

Internet Area Director(s):

Thomas Narten <narten@us.ibm.com>
Margaret Wasserman <margaret@thingmagic.com>

Internet Area Advisor:

Margaret Wasserman <margaret@thingmagic.com>

Mailing Lists:

General Discussion: ipoverib@ietf.org
To Subscribe: ipoverib-request@ietf.org
In Body: subscribe ipoverib
Archive: http://www.ietf.org/mail-archive/web/ipoverib/index.html

Description of Working Group:

E-mail archive:

InfiniBand is an emerging standard intended as an interconnect for
processor and I/O systems and devices (see the Infiniband Trade
Association web site at http://www.infinibandta.org for details). IP
is one type of traffic (and a very important one) that could use this
interconnect. InfiniBand would benefit greatly from a standardized
method of handling IP traffic on IB fabrics. It is also important to
be able to manage InfiniBand devices in a common way.

This working group has two tasks:

- specify the protocols and encapsulations to transport IPv4/v6 over
  an InfiniBand fabric.

- specify a set of MIB objects to allow management of the InfiniBand
  fabric itself.

The initial scope of the WG was limited to the use of the basic IB
Unreliable Datagram (UD) transport mode for transporting IP over
Infiniband. With that work mostly done, the scope has been extended to
develop an optional mechanism for transporting IP over other IB
transport modes. In particular, there is a desire to transport IP over
one or both of IB's connected modes, which enable the use of a much
larger MTU than the IB link MTU size. They also provide improved
reliability and performance through the use of link level orderly and
reliable delivery, and IB's automatic path migration (APM) feature.
However, care must be taken to ensure that use of an IB reliable
transport does not unduly interfere with the retransmission and
congestion control mechanisms used by higher layers (e.g., TCP and

Other more advanced functionalities such as mapping IP QOS into
IB-specific capabilities remain out of scope of the WG charter.

Work items

1. Specify standards track procedures for transporting IP over
IB. This includes:

- supporting ARP/ND packets in order to map IP addresses into IB
  link-layer addresses.

- define encapsulations for carrying ARP, IPv4 and IPv6
  packets over IB

- Define  how to transport IP multicast over IB.

2. Specify a standards track channel adapter MIB that allows
management of an InfiniBand channel adapter. There will require that
InfiniBand types be added to the ifType defined by IANA

3. Specify a standards track baseboard management MIB that will allow
management of specified device properties

4. Specify sample counter MIBs to allow InfiniBand sample counters to
exposed to external SNMP management applications

5. Specify an optional, standards track encapsulation for carrying IPv4
and IPv6 packets over either IB unreliable connections or reliable

Goals and Milestones:

Done  Submit initial Internet-Draft of ARP encapsulation
Done  Submit initial Internet-Draft of Requirements/Overview
Done  Submit initial Internet-Draft of IP V4/V6 Encapsulation
Done  Submit initial Internet-Draft of Infiniband-Like MIB
Done  Submit initial Internet-Draft of Channel Adapter MIB
Done  Submit initial Internet-Draft of Multicast
Done  Submit initial Internet-Draft of Baseboard MIB
Done  Submit initial Internet-Draft of Subnet Mangement MIB
Done  Submit ARP/IP/Multicast encapsulation drafts for IESG Last Call
Mar 02  Submit Infiniband-Like MIB for IESG Last Call
Mar 02  Submit Channel Adapter MIB for IESG Last Call
Nov 02  Submit initial Internet-Draft of Advanced Encapsulation over Connected Transports
Mar 03  Submit initial Internet-Draft of Sample Counter MIB


  • draft-ietf-ipoib-ibif-mib-07.txt
  • draft-ietf-ipoib-subnet-mgmt-agent-mib-07.txt
  • draft-ietf-ipoib-architecture-04.txt
  • draft-ietf-ipoib-ip-over-infiniband-07.txt
  • draft-ietf-ipoib-channel-adapter-mib-06.txt
  • draft-ietf-ipoib-baseboard-mgmt-agent-mib-02.txt
  • draft-ietf-ipoib-perf-mgmt-agent-mib-03.txt

    No Request For Comments

    Current Meeting Report

    IETF61 IPoIB meeting minutes:

    Agenda -

    1. Drafts update by Vivek Kashyap

    The encapsulation draft has gone through IESG review and received numerous comments. The revision has been made and awaiting IESG review.

    Jerry Chu asked for clarification from Thomas Narten on the "IANA consideration" for the "reserved flags" field of the link-layer address. Some discussion continued on the wording about the configuration of the IPoIB links. The room concensus was that this is best left to the admistrative control, and the draft will RECOMMEND so.

    The connected mode draft has undergone some review. Jerry made two suggestions. One is to make the per-connection MTU a "receive MTU". This will remove all the complexity of ensuring both endpoints agreeing upon a single MTU. The second was to fold the current "fully-connected overlay subnet" into the default IPoIB UD subnet, allowing each node pair to simply negotiate an optional IB connection between themselves separately, since there won't a single, nice link MTU anyway due to multicast.

    Vivek countered with some pros and cons on the latter. Some discussion ensued on the managemnet of the IB connections and how to allow multiple connections to be used and the mapping between multiple IP addresses (multi-homing) and multiple IB connections per node pair.

    There was some question/comment on the new "shared receive queue" feature in IBA 1.2 (?)

    2. MIB drafts update by Bill Strahm (on behalf of Sean Harnedy) The drafts are updated. Bill owns reviewing them for new boilerplate and submitting them to the MIB doctor and IESG.

    3. Solaris IPoIB implementation report by Bill Strahm (on behalf of Kanoj Sarcar)

    See slides. One major area of difficulties has been the multicast related event traps, given the unreliable nature of the latter.

    Bill Asked for more implementation reports from other OSes. Voltair confirmed IPoIB interoperability with Solaris.

    4. The advisor AD (Margaret Wasserman) commented on the next step for moving the encapsulation draft through the IESG again.

    Meeting adjourned.


    IP Over InfiniBand (IPoIB) Working Group Management Information Bases
    Solaris IPoIB (ibd) Implementation