Current Meeting Report

2.2.7 IP over InfiniBand (ipoib)

NOTE: This charter is a snapshot of the 52nd IETF Meeting in Salt Lake City, Utah USA. It may now be out-of-date. Last Modified: 10-Oct-01
Bill Strahm <>
Internet Area Director(s):
Thomas Narten <>
Erik Nordmark <>
Internet Area Advisor:
Thomas Narten <>
Mailing Lists:
To Subscribe:
In Body: subscribe ipoverib
Description of Working Group:
E-mail archive: t.html

InfiniBand is an emerging standard intended as an interconnect for processor and I/O systems and devices (see the Infiniband Trade Association web site at for details). IP is one type of traffic (and a very important one) that could use this interconnect. InfiniBand would benefit greatly from a standardized method of handling IP traffic on IB fabrics. It is also important to be able to manage InfiniBand devices in a common way.

The work group will specify the procedures and protocols to support IPv4/v6 over an InfiniBand fabric. Further, they will specify the set of MIB objects to allow management of the InfiniBand protocol.

The scope of this WG is limited to the definition of an encapsulation format for carrying IPv4 and IPv6 over IB networks and for performing address resolution between IP address and IB link-layer addresses. At the present time, more advanced functionalities such as mapping IP QOS into IB-specific capabilities is out of scope. Such work items may be considered in the future, but will require a recharter.

Work items

1. Specify a standards track procedure for supporting ARP/ND packets, and resolving IP addresses to IB link addresses.

2. Specify a standards track encapsulation for carrying IPv4 and IPv6 packets over IB.

3. Determine how to and specify a standard for transfering IP multicast over IB. IB has an optional receiver join multicast capability. Current working group plans are to use IB multicast as part of ARP, so using it for IP multicast as well may be a reasonable approach.

4. Specify a standards track channel adapter MIB that will allow management of an InfiniBand channel adapter. There will also need to be InfiniBand types approved and added to the ifType defined by IANA

5. Specify a standards track baseboard management MIB that will allow management of specified device properties

6. Specify sample counter MIBs to allow InfiniBand sample counters to be exposed to external SNMP management applications

Goals and Milestones:
Done   Submit initial Internet-Draft of ARP encapsulation
Done   Submit initial Internet-Draft of Requirements/Overview
Done   Submit initial Internet-Draft of IP V4/V6 Encapsulation
Done   Submit initial Internet-Draft of Infiniband-Like MIB
Jul 01   Submit initial Internet-Draft of Channel Adapter MIB
Done   Submit initial Internet-Draft of Multicast
Nov 01   Submit initial Internet-Draft of Baseboard MIB
Nov 01   Submit initial Internet-Draft of Sample Counter MIB
Feb 02   Submit initial Internet-Draft of Subnet Mangement MIB
Mar 02   Submit ARP/IP/Multicast encapsulation drafts for IESG Last Call
Mar 02   Submit Infiniband-Like MIB for IESG Last Call
Mar 02   Submit Channel Adapter MIB for IESG Last Call
No Request For Comments

Current Meeting Report

Minutes from IPoIB meeting on 12/10/01. Reported by Jim Pinkerton Microsoft Corp.

Bill reviewed the agenda.

Shawn has taken over from the original author for the IPoIB MIBS. IANA has defined an IBMIB as experimental 117.The IB MIBs need to be assigned hierarchically from there.

Bill Strahm asked whether IANA object Ids can be informally assigned or does each need to go to IANA. Bill recommends Shawn be the designee for assigning IANA MIB numbers. There were no objections.

Bill summarizes the IPoIB MIBS that are required:
- Bill comments that the sample counters are not useful.
- Channel adapter MIB
- specify baseboard management MIB
- device and switch MIBs

Bill asked that the textual conventions be pulled out separately. This would allow the conventions to be defined independently of the MIBs. Bill walked briefly through the high level textual conventions to give a flavor for the approach. Comment from the floor was that the language in RFC2578 is "SHOULD" language, so if there is a good reason to do zero-based indexing you can. The concern is that using a one-based offset doesn't match the hardware. Thus zero-based indexing is now deemed "acceptable".

Bill then walked through the subnet interface MIB. Talked about the iblfStatGroup and iblfVLTrafficGroup and the types of stats in each. Textual convention MIB was removed from this document. Authors have transitioned on this doc. Bill then walked through the main differences in the update to the -00 version. Open issues:

- should 32-bit error counters be made 64-bit (counter wrap frequency analysis)? Jeff Young claims that software stacks are implementing 64 bit counters. Point was made that the analysis could be led by what is the fastest rate that these counters can increment? RFC2578 has some insight into this.

- consensus question on this mib - they are concerned about the level of review that has occurred. Bill is not ready to call consensus on this document because of the level of review. Bill believes the document is close, but needs additional review.

Carl Yang presented the IB Subnet Management MIB.

He first reviewed the possible roles that SNMP can play in managing an IB network. Missed first option. Second option is to enable device management over SNMP by allowing the subnet manager to change the values. But not all variables are defined through the current MIB (e.g. multicast group creation and deletion). Third is to enable SNMP to configure and monitor IB subnets by communicating with the SM. The third approach is the primary focus of this MIB. Point was made that with the SMA MIB, only one variable is actually writable.

One of the main differences between the SM MIB and the SMA MIB is that the SMA only deals with one value of a variable whereas the SM MIB deals with a series of values. There was some discussion of how to document this. One suggestion was to document the MIB for the SMA, but use an index and only allow the SMA to use index zero, for example. Then the SM can reference this.

There were some questions on the exact transport mechanism. Does the SM proxy this. Bill commented that he'd like to enable a range of implementations. There was concern that there not be a requirement for an IB out-of-band management solution. Bill pointed out that there are a variety of solutions possible - from doing it on top of UDP on IB, to tunneled, to many other solutions.

There is some concern about whether the management key should be exposed or should it be a secret? Bill claims since the management key is sent in the clear, it's not a crypto key, and thus it's okay to expose it. Bill asks for consensus. Jeff Young was concerned. There was a proposal that the SNMP interface could be to have the variable read/write, where the read always returns a zero. Bill pointed out that it's fairly easy to snoop the value, thus not clear it needs to be protected. Another statement pointed out that if SNMP is used over the internet and this really is a sensitive value, maybe it shouldn't be visible by SNMP. Jeff said that clearly there needs to be a mechanism to set the M_KEY. Not clear that it needs to be through SNMP.

On the configuration table, the number for row status - what is the persistence of this information? Walked through the three possibilities - no persistence, persistence, and allowing another variable to control the persistence. Bill asked the crowd on why we would want SM state to not be persistent across. No one came up with a reason or objected (after some discussion and a walk through of a few scenarios).

Carl continued to do a brief overview of the ID. Comment Switch Info PortStateChange should be a Boolean - only one value is currently defined. Bill asked whether exposing P_Keys is a problem. Jeff Young said he has the opposite opinion. They'll take it off line.

There is a problem with the trap. <insert text here>. See RFC2576.

Next steps
- modify to use IB MIB textual convention
- can we consildate SMA MIAB and SM MIB?
- - some MIBS are identical, other than their index
- add missing MIBs
- - ServiceRecord, LinkRecord, and RenageRecord
- Security consdertions
- more trap definitions
- compliance statements
Bill comments that because of the outstanding issues, there needs to be at least one more draft. This draft is currently an individual contribution. Bill's concern is whether this MIB could be collapsed with the next MIB (SMA) being presented (from Bill Swortwood).

Bill Strahm presented Bill Swortwood's SMA proposal.

This is a working group item. Next steps are:
- creation of compliance/required/optional groups
- HCAs normally do not need wswitch groups
- Addition of some references
- <other stuff>

Point was made that this MIB has the same problem around TRAPs as the last one.

Bill points out that this does not allow management - it allows monitoring the network because you can't set the values of the variables.

Question was asked why you wouldn't just do this through the SA? It was pointed out that there was really two proposals on the table - Bill's proposal and Carl's proposal. Bill's addresses the SMA, Carl's addresses the SA and SM.

Bill outlined three options - keep it as it is, single MIB that has compliance statements separately for SM and SMA, third is to have a single SMA MIB, with an SM MIB that includes the SMA and extends it for SM unique issues. A fourth option is to leverage the existing IB Management infrastructure to the SA and map that to SNMP.

Bill Strahm presented Bill Swortwood's Device management MIB.

Bill overviews the proposal. Focused on monitoring hosts with IB device management agents. It is missing baseboard management.

Next steps:
- Creation of compliance statements
- Addition of some references
- Any other rational changes

Bill Strahm presented Connection MIB.

This is just an initial draft. Missing security consideration section and need better explanatory text. The purpose is to model IB connections, but this is not directly modeled after connection manager.

The point was made that it should be either an "all or none" access to sensitive tables. A partial controlled access is much more complex.

- Objects that aren't in the IB spec
- Is this a WG item
- Does this cover the WG Item for Channel adapter MIB - should we rename the WG item?

Bill stated his opinion that he'd prefer to keep the separate MIBs separate because it allows focused discussion and focused iteration. It also allows the initial MIBs to get out to last call faster.

There was a proposal to have an interim meeting co-located with the next IBTA management-working group to get better attendance. It was noted that the management-working group is currently meeting monthly. Another possibility is to request the IBTA management-working group to co-locate at the next IETF. Bill will formally ask the Management co-chairs to work this out with him.


IPOIB Interfaces ID
IP Over InfiniBand (IPOIB) MIBs
IPOIB Textual Convention
InfiniBand Subnet Management MIB