[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
RE: [rddp] verb spec clarification
First and foremost, I want to be certain that everyone agrees
that zero length RDMA Reads must be accounted for in IRD/ORD
resource tracking. They are not processed out-of-order and
cannot be considered as some sort of "out-of-band" message.
For most local interfaces the IRD and ORD limits are set
locally when the QP is created and MAY be modified. As far
as IETF specifications are concerned, the existence of some
sort of IRD/ORD tracking is assumed. However, there is no
mandate that it be part of the local interface. It is merely
difficult to imagine complying with some of the security
requirements without doing so.
That leaves the question as to what was meant by having
both the enable and the IRD count in the earlier proposed
verbs draft. While techincally off topic, I think this is
of general interest, so I'd like to comment.
I have seen many translations of those verb requirements
to APIs that treated the enable as redundant, and merely
enabled/disabled "inbound RDMA Read capability" based on
whether the IRD was zero or non-zero.
Pat's earlier comment suggested that the enable bit was
a hint that the IRD would be modified to a non-zero value
at a later time and therefore the RI should ensure that
the appropriate data structures were setup, even if credit
was not granted. I can see that argument, but I would recommend
applications instead create the QP with the maximum IRD that
they intend to use, and then use modify (before connecting)
to lower the limit to block excess incoming RDMA Reads.
Paul's response seems to imply that you could create a non-zero
IRD and *not* set the enable flag and that the result would be
that *only* zero length RDMA Reads would be allowed.
If that was the intent it is sufficiently unclear that I would
strongly recommend that application developers not rely on that
method. If you only want to enable zero-length RDMA Reads on a
given connection, simply do not enable any STags on that connection.
That capability clearly exists, and the ability to ensure that
no STags are valid on a connection is explicit in IETF specs.
> -----Original Message-----
> From: rddp-bounces at ietf.org [mailto:rddp-bounces at ietf.org] On
> Behalf Of Culley, Paul
> Sent: Wednesday, March 30, 2005 5:51 AM
> To: pat_thaler at agilent.com; cait at asomi.com
> Cc: pkaplan at chelsio.com; rddp at ietf.org
> Subject: RE: [rddp] verb spec clarification
>
> I've always thought that "Inbound RDMA Write and inbound RDMA
> Read Response" were the same thing (one bit) for purposes of
> the QP enable.
> Basically an inbound RDMA Read Response IS an inbound RDMA
> Write (same DDP opcode). The idea for this enable was
> inherited from VI.
>
> The sections I referenced DO address this; on page 111 for
> Memory Region checks the following is stated:
>
> "When a local or remote operation attempts to access a
> registered Memory Region, the RI MUST ensure that:
> ...
> The Access Rights of the QP allow the type of access being
> performed by the operation,"
>
> And the zero length check (stated in thread below) over-rides this.
>
> There is similar text under the Memory Window section.
>
> Paul R. Culley
> HP Fellow
> 281-514-5543
>
>
> > -----Original Message-----
> > From: pat_thaler at agilent.com [mailto:pat_thaler at agilent.com]
> > Sent: Tuesday, March 29, 2005 7:08 PM
> > To: cait at asomi.com; Culley, Paul
> > Cc: pkaplan at chelsio.com; rddp at ietf.org
> > Subject: RE: [rddp] verb spec clarification
> >
> > I think Paul missed the context of the question. Not surprising
> > because the question is a little confusing:
> > "Section 9.2.5.1 states that QPs have Inbound RDMA Read enable and
> > Inbound RDMA Write and inbound RDMA Read Response enable.
> >
> > Does the inbound RDMA Read Response enable check apply to a zero
> > length read response?"
> >
> > The sections Paul mentioned don't address this question.
> >
> > There is no RDMA Read Response enable check. Verbs seems to
> lack any
> > description of the effect of these enables.
> >
> > RDMA Read Response enable is a feature of queue creation.
> > Presumably if RDMA Read Response handling is not enabled,
> there will
> > be no resources provided for initiating RDMA Reads, i.e.
> ORD will be
> > zero. Attempting to initiate an RDMA Read will fail.
> >
> > These enables are not described very much and they may seem
> redundant
> > with setting IRD and ORD to zero. A modify QP can change
> IRD and ORD.
> > Setting up the QP with Inbound RDMA Read Response enabled
> and ORD zero
> > would allow a later attempt to modify the QP to increase ORD. My
> > expectation is that when a QP was set up with Inbound RDMA Read
> > Response disabled, the RNIC can assume that RDMA Read
> initiation will
> > never be allowed and any attempt to allocate ORD depth could be
> > failed. This is the way I think it should work, but the
> reaction of an
> > RNIC to a Create QP with Inbound RDMA write and RDMA Read Response
> > disabled and ORD non-zero or a Modify QP for non-zero ORD
> when this is
> > disabled are not described.
> >
> > It also isn't clear why Inbound RDMA Write and inbound RDMA Read
> > response share a single enable. They require different
> resources since
> > the RDMA Read Response requires ORD depth.
> > They aren't always needed together. An iSCSI target will receive
> > inbound RDMA Writes and will not receive RDMA Read
> responses. An iSCSI
> > initiator has the reverse situation.
> >
> > Regards,
> > Pat
> >
> > -----Original Message-----
> > From: rddp-bounces at ietf.org
> [mailto:rddp-bounces at ietf.org]On Behalf Of
> > Caitlin Bestler
> > Sent: Tuesday, 29 March, 2005 7:41 AM
> > To: Culley, Paul
> > Cc: Patricio Kaplan; rddp at ietf.org
> > Subject: Re: [rddp] verb spec clarification
> >
> >
> > Culley, Paul wrote:
> > > For Verbs, The answer is no.
> > >
> > > Section 7.6.3 pg 112 near top
> > > "If the length of the access is zero, the RI MUST NOT
> > perform any of
> > > the above checks on the Memory Region."
> > >
> > > Section 7.10.6.2 pg 124, 2/3 down
> > > "If the length of the access is zero, the RI MUST NOT
> > perform any of
> > > the above checks on the Memory Window."
> > >
> >
> > Additional point: checking IRD credits is a check on the
> > connection(QP) not on the Memory Region/Memory Window.
> >
> > _______________________________________________
> > rddp mailing list
> > rddp at ietf.org
> > https://www1.ietf.org/mailman/listinfo/rddp
> >
>
> _______________________________________________
> rddp mailing list
> rddp at ietf.org
> https://www1.ietf.org/mailman/listinfo/rddp
>
_______________________________________________
rddp mailing list
rddp at ietf.org
https://www1.ietf.org/mailman/listinfo/rddp