Re: [bmwg] New work item proposal: Benchmarks for DCB switches

[Al Morton <acmorton at att.com>]

At 08:02 PM 8/14/2009, David Newman wrote:
> ...Timmons Player and I would like to propose extensions to 2544 and 2889
> to test many switches now or soon coming to market with "data center
> bridging" features such as priority-based flow control.
>
> A full work item proposal follows. The short version is "RFC 1242
> throughput is no longer meaningful if packet loss never occurs." ...

Hi David and Timmons,

This is an interesting proposal, IMO *as a participant*,
and I have a few comments on the memo now that I've
made time to read it. See below.

*As chair*, I'm reminding folks that this proposal and memo are:
   o defining updates to our fundamental RFCs
   o available to be reviewed now
   o going to be discussed at our Interim meeting, along with several others

And when we approach a meeting, everyone reads ALL the memos, yes?

Lets go,
Al

COMMENTS and QUESTIONS:

S4 Terminology
      Conformance Vector: A set of measurable stream result bounds, e.g.
      latency, jitter, sequencing, etc., that specify whether a frame is
      Conformant or Nonconformant.
This is the multi-dimensional performance criteria that must be
measured as part of the definition of a benchmark, and not the
criteria for a DUT/SUT to meet.  A few more words could help it make
clear(er) that this is not used a DUT/SUT PASS/FAIL criteria.
Also a nit:  s/Nonconformant/Non-conformant/

Current:
      Forwarding Congestion: In the context of DCB benchmarking,
      Forwarding Congestion is extended to include PFC pause frame
      transmissions from the DUT.
Suggest:
      Forwarding Congestion: In the context of DCB benchmarking,
      the detection of Forwarding Congestion is extended to include
      the observation of PFC pause frame transmissions from the DUT.

Current:
      Queueput: The maximum Offered Load than can be transmitted into a
      DUT/SUT such that every transmitted frame matches a specific
      Classification rule, the DUT/SUT does NOT use priority-based flow
      control mechanisms to manage the ingress traffic rate, and all
      ingress frames are forwarded to the correct egress port.  A DUT
      may have a different Queueput rate for each configured
      Classification.
Suggest:
      ... the DUT/SUT does NOT use priority-based flow
      control mechanisms to manage the ingress traffic rate
>>>>  of the Classification(s) of interest, ...

S5 Test Setup
   ... In DCB networks, the DUT
   will transmit PFC Pause frames as a Congestion Management method to
   throttle network endpoints, thus minimizing the probability of frame
   loss in the network.
Comment:
There's a feedback loop here, and if the feedback mechanism is
activated (PFC Pause frames), then rate is too high in some regard.
Need to think about this some more, but how the rate is approached
from below/above seems important (as always?).

Current:
S 5.1.4.  Frame Formats

   This testing document does not mandate the use of any particular
   frame format for testing.  ...
It appears that the rest of this sub-section could be titled
"Tester Capabilities".  There's some value in having a section
like this, IMHO.

S 6.1.2.  Setup Parameters

   The following parameters MUST be defined.  Each variable is
   configured with the following considerations.
...
         Burst Size - The burst size specifies the number of frames
         transmitted with the minimum legal IFG before pausing.  A burst
         size of 1 specifies a constant load.

As I read the Burst Size spec,
it tells me the number of frames in a burst,
assuming that some additional load spec or rate limit exists.
Otherwise, a burst size of 1 produces a single frame,
not a constant load.  A little more exp. would help.

...
      Priority-based flow control - PFC mechanisms MUST be enabled.
This is something to discuss further, but the methodology *might*
need some vendor-agnostic configuration details, at least specifying
what configuration aspects will be reported with the results
(sufficient for others to repeat the tests).

Possibly the same for Backoff method, mentioned later.


NITS:
The source is in XML, right? It would be great to drop the page break
after major sections. There must be a flag for this, I don't get this
in my XML docs. It might be:
<?rfc compact="yes"?>

S1 Intro
   This RFC primarily deals with devices that use priority-based flow
s/RFC/memo/ or other general term like document, not an RFC yet.

S 5.1.5.1
s/recommended/RECOMMENDED/

S7 Security Considerations
We have some nice, standard paragraphs to add here...

Bonus if you read this far:
I have to add my disappointment that Timmons and David did not
supply a catchy name for this proposal, as they did with
"Hash and Stuffing".  As long-time BMWG'ers know, I used that
name to liven-up the status on their work item and it evoked
many chuckles and groans from the benchmarking faithful.
The best I can do is suggest "Mind your P's and Queues",
knowing that the authors will offer something infinitely better.