RE: [bmwg] Is the BMWG a proper home for this I-D?ch

[Jim McQuaid <jim.mcquaid at netiq.com>]

You make an excellent point (can it be tested with created traffic or does
it require live network traffic in some form).  That's the "bench" in
benchmark; it can be done with lab equipment.

I would agree with that distinction.  On the other hand, if there is
enthusiasm to do this work within BMWG, tthat may be more important than
legalisms, however sound.

Jim McQuaid

-----Original Message-----
From: Howard C. Berkowitz [mailto:hcb@gettcomm.com]
Sent: Wednesday, April 07, 2004 9:29 AM
To: bmwg@ietf.org
Subject: RE: [bmwg] Is the BMWG a proper home for this I-D?


At 10:56 AM +0000 4/7/04, Jim McQuaid wrote:
><sorry for the blank reply>
>
>At the time the IPPM was "spun off" of the BMWG, the general understanding
>was that BMWG would look at "things" more or less in isolation, although
>that concept is somewhat elastic.  IPPM would look at end to end network
>kinds of performance issues.
>
>It seems clear to me that convergence is a performance characteristic of
>complete networks and primarily an end to end or at least, core element to
>core element characteristic.
>
>My thought would be that this is interesting and useful work but not
>necessarily for BMWG.
>
>Jim McQuaid


I don't completely disagree with you.  I do feel that single-platform 
performance definition and benchmarking is definitely within the 
scope of BMWG, and some of my comments relating to clarifying test 
methods (e.g., black box vs. white box vs. passive analysis) 
definitely can be applied to one box.  BMWG has always looked at 
forwarding plane performance of single boxes, and I really don't 
think we are straying far to look at there control plane performance.

You are correct that IPPM, as well as some of the internet 
scalability issues both in the IETF and IRTF, are probably better 
places for end-to-end convergence.

At least in the case of BGP, however, there is a meaningful 
intermediate level that isn't single box and isn't end-to-end: 
convergence within an AS, and conceivably subsets thereof such as 
route reflector clusters.  Similar points can be made about 
intra-area convergence in OSPF and ISIS.  Where do these fit?

I'm inclined to say BMWG, and let me suggest what might be a very 
crude rule of thumb in saying BMWG or not-BMWG.  Many of the 
Internet-wide experiments have to rely on sampling live data, with 
perhaps some selective probes.  Single-box, or small-system-of-boxes 
measurement, plausibly can be done with an artificial benchmark in a 
testing lab.

Given the name of this group, perhaps, then, the decision if 
something belongs here depends on asking the question early, "can the 
measurement be made with a synthetic benchmark and test equipment," 
or does some of it really have to come from sampling and probing live 
networks?  If the former, it's BMWG eligible. If the latter, it's 
interesting, might even warrant BMWG liaison and/or help in placing 
it elsewhere, but shouldn't stay in BMWG.



_______________________________________________
bmwg mailing list
bmwg@ietf.org
https://www1.ietf.org/mailman/listinfo/bmwg

_______________________________________________
bmwg mailing list
bmwg@ietf.org
https://www1.ietf.org/mailman/listinfo/bmwg