RE: [bmwg] RE: [Sipping] Draft: SIP Performance Metrics

[sporetsky at reefpoint.com]

The authors of this work have been in regular communication from Erich M.
Nahum of SPEC.  Here is a snip from one of the emails that well summmarizes
the relationship:

> -----Original Message-----
> From: nahum at watson.ibm.com [mailto:nahum at watson.ibm.com]
> Sent: Friday, May 26, 2006 2:51 PM
> To: Vijay K. Gurbani
> Cc: Malas, Daryl; sipping at ietf.org; acmorton at att.com; bmwg at ietf.org
> Subject: [bmwg] SPEC SIP Benchmarking Effort
>
> [snip]
>
> I have been organizing the SPEC SIP benchmarking effort, and would
> love to get participation from IETF folks.  It seems like the two
> activities could be very complementary.  
> 
> Based on what I've seen, it seems like the IETF BMWG and Daryl Malas' 
> draft posted on SIPPING a few days ago are more oriented towards defining 
> the proper methods and metrics to measure (e.g., call success rate,
defects 
> per million, etc.).  
> 
> In contrast, SPEC wants to develop and distribute common code for 
> benchmarking, as is done with SPECWeb and SPECJAppServer.  That code can 
> and should use the standardized peformance definitions agreed to by 
> SIPPING and/or BMWG.  

Scott

-----Original Message-----
From: Romascanu, Dan (Dan) [mailto:dromasca at avaya.com]
Sent: Sunday, June 25, 2006 6:00 AM
To: Burger, Eric; Malas, Daryl
Cc: sipping at ietf.org; bmwg at ietf.org
Subject: RE: [bmwg] RE: [Sipping] Draft: SIP Performance Metrics


Eric,

>From the bmwg charter: 

> The major goal of the Benchmarking Methodology Working Group is to
make
a series of recommendations concerning the measurement of the
performance characteristics of various internetworking technologies;

To the extend that we consider SIP an internetworking technology, I
believe that the scope of the 'SIP Performance Metrics' draft is within
the scope of what bmwg is doing for a while, quite successfully, some
say. 

On a more 'philosophical plan', there is nothing that says that the IETF
work must strictly deal with defining the bits in the Internet Protocols
- see http://www.ietf.org/internet-drafts/draft-hoffman-taobis-08.txt.
And in any case, measuring how a protocol or a device implementing a
protocol behaves can be considered also 'DIRECTLY related to protocol
development'.

Dan

PS - This does not mean that I do not have my own questions about the
I-D :-)

PPS - If we end by deciding that we do this work in the IETF we should
certainly coordinate with other organizations developing standards,
performance measurement and test methodologies in this space. 




 
 

> -----Original Message-----
> From: Burger, Eric [mailto:eburger at cantata.com] 
> Sent: Friday, June 23, 2006 11:18 PM
> To: Malas, Daryl
> Cc: sipping at ietf.org; bmwg at ietf.org
> Subject: [bmwg] RE: [Sipping] Draft: SIP Performance Metrics
> 
> More to the point, is this work appropriate for the IETF at 
> all?  I would offer that if we needed additions to SIP to 
> instrument it for monitoring, then sure.  However, I think 
> this would much better be done in another venue such as the 
> SIP Forum or IMS Forum or even SPEC.  If it turns out that 
> there need to be some SIP changes, then we could bring that 
> to the IETF.
> 
> We have way too much on our plate as it is.
> 
> 
> 
> Disclaimer: I'm on the boards of both the SIP Forum and IMS Forum.
> However, this isn't make-work for those groups.  Rather, this 
> is a plea to NOT make more work in the IETF that is not 
> DIRECTLY related to protocol development.
> 
> -----Original Message-----
> From: Henning Schulzrinne [mailto:hgs at cs.columbia.edu]
> Sent: Wednesday, May 24, 2006 2:44 PM
> To: Vijay K. Gurbani
> Cc: Malas, Daryl; sipping at ietf.org; acmorton at att.com; bmwg at ietf.org
> Subject: Re: [Sipping] Draft: SIP Performance Metrics
> 
> A working group has just been formed (this week) in the SPEC 
> consortium.
> 
> Vijay K. Gurbani wrote:
> > Daryl: There is work starting in the BMWG along SIP performance 
> > metrics, being worked by Scott Poretsky and me.  At the 
> very least, we 
> > should co-ordinate your draft with the thoughts there, and then can 
> > figure out with the help of chairs and ADs as to where the 
> work should 
> > proceed.
> > 
> > I have also heard about some performance metrics work in TISPAN and 
> > SPEC, but I do not have any more details about these.
> > 
> 
> _______________________________________________
> Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
> This list is for NEW development of the application of SIP 
> Use sip-implementors at cs.columbia.edu for questions on current 
> sip Use sip at ietf.org for new developments of core SIP
> 
> _______________________________________________
> bmwg mailing list
> bmwg at ietf.org
> https://www1.ietf.org/mailman/listinfo/bmwg
> 

_______________________________________________
bmwg mailing list
bmwg at ietf.org
https://www1.ietf.org/mailman/listinfo/bmwg

_______________________________________________
Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
This list is for NEW development of the application of SIP
Use sip-implementors at cs.columbia.edu for questions on current sip
Use sip at ietf.org for new developments of core SIP