RE: [bmwg] Work on SIP performance issues

["Poretsky, Scott" <scottp at reefpoint.com>]

Hi Jim,

Thank you for pointing this out.  What we are trying to achieve is a
practical algorithm to efficiently result in lab measurement of the
Maximum Session Establishment Rate.  

An incremental step of 100 SPS attempted should be applied until first
failure occurs.  Upon failure at iteration n, the 50% rule should be
applied for iteration n+1 using the difference of SPS between failed
iteration n and previous successful iteration n-1.  If n+1 results in
failure then this is repeated for iteration n+2 using the difference of
iterations n-1 and n+1.  If n+1 is successful then the 50% rule should
be applied for iteration n+2 using the difference of SPS between failed
iteration n and n+1.  This is repeated until it is no longer possible to
apply the 50% rule.

Using your example of 390 SPS, the iterations would look as follow:

Iteration, Attempted Establishment Rate - Result
1, 100 - Successful
2, 200 - Successful
3 (n-1), 300 - Successful
4 (n), 400 - Failure
5 (n+1), 350 - Successful
6 (n+2), 375 - Successful
7, 387 - Successful
8, 393 - Failure
9, 390 - Successful
10, 391 - Failure

Max Session Establishment Rate = 390 SPS

The next rev of the document will be updated to have clarifying wording
to reflect this.

Scott


-----Original Message-----
From: McQuaid, Jim [mailto:Jim.McQuaid at netqos.com] 
Sent: Tuesday, December 18, 2007 5:45 PM
To: Gonzalo Camarillo; Al Morton; Romascanu, Dan (Dan)
Cc: bmwg at ietf.org; Jon Peterson; Mary Barnes
Subject: RE: [bmwg] Work on SIP performance issues

Maybe I don't understand the algorithm or perhaps there are some
unstated assumptions, but it seems to me that part of the general test
procedure is cumbersome and results in many extra test iterations and
may not achieve closure. [ referenced section quoted below]

I did a back of the envelope calculation where the test rate starts at
100 sessions per second.  Assume that the DUT will pass 390 SPS but fail
at higher rates.

Increasing each test cycle by 50% gives us:

100
150 (+50%)
225
338
506 => this one fails

cut that rate by 50%, i.e. in half = 253; next sequence =

253
380
570 => this one fails

cut that rate in half = 285; next sequence =

285
427 => this one fails

cut that rate in half = 214; next sequence =

214
320
481 => this one fails

If I understand this correctly, the algorithm fails to achieve closure
in an efficient manner.  I think the description was meant to implement
a different kind of seeking behavior.  Obviously once a pass thru the
first cycle identifies a low rate (that works) and a high rate (that
fails) all subsequent test runs should be within that range.  Cutting
the high rate by 50% doesn't achieve that.  I think the writer assumed
something that is missing in the actual description.

Jim McQuaid / NetQoS


(referenced procedure - one example ====================================
)
     1.  Configure the DUT in the test topology shown in Figure 1 or
          SUT as shown in Figures 2 or 3.
      2.  Configure Tester for SIP UDP with an Attempted Session Rate =
          100 SPS, Session Duration = 0 sec, Maximum Sessions Attempted
          = 100,000 and media streams per session=0.
      3.  Start Tester to initiate SIP Session establishment with the
          DUT.
      4.  Measure Failed Session Attempts and Total Sessions Established
          at the Tester.
      5.  If a Failed Session Attempt is recorded then reduce the
          Attempted Session Rate configured on the Tester by 50%.
      6.  If no Failed Session Attempt is recorded then increase the
          Attempted Session Rate configured on the Tester by 50%.
      7.  Repeat steps 3 through 6 until the Maximum Session
          Establishment Rate is obtained.


-----Original Message-----
From: Gonzalo Camarillo [mailto:Gonzalo.Camarillo at ericsson.com]
Sent: Tuesday, December 18, 2007 1:31 AM
To: Al Morton; Romascanu, Dan (Dan)
Cc: Jon Peterson; bmwg at ietf.org; Mary Barnes
Subject: [bmwg] Work on SIP performance issues

Hi,

I understand that BMWG is in the process of deciding whether or not to
work on SIP performance and benchmarking issues. The following drafts
were brought to my attention:

http://tools.ietf.org/html/draft-poretsky-sip-bench-term-03
http://tools.ietf.org/html/draft-poretsky-bmwg-sip-bench-meth-02

In July 2006 (IETF 66 in Montreal), as you can see in the MoMs below,
the SIPPING WG showed support to work on SIP performance and
measurements issues. This was seen as an interesting and useful topic by
the SIP community.

http://www3.ietf.org/proceedings/06jul/minutes/sipping.txt

Cheers,

Gonzalo
SIPPING WG co-chair


_______________________________________________
bmwg mailing list
bmwg at ietf.org
https://www1.ietf.org/mailman/listinfo/bmwg

_______________________________________________
bmwg mailing list
bmwg at ietf.org
https://www1.ietf.org/mailman/listinfo/bmwg

_______________________________________________
bmwg mailing list
bmwg at ietf.org
https://www1.ietf.org/mailman/listinfo/bmwg