[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[bmwg] comments on draft-ietf-bmwg-sip-bench-term-00
BMWG,
I have the following comments/suggestions on term-00.
Al
(as a bmwg participant)
----------------------------------------------------------------------
OLD
3.1.2. Signaling Plane
Definition:
The control plane in which SIP messages [RFC3261] are exchanged
between SIP Agents [RFC3261] to establish a connection for media
exchange.
Discussion:
SIP messages are used to establish sessions in several ways:
directly between two User Agents [RFC3261], through a Proxy Server
...
SUGGEST:
3.1.2. Signaling Plane
Definition:
The plane in which SIP messages [RFC3261] are exchanged
^
between SIP Agents [RFC3261] to establish a connection for media
exchange.
Discussion:
| The Signaling Plane is sometimes considered part of the
| Network Control Plane.
SIP messages are used to establish sessions in several ways:
directly between two User Agents [RFC3261], through a Proxy Server
...
-----------------------------------------------
OLD
3.1.3. Media Plane
Definition:
The data plane in which one or more media streams and their
associated media control protocols are exchanged after a media
connection has been created by the exchange of signaling messages
in the Signaling Plane.
Discussion:
Media may also be known as the the "payload" or "bearer channel".
...
SUGGEST:
3.1.3. Media Plane
Definition:
The plane in which one or more media streams and their
^
associated media control protocols are exchanged after a media
connection has been created by the exchange of signaling messages
in the Signaling Plane.
Discussion:
| The Media Plane is sometimes called the Data Plane.
Media may also be known as the the "payload" or "bearer channel".
...
------------------------------------------------------
3.1.4. Associated Media
Definition:
Media that corresponds to an 'm' line in the SDP payload of the
Signaling Plane.
Comment: This doesn't seem to define the term, it's the answer to the
question, "where can I find the specification for the media stream
controlled by this signalling message?"
Suggest:
3.1.4. Associated Media
Definition:
Media (stream) that is the subject of a specific signaling plane
exchange.
Discussion:
The specification of Associated Media may be found in the 'm' line
in the SDP payload on the Signaling Plane.
Any Media Protocol MAY be used.
...
----------------------------------------------------
OLD
3.1.6. Session Attempt
Definition:
A SIP Session for which the Emulated Agent has sent the SIP INVITE
or SUBSCRIBE NOTIFY and has not yet received a message response
from the DUT/SUT.
SUGGEST:
3.1.6. Session Attempt
Definition:
A SIP Session for which the Emulated Agent has sent the SIP INVITE
or SUBSCRIBE NOTIFY and expects to receive a message response
^^^^^^^^^^^^^^^^^^^
from the DUT/SUT.
----------------------------------------------------------
OLD
3.1.7. Established Session
Definition:
A SIP session for which the Emulated Agent acting as the UE/UA has
received a 200OK message from the DUT/SUT.
SUGGEST:
move this section after 3.1.8 and 3.1.9, so that IS and NS are defined
before you read the "Established Session" Definition.
THEN, move the IS and NS Established Session Discussion text
to this Definition, as below:
3.1.7. Established Session
Definition:
A SIP session for which the Emulated Agent acting as the UE/UA has
| sent an INVITE or a SUBSCRIBE NOTIFY, and
received a 200OK message from the DUT/SUT.
| An IS is successfully established if the
| following two conditions are met:
| 1. Sess.sig is established by the end of Establishment Threshold
| Time (c.f. Section 3.3.3), and
| 2. The media session described in the SDP body of the signaling
| message is present.
| An NS is successfully established if the Session Attempt via a
| non-INVITE request results in the EA receiving a 2xx or 3xx reply
| from the DUT/SUT before the expiration of the Establishment
| Threshold timer (c.f., Section 3.3.3).
(the 2 paragraphs above are deleted from Discussion in IS and NS sections)
-------------------------------------------------------------
OLD
3.1.10. Session Attempt Failure
Definition:
A session attempt that does not result in an Established Session.
Discussion:
The session attempt failure may be indicated by the following
observations at the Emulated Agent:
1. Receipt of a SIP 4xx, 5xx, or 6xx class response to a Session
Attempt.
2. The lack of any received SIP response to a Session Attempt.
***
QUESTION: How is an attempt that results in a 180 classified?
----------------------------------------------------
OLD
3.1.11. Standing Sessions
Definition:
The number of Sessions currently established on the DUT/SUT at any
instant.
Discussion:
The number of Standing Sessions is influenced by the Session
Duration and the Session Attempt Rate. Benchmarks MUST be
reported with the maximum and average Standing Sessions for the
DUT/SUT. In order to determine the maximum and average Standing
Sessions on the DUT/SUT for the duration of the test it is
necessary to make periodic measurements of the number of Standing
Sessions on the DUT/SUT. The recommended value for the
measurement period is 1 second.
QUESTIONS:
A counter of the number of sessions can be read in an instant,
so why is there a recommended measurement period?
Is this how frequently the sessions counter should be read?
On the other hand, the Discussion may be implying that there are
actually three terms being defined: an instantaneous count of
Standing Sessions, the Average Standing Sessions (over a time interval
with many readings of Standing Sessions), and the Maximum Standing Sessions
(over a time interval with many readings of Standing Sessions).
To clarify Standing Sessions, SUGGEST:
Definition:
The count of Sessions established on the DUT/SUT at any instant.
New terms and defs needed for Average and Max.
-------------------------------------------------------------
3.3.1. Session Attempt Rate
Definition:
Configuration of the Emulated Agent for the number of sessions
that the Emulated Agent attempts to establish with the DUT/SUT
over a specified time interval.
Discussion:
The Session Attempt Rate can cause variation in performance
benchmark measurements. Since this is the number of sessions
configured on the Tester, some sessions may not be successfully
established on the DUT. A session may be either an IS or an NS.
Measurement Units:
Session attempts per second
Comment:
I see some wiggle-room here for interpretation that might have
a significant effect on the results. Suppose one tester generates
all Session Attempts with equal spacing and linearly maintains the
attempt rate even when viewed at sub-second intervals.
Another tester bursts all Session Attempts at the line rate,
and then waits for the beginning of the next second to produce
another Attempt burst.
I imagine that the DUT performance may differ under these circumstances,
and suggest to tighten this wording if all agree this is an issue.
------------------------------------------------
OLD
3.3.3. Establishment Threshold Time
SUGGEST
3.3.3. Establishment Failure Time Threshold
-----------------------------------------------------
OLD
3.3.6. Media Session Hold Time
Definition:
Configuration of the Emulated Agent for the amount of time
measured at the Emulated Agent for the Associated Media for an
Established Session of type IS media to flow between the Emulated
Agent and the DUT/SUT.
Comment: That's a tough definition to read. Let me try...
SUGGEST
Definition:
The time duration of Associated Media (for a specific IS),
flowing from an EA to the DUT/SUT, as measured at the EA.
-------------------------------------------------------
I also have some comments to type up on section 3.4, Benchmarks.
but they will have to wait...