idnits 2.17.1 draft-ietf-bmwg-sip-bench-term-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** You're using the IETF Trust Provisions' Section 6.b License Notice from 12 Sep 2009 rather than the newer Notice from 28 Dec 2009. (See https://trustee.ietf.org/license-info/) Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 12, 2010) is 5036 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Downref: Normative reference to an Informational RFC: RFC 2544 == Outdated reference: A later version (-12) exists of draft-ietf-bmwg-sip-bench-meth-02 ** Downref: Normative reference to an Informational draft: draft-ietf-bmwg-sip-bench-meth (ref. 'I-D.ietf-bmwg-sip-bench-meth') Summary: 3 errors (**), 0 flaws (~~), 3 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Benchmarking Methodology Working S. Poretsky 3 Group Allot Communications 4 Internet-Draft V. Gurbani 5 Expires: January 13, 2011 Bell Laboratories, Alcatel-Lucent 6 C. Davids 7 Illinois Institute of Technology 8 July 12, 2010 10 Terminology for Benchmarking Session Initiation Protocol (SIP) 11 Networking Devices 12 draft-ietf-bmwg-sip-bench-term-02 14 Abstract 16 This document provides a terminology for benchmarking SIP performance 17 in networking devices. Terms are included for test components, test 18 setup parameters, and performance benchmark metrics for black-box 19 benchmarking of SIP networking devices. The performance benchmark 20 metrics are obtained for the SIP control plane and media plane. The 21 terms are intended for use in a companion methodology document for 22 complete performance characterization of a device in a variety of 23 conditions making it possible to compare performance of different 24 devices. It is critical to provide test setup parameters and a 25 methodology document for SIP performance benchmarking because SIP 26 allows a wide range of configuration and operational conditions that 27 can influence performance benchmark measurements. It is necessary to 28 have terminology and methodology standards to ensure that reported 29 benchmarks have consistent definition and were obtained following the 30 same procedures. Benchmarks can be applied to compare performance of 31 a variety of SIP networking devices. 33 Status of this Memo 35 This Internet-Draft is submitted to IETF in full conformance with the 36 provisions of BCP 78 and BCP 79. 38 Internet-Drafts are working documents of the Internet Engineering 39 Task Force (IETF), its areas, and its working groups. Note that 40 other groups may also distribute working documents as Internet- 41 Drafts. 43 Internet-Drafts are draft documents valid for a maximum of six months 44 and may be updated, replaced, or obsoleted by other documents at any 45 time. It is inappropriate to use Internet-Drafts as reference 46 material or to cite them other than as "work in progress." 48 The list of current Internet-Drafts can be accessed at 49 http://www.ietf.org/ietf/1id-abstracts.txt. 51 The list of Internet-Draft Shadow Directories can be accessed at 52 http://www.ietf.org/shadow.html. 54 This Internet-Draft will expire on January 13, 2011. 56 Copyright Notice 58 Copyright (c) 2010 IETF Trust and the persons identified as the 59 document authors. All rights reserved. 61 This document is subject to BCP 78 and the IETF Trust's Legal 62 Provisions Relating to IETF Documents 63 (http://trustee.ietf.org/license-info) in effect on the date of 64 publication of this document. Please review these documents 65 carefully, as they describe your rights and restrictions with respect 66 to this document. Code Components extracted from this document must 67 include Simplified BSD License text as described in Section 4.e of 68 the Trust Legal Provisions and are provided without warranty as 69 described in the BSD License. 71 Table of Contents 73 1. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 74 2. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 6 75 2.1. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . 7 76 2.2. Benchmarking Models . . . . . . . . . . . . . . . . . . . 8 77 3. Term Definitions . . . . . . . . . . . . . . . . . . . . . . . 11 78 3.1. Protocol Components . . . . . . . . . . . . . . . . . . . 11 79 3.1.1. Session . . . . . . . . . . . . . . . . . . . . . . . 11 80 3.1.2. Signaling Plane . . . . . . . . . . . . . . . . . . . 14 81 3.1.3. Media Plane . . . . . . . . . . . . . . . . . . . . . 15 82 3.1.4. Associated Media . . . . . . . . . . . . . . . . . . . 15 83 3.1.5. Overload . . . . . . . . . . . . . . . . . . . . . . . 16 84 3.1.6. Session Attempt . . . . . . . . . . . . . . . . . . . 17 85 3.1.7. Established Session . . . . . . . . . . . . . . . . . 17 86 3.1.8. Invite-initiated Session (IS) . . . . . . . . . . . . 18 87 3.1.9. Non-INVITE-initiated Session (NS) . . . . . . . . . . 18 88 3.1.10. Session Attempt Failure . . . . . . . . . . . . . . . 19 89 3.1.11. Standing Sessions Count . . . . . . . . . . . . . . . 19 90 3.2. Test Components . . . . . . . . . . . . . . . . . . . . . 20 91 3.2.1. Emulated Agent . . . . . . . . . . . . . . . . . . . . 20 92 3.2.2. Signaling Server . . . . . . . . . . . . . . . . . . . 20 93 3.2.3. SIP-Aware Stateful Firewall . . . . . . . . . . . . . 21 94 3.2.4. SIP Transport Protocol . . . . . . . . . . . . . . . . 21 95 3.3. Test Setup Parameters . . . . . . . . . . . . . . . . . . 22 96 3.3.1. Session Attempt Rate . . . . . . . . . . . . . . . . . 22 97 3.3.2. IS Media Attempt Rate . . . . . . . . . . . . . . . . 22 98 3.3.3. Establishment Threshold Time . . . . . . . . . . . . . 23 99 3.3.4. Session Duration . . . . . . . . . . . . . . . . . . . 24 100 3.3.5. Media Packet Size . . . . . . . . . . . . . . . . . . 24 101 3.3.6. Media Offered Load . . . . . . . . . . . . . . . . . . 25 102 3.3.7. Media Session Hold Time . . . . . . . . . . . . . . . 25 103 3.3.8. Loop Detection Option . . . . . . . . . . . . . . . . 26 104 3.3.9. Forking Option . . . . . . . . . . . . . . . . . . . . 26 105 3.4. Benchmarks . . . . . . . . . . . . . . . . . . . . . . . . 27 106 3.4.1. Registration Rate . . . . . . . . . . . . . . . . . . 27 107 3.4.2. Session Establishment Rate . . . . . . . . . . . . . . 28 108 3.4.3. Session Capacity . . . . . . . . . . . . . . . . . . . 29 109 3.4.4. Session Overload Capacity . . . . . . . . . . . . . . 30 110 3.4.5. Session Establishment Performance . . . . . . . . . . 30 111 3.4.6. Session Attempt Delay . . . . . . . . . . . . . . . . 31 112 3.4.7. IM Rate . . . . . . . . . . . . . . . . . . . . . . . 31 113 4. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 32 114 5. Security Considerations . . . . . . . . . . . . . . . . . . . 32 115 6. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 33 116 7. References . . . . . . . . . . . . . . . . . . . . . . . . . . 33 117 7.1. Normative References . . . . . . . . . . . . . . . . . . . 33 118 7.2. Informational References . . . . . . . . . . . . . . . . . 33 120 Appendix A. White Box Benchmarking Terminology . . . . . . . . . 34 121 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 34 123 1. Terminology 125 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 126 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 127 document are to be interpreted as described in BCP 14, RFC2119 128 [RFC2119]. RFC 2119 defines the use of these key words to help make 129 the intent of standards track documents as clear as possible. While 130 this document uses these keywords, this document is not a standards 131 track document. The term Throughput is defined in RFC2544 [RFC2544]. 133 For the sake of clarity and continuity, this document adopts the 134 template for definitions set out in Section 2 of RFC 1242 [RFC1242]. 135 Definitions are indexed and grouped together for ease of reference. 137 This document uses existing terminology defined in other BMWG work. 138 Examples include, but are not limited to: 140 Device under test (DUT) (c.f., Section 3.1.1 RFC 2285 [RFC2285]). 141 System under test (SUT) (c.f., Section 3.1.2, RFC 2285 [RFC2285]). 143 Many commonly used SIP terms in this document are defined in RFC 3261 144 [RFC3261]. For convenience the most important of these are 145 reproduced below. Use of these terms in this document is consistent 146 with their corresponding definition in [RFC3261]. 147 o Call Stateful: A proxy is call stateful if it retains state for a 148 dialog from the initiating INVITE to the terminating BYE request. 149 A call stateful proxy is always transaction stateful, but the 150 converse is not necessarily true. 151 o Stateful Proxy: A logical entity that maintains the client and 152 server transaction state machines defined by this specification 153 during the processing of a request, also known as a transaction 154 stateful proxy. The behavior of a stateful proxy is further 155 defined in Section 16. A (transaction) stateful proxy is not the 156 same as a call stateful proxy. 157 o Stateless Proxy: A logical entity that does not maintain the 158 client or server transaction state machines defined in this 159 specification when it processes requests. A stateless proxy 160 forwards every request it receives downstream and every response 161 it receives upstream. 162 o Back-to-back User Agent: A back-to-back user agent (B2BUA) is a 163 logical entity that receives a request and processes it as a user 164 agent server (UAS). In order to determine how the request should 165 be answered, it acts as a user agent client (UAC) and generates 166 requests. Unlike a proxy server, it maintains dialog state and 167 must participate in all requests sent on the dialogs it has 168 established. Since it is a concatenation of a UAC and a UAS, no 169 explicit definitions are needed for its behavior. 171 o Loop: A request that arrives at a proxy, is forwarded, and later 172 arrives back at the same proxy. When it arrives the second time, 173 its Request-URI is identical to the first time, and other header 174 fields that affect proxy operation are unchanged, so that the 175 proxy will make the same processing decision on the request it 176 made the first time. Looped requests are errors, and the 177 procedures for detecting them and handling them are described by 178 the protocol. 180 2. Introduction 182 Service Providers are now planning Voice Over IP (VoIP) and 183 Multimedia network deployments using the IETF developed Session 184 Initiation Protocol (SIP) [RFC3261]. SIP is a signaling protocol 185 originally intended to be used for the dynamic establishment, 186 disconnection and modification of streams of media between end users. 187 As it has evolved it has been adopted for use in a growing number of 188 applications and features. Many of these result in the creation of a 189 media stream, but some do not. Instead, they create other services 190 tailored to the end-users' immediate needs or preferences. The set 191 of benchmarking terms provided in this document is intended for use 192 with any SIP-enabled device performing SIP functions in the interior 193 of the network. The performance of end-user devices is outside the 194 scope of this document. 196 VoIP with SIP has led to the development of new networking devices 197 including SIP Server, Session Border Controllers (SBC), Back-to-back 198 user agents (B2BUA) and SIP-Aware Stateful Firewall. The mix of 199 voice and IP functions in these various devices has produced 200 inconsistencies in vendor reported performance metrics and has caused 201 confusion in the service provider community. SIP allows a wide range 202 of configuration and operational conditions that can influence 203 performance benchmark measurements. When the device under test 204 terminates or relays both media and signaling, for example, it is 205 important to be able to correlate a signaling measurement with the 206 media plane measurements to determine the system performance. As 207 devices and their functions proliferate, the need to have a 208 consistent set of metrics to compare their performance becomes 209 increasingly urgent. This document and its companion methodology 210 document [I-D.ietf-bmwg-sip-bench-meth] provide a set of black-box 211 benchmarks for describing and comparing the performance of devices 212 that incorporate the SIP User Agent Client and Server functions and 213 that operate in the network's core. 215 The definition of SIP performance benchmarks necessarily includes 216 definitions of Test Setup Parameters and a test methodology. These 217 enable the Tester to perform benchmarking tests on different devices 218 and to achieve comparable and repeatable results. This document 219 provides a common set of well-defined terms for Test Components, Test 220 Setup Parameters, and Benchmarks. All the benchmarks defined are 221 black-box measurements of the SIP Control (Signaling) plane. The 222 Test Setup Parameters and Benchmarks defined in this document are 223 intended for use with the companion Methodology document. Benchmarks 224 of internal DUT characteristics (also known as white-box benchmarks) 225 such as Session Attempt Arrival Rate, which is measured at the DUT, 226 are described in Appendix A to allow additional characterization of 227 DUT behavior with different distribution models. 229 2.1. Scope 231 The scope of this work item is summarized as follows: 232 o This terminology document describes SIP signaling (control- plane) 233 performance benchmarks for black-box measurements of SIP 234 networking devices. Stress and debug scenarios are not addressed 235 in this work item. 236 o The DUT must be an RFC 3261 capable network equipment. This may 237 be a Registrar, Redirect Server, Stateless Proxy or Stateful 238 Proxy. A DUT MAY also include a B2BUA, SBC functionality (this is 239 referred to as the "Signaling Server".) The DUT MAY be a multi- 240 port SIP-to-switched network gateway implemented as a SIP UAC or 241 UAS. 242 o The DUT MAY have an internal SIP Application Level Gateway (ALG), 243 firewall, and/or a Network Address Translator (NAT). This is 244 referred to as the "SIP Aware Stateful Firewall." 245 o The DUT or SUT MUST NOT be end user equipment, such as personal 246 digital assistant, a computer-based client, or a user terminal. 247 o The Tester acts as multiple "Emulated Agents" that initiate (or 248 respond to) SIP messages as session endpoints and source (or 249 receive) associated media for established connections. 250 o Control Signaling in presence of Media 251 * The media performance is not benchmarked in this work item. 252 * It is RECOMMENDED that control plane benchmarks are performed 253 with media present, but this is optional. 254 * The SIP INVITE requests MUST include the SDP body. 255 * The type of DUT dictates whether the associated media streams 256 traverse the DUT or SUT. Both scenarios are within the scope 257 of this work item. 258 * SIP is frequently used to create media streams; the control 259 plane and media plane are treated as orthogonal to each other 260 in this document. While many devices support the creation of 261 media streams, benchmarks that measure the performance of these 262 streams are outside the scope of this document and its 263 companion methodology document [I-D.ietf-bmwg-sip-bench-meth]. 264 Tests may be performed with or without the creation of media 265 streams. The presence or absence of media streams MUST be 266 noted as a condition of the test as the performance of SIP 267 devices may vary accordingly. Even if the media is used during 268 benchmarking, only the SIP performance will be benchmarked, not 269 the media performance or quality. 270 o Both INVITE and non-INVITE scenarios (such as Instant Messages or 271 IM) are addressed in this document. However, benchmarking SIP 272 presence is not a part of this work item. 273 o Different transport mechanisms -- such as UDP, TCP, SCTP, or TLS 274 -- may be used; however, the specific transport mechanism MUST be 275 noted as a condition of the test as the performance of SIP devices 276 may vary accordingly. 277 o Looping and forking options are also considered since they impact 278 processing at SIP proxies. 279 o REGISTER and INVITE requests may be challenged or remain 280 unchallenged for authentication purpose as this may impact the 281 performance benchmarks. Any observable performance degradation 282 due to authentication is of interest to the SIP community. 283 Whether or not the REGISTER and INVITE requests are challenged is 284 a condition of test and will be recorded and reported. 285 o Re-INVITE requests are not considered in scope of this work item. 286 o Only session establishment is considered for the performance 287 benchmarks. Session disconnect is not considered in the scope of 288 this work item. 289 o SIP Overload [I-D.ietf-sipping-overload-reqs] is within the scope 290 of this work item. We test to failure and then can continue to 291 observe and record the behavior of the system after failures are 292 recorded. The cause of failure is not within the scope of this 293 work. We note the failure and may continue to test until a 294 different failure or condition is encountered. Considerations on 295 how to handle overload are deferred to work progressing in the 296 SIPPING working group [I-D.ietf-sipping-overload-design]. Vendors 297 are, of course, free to implement their specific overload control 298 behavior as the expected test outcome if it is different from the 299 IETF recommendations. However, such behavior MUST be documented 300 and interpreted appropriately across multiple vendor 301 implementations. This will make it more meaningful to compare the 302 performance of different SIP overload implementations. 303 o IMS-specific scenarios are not considered, but test cases can be 304 applied with 3GPP-specific SIP signaling and the P-CSCF as a DUT. 306 2.2. Benchmarking Models 308 This section shows the five models to be used when benchmarking SIP 309 performance of a networking device. Figure 1 shows a configuration 310 in which the Tester acting as the Emulated agents is in loopback 311 testing itself for the purpose of baselining its performance. 313 +--------+ Signaling request +--------+ 314 | +----------------------------->| | 315 | Tester | | Tester | 316 | EA | Signaling response | EA | 317 | |<-----------------------------+ | 318 +--------+ +--------+ 320 Figure 1: Test topology 1 - Emulated agent (EA) baseline performance 321 measurement 323 Figure 2 shows the basic configuration for benchmarking the 324 Registration of the DUT/SUT. 326 +--------+ Registration +--------+ 327 | +----------------------------->| | 328 | Tester | | DUT | 329 | EA | Response | | 330 | |<-----------------------------+ | 331 +--------+ +--------+ 333 Figure 2: Test topology 2 - Emulated agent (EA) registration to DUT/ 334 SUT 336 Figure 3 shows that the Tester acts as the initiating and responding 337 Emulated Agents as the DUT/SUT forwards Session Attempts. 339 +--------+ Session +--------+ Session +--------+ 340 | | Attempt | | Attempt | | 341 | |<------------+ |<------------+ | 342 | | | | | | 343 | | Response | | Response | | 344 | Tester +------------>| DUT +------------>| Tester | 345 | (EA) | | | | (EA) | 346 | | | | | | 347 +--------+ +--------+ +--------+ 349 Figure 3: Test topology 3 - DUT/SUT performance benchmark for session 350 establishment without media 352 Figure 4 is used when performing those same benchmarks with 353 Associated Media traversing the DUT/SUT. 355 +--------+ Session +--------+ Session +--------+ 356 | | Attempt | | Attempt | | 357 | |<------------+ |<------------+ | 358 | | | | | | 359 | | Response | | Response | | 360 | Tester +------------>| DUT +------------>| Tester | 361 | | | | | (EA) | 362 | | Media | | Media | | 363 | |<===========>| |<===========>| | 364 +--------+ +--------+ +--------+ 366 Figure 4: Test topology 4 - DUT/SUT performance benchmark for session 367 establishment with media traversing the DUT 369 Figure 5 is to be used when performing those same benchmarks with 370 Associated Media, but the media does not traverse the DUT/SUT. 371 Again, the benchmarking of the media is not within the scope of this 372 work item. The SIP control signaling is benchmarked in the presence 373 of Associated Media to determine if the SDP body of the signaling and 374 the handling of media impacts the performance of the DUT/SUT. 376 +--------+ Session +--------+ Session +--------+ 377 | | Attempt | | Attempt | | 378 | |<------------+ |<------------+ | 379 | | | | | | 380 | | Response | | Response | | 381 | Tester +------------>| DUT +------------>| Tester | 382 | | | | | (EA) | 383 | | | | | | 384 +--------+ +--------+ +--------+ 385 /|\ /|\ 386 | Media | 387 +=============================================+ 389 Figure 5: Test topology 5 - DUT/SUT performance benchmark for session 390 establishment with media external to the DUT 392 Figure 6 illustrates the SIP signaling for an Established Session. 393 The Tester acts as the Emulated Agent(s) and initiates a Session 394 Attempt with the DUT/SUT. When the Emulated Agent (EA) receives a 395 200 OK from the DUT/SUT that session is considered to be an 396 Established Session. The illustration indicates three states of the 397 session bring created by the EA - Attempting, Established, and 398 Disconnecting. Sessions can be one of two type: Invite-Initiated 399 Session (IS) or Non-Invite Initiated Session (NS). Failure for the 400 DUT/SUT to successfully respond within the Establishment Threshold 401 Time is considered a Session Attempt Failure. SIP Invite messages 402 MUST include the SDP body to specify the Associated Media. Use of 403 Associated Media, to be sourced from the EA, is optional. When 404 Associated Media is used, it may traverse the DUT/SUT depending upon 405 the type of DUT/SUT. The Associated Media is shown in Figure 6 as 406 "Media" connected to media ports M1 and M2 on the EA. After the EA 407 sends a BYE, the session disconnects. Performance test cases for 408 session disconnects are not considered in this work item (the BYE 409 request is shown for completeness.) 411 EA DUT/SUT M1 M2 412 | | | | 413 | INVITE | | | 414 --------+--------------->| | | 415 | | | | 416 Attempting | | | 417 | 200 OK | | | 418 --------|<-------------- | | | 419 | | | | 420 | | | | 421 | | | Media | 422 Established | |<=====>| 423 | | | | 424 | BYE | | | 425 --------+--------------> | | | 426 | | | | 427 Disconnecting | | | 428 | 200 OK | | | 429 --------|<-------------- | | | 430 | | | | 432 Figure 6: Basic SIP test topology 434 3. Term Definitions 436 3.1. Protocol Components 438 3.1.1. Session 440 Definition: 442 The combination of signaling and media messages and processes that 443 enable two or more participants to communicate. 445 Discussion: 446 SIP messages in the signaling plane can be used to create and 447 manage applications for one or more end users. SIP is often used 448 to create and manage media streams in support of applications. A 449 session always has a signaling component and may have a media 450 component. Therefore, a Session may be defined as signaling only 451 or a combination of signaling and media (c.f. Associated Media, 452 see Section 3.1.4). SIP includes definitions of a Call-ID, a 453 dialogue and a transaction that support this application. A 454 growing number of usages and applications do not require the 455 creation of associated media. The first such usage was the 456 REGISTER. Applications that use the MESSAGE and SUBSCRIBE/NOTIFY 457 methods also do not require SIP to manage media streams. The 458 terminology Invite-initiated Session (IS) and Non-invite initiated 459 Session (NS) are used to distinguish between these different 460 usages. 462 A Session in the context of this document, is considered to be a 463 vector with three components: 465 1. A component in the signaling plane (SIP messages), sess.sig; 466 2. A media component in the media plane (RTP and SRTP streams for 467 example), sess.med (which may be null); 468 3. A control component in the media plane (RTCP messages for 469 example), sess.medc (which may be null). 471 An IS is expected to have non-null sess.sig and sess.med 472 components. The use of control protocols in the media component 473 is media dependent, thus the expected presence or absence of 474 sess.medc is media dependent and test-case dependent. An NS is 475 expected to have a non-null sess.sig component, but null sess.med 476 and sess.medc components. 478 Packets in the Signaling Plane and Media Plane will be handled by 479 different processes within the DUT. They will take different 480 paths within a SUT. These different processes and paths may 481 produce variations in performance. The terminology and benchmarks 482 defined in this document and the methodology for their use are 483 designed to enable us to compare performance of the DUT/SUT with 484 reference to the type of SIP-supported application it is handling. 486 Note that one or more sessions can simultaneously exist between 487 any participants. This can be the case, for example, when the EA 488 sets up both an IM and a voice call through the DUT/SUT. These 489 sessions are represented as an array session[x]. 491 Sessions will be represented as a vector array with three 492 components, as follows: 493 session-> 494 session[x].sig, the signaling component 495 session[x].medc, the media control component (e.g. RTCP) 496 session[x].med[y], an array of associated media streams (e.g. 497 RTP, SRTP, RTSP, MSRP). This media component may consist of zero 498 or more media streams. 499 Figure 7 models the vectors of the session. 501 Measurement Units: 502 N/A. 504 Issues: 505 None. 507 See Also: 508 Media Plane 509 Signaling Plane 510 Associated Media 511 Invite-initiated Session (IS) 512 Non-invite-initiated Session (NS) 513 |\ 514 | 515 | \ 516 sess.sig| 517 | \ 518 | 519 | \ 520 | o 521 | / 522 | / | 523 | / 524 | / | 525 | / 526 | / | 527 | / 528 | / | sess.medc 529 |/_____________________ 530 / / 531 / | 532 / / 533 sess.med / | 534 /_ _ _ _ _ _ _ _/ 535 / 536 / 537 / 538 / 540 Figure 7: Application or session components 542 3.1.2. Signaling Plane 544 Definition: 545 The control plane in which SIP messages [RFC3261] are exchanged 546 between SIP Agents [RFC3261] to establish a connection for media 547 exchange. 549 Discussion: 550 SIP messages are used to establish sessions in several ways: 551 directly between two User Agents [RFC3261], through a Proxy Server 552 [RFC3261], or through a series of Proxy Servers. The Signaling 553 Plane MUST include the Session Description Protocol (SDP). 554 The Signaling Plane for a single Session is represented by 555 session.sig. 557 Measurement Units: 558 N/A. 560 Issues: 561 None. 563 See Also: 564 Media Plane 565 Emulated Agents 567 3.1.3. Media Plane 569 Definition: 570 The data plane in which one or more media streams and their 571 associated media control protocols are exchanged after a media 572 connection has been created by the exchange of signaling messages 573 in the Signaling Plane. 575 Discussion: 576 Media may also be known as the "bearer channel". The Media Plane 577 MUST include the media control protocol, if one is used, and the 578 media stream(s). Examples of media are audio, video, whiteboard, 579 and instant messaging service. The media stream is described in 580 the SDP of the Signaling Plane. 581 The media for a single Session is represented by session.med. The 582 media control protocol is represented by session.medc. 584 Measurement Units: 585 N/A. 587 Issues: 588 None. 590 See Also: 591 Signaling Plane 593 3.1.4. Associated Media 595 Definition: 596 Media that corresponds to an 'm' line in the SDP payload of the 597 Signaling Plane. 599 Discussion: 601 Any media protocol MAY be used. 602 For any session's signaling component, represented as session.sig, 603 there may be one or multiple associated media streams which are 604 represented be a vector array session.med[y], which is referred to 605 as the Associated Media. 607 Measurement Units: 608 N/A. 610 Issues: 611 None. 613 3.1.5. Overload 615 Definition: 616 Overload is defined as the state where a SIP server does not have 617 sufficient resources to process all incoming SIP messages 618 [I-D.ietf-sipping-overload-reqs]. The distinction between an 619 overload condition and other failure scenarios is outside the 620 scope of this document which is blackbox testing. 622 Discussion: 623 Under overload conditions, all or a percentage of Session Attempts 624 will fail due to lack of resources. SIP server resources may 625 include CPU processing capacity, network bandwidth, input/output 626 queues, or disk resources. Any combination of resources may be 627 fully utilized when a SIP server (the DUT/SUT) is in the overload 628 condition. For proxy-only type of devices, overload issues will 629 be dominated by the number of signaling messages they can handle 630 in a unit time before their throughput starts to drop. 631 For UA-type of network devices (e.g., gateways), overload must 632 necessarily include both the signaling traffic and media streams. 633 It is expected that the amount of signaling that a UA can handle 634 is inversely proportional to the amount of media streams currently 635 handled by that UA. 637 Measurement Units: 638 N/A. 640 Issues: 641 The issue of overload in SIP networks is currently a topic of 642 discussion in the SIPPING WG. The normal response to an overload 643 stimulus -- sending a 503 response -- is considered inadequate and 644 new response codes and behaviors may be specified in the future. 645 From the perspective of this document, all these responses will be 646 considered to be failures. There is thus no dependency between 647 this document and the ongoing work on the treatment of overload 648 failure. 650 3.1.6. Session Attempt 652 Definition: 653 A SIP Session for which the Emulated Agent has sent the SIP INVITE 654 or SUBSCRIBE NOTIFY and has not yet received a message response 655 from the DUT/SUT. 657 Discussion: 658 The attempted session may be an IS or an NS. The Session Attempt 659 includes SIP INVITEs and SUBSCRIBE/NOTIFY messages. It also 660 includes all INVITEs that are rejected for lack of authentication 661 information. 663 Measurement Units: 664 N/A. 666 Issues: 667 None. 669 See Also: 670 Session 671 Session Attempt Rate 672 Invite-initiated Session 673 Non-Invite initiated Session 675 3.1.7. Established Session 677 Definition: 678 A SIP session for which the Emulated Agent acting as the UE/UA has 679 received a 200 OK message from the DUT/SUT. 681 Discussion: 682 An Established Session MAY be type INVITE-Session (IS) or Non- 683 INVITE Session (NS). 685 Measurement Units: 686 N/A. 688 Issues: 689 None. 691 See Also: 693 Invite-initiated Session 694 Session Attempting State 695 Session Disconnecting State 697 3.1.8. Invite-initiated Session (IS) 699 Definition: 700 A Session that is created by an exchange of messages in the 701 Signaling Plane, the first of which is a SIP INVITE request. 703 Discussion: 704 An IS is identified by the Call-ID, To-tag, and From-tag of the 705 SIP message that establishes the session. These three fields are 706 used to identify a SIP Dialog (RFC3261 [RFC3261]). An IS may have 707 Associated Media description in the SDP body. An IS may have 708 multiple Associated Media streams. The inclusion of media is test 709 case dependent. An IS is successfully established if the 710 following two conditions are met: 711 1. Sess.sig is established by the end of Establishment Threshold 712 Time (c.f. Section 3.3.3), and 713 2. If a media session is described in the SDP body of the 714 signaling message, then the media session is established by 715 the end of Establishment Threshold Time (c.f. Section 3.3.3). 717 Measurement Units: 718 N/A. 720 Issues: 721 None. 723 See Also: 724 Session 725 Non-Invite initiated Session 726 Associated Media 728 3.1.9. Non-INVITE-initiated Session (NS) 730 Definition: 731 A session that is created by an exchange of messages in the 732 Signaling Plane that does not include an initial SIP INVITE 733 message. 735 Discussion: 736 An NS is successfully established if the Session Attempt via a 737 non- INVITE request results in the EA receiving a 2xx reply from 738 the DUT/SUT before the expiration of the Establishment Threshold 739 timer (c.f., Section 3.3.3). An example of a NS is a session 740 created by the SUBSCRIBE request. 742 Measurement Units: 743 N/A. 745 Issues: 746 None. 748 See Also: 749 Session 750 Invite-initiated Session 752 3.1.10. Session Attempt Failure 754 Definition: 755 A session attempt that does not result in an Established Session. 757 Discussion: 758 The session attempt failure may be indicated by the following 759 observations at the Emulated Agent: 760 1. Receipt of a SIP 4xx, 5xx, or 6xx class response to a Session 761 Attempt. 762 2. The lack of any received SIP response to a Session Attempt 763 within the Establishment Threshold Time (c.f. Section 3.3.3). 765 Measurement Units: 766 N/A. 768 Issues: 769 None. 771 See Also: 772 Session Attempt 774 3.1.11. Standing Sessions Count 776 Definition: 777 The number of Sessions currently established on the DUT/SUT at any 778 instant. 780 Discussion: 781 The number of Standing Sessions is influenced by the Session 782 Duration and the Session Attempt Rate. Benchmarks MUST be 783 reported with the maximum and average Standing Sessions for the 784 DUT/SUT. In order to determine the maximum and average Standing 785 Sessions on the DUT/SUT for the duration of the test it is 786 necessary to make periodic measurements of the number of Standing 787 Sessions on the DUT/SUT. The recommended value for the 788 measurement period is 1 second. 790 Measurement Units: 791 Number of sessions 793 Issues: 794 None. 796 See Also: 797 Session Duration 798 Session Attempt Rate 799 Session Attempt Rate 801 3.2. Test Components 803 3.2.1. Emulated Agent 805 Definition: 806 A device in test topology that initiates/responds to SIP messages 807 as one or more session endpoints and, wherever applicable, 808 sources/receives Associated Media for Established Sessions. 810 Discussion: 811 The Emulated Agent functions in the signaling and media planes. 812 The Tester may act as multiple Emulated Agents. 814 Measurement Units: 815 N/A 817 Issues: 818 None. 820 See Also: 821 Media Plane 822 Signaling Plane 823 Established Session 824 Associated Media 826 3.2.2. Signaling Server 828 Definition: 829 Device in test topology that acts to create sessions between 830 Emulated Agents in the media plane. This device is either a DUT 831 or component of a SUT. 833 Discussion: 834 The DUT MUST be a RFC 3261 capable network equipment such as a 835 Registrar, Redirect Server, User Agent Server, Stateless Proxy, or 836 Stateful Proxy. A DUT MAY also include B2BUA or SBC. 838 Measurement Units: 839 NA 841 Issues: 842 None. 844 See Also: 845 Signaling Plane 847 3.2.3. SIP-Aware Stateful Firewall 849 Definition: 850 Device in test topology that provides Denial-of-Service (DoS) 851 Protection to the Signaling and Media Planes for the Emulated 852 Agents and Signaling Server 854 Discussion: 855 The SIP-Aware Stateful Firewall MAY be an internal component or 856 function of the Session Server. The SIP-Aware Stateful Firewall 857 MAY be a standalone device. If it is a standalone device it MUST 858 be paired with a Signaling Server. If it is a standalone device 859 it MUST be benchmarked as part of a SUT. SIP-Aware Stateful 860 Firewalls MAY include Network Address Translation (NAT) 861 functionality. Ideally, the inclusion of the SIP-Aware Stateful 862 Firewall as a SUT has no degradation to the measured performance 863 benchmarks. 865 Measurement Units: 866 N/A 868 Issues: 869 None. 871 See Also: 873 3.2.4. SIP Transport Protocol 875 Definition: 877 The protocol used for transport of the Signaling Plane messages. 879 Discussion: 880 Performance benchmarks may vary for the same SIP networking device 881 depending upon whether TCP, UDP, TLS, SCTP, or another transport 882 layer protocol is used. For this reason it MAY be necessary to 883 measure the SIP Performance Benchmarks using these various 884 transport protocols. Performance Benchmarks MUST report the SIP 885 Transport Protocol used to obtain the benchmark results. 887 Measurement Units: 888 TCP,UDP, SCTP, TLS over TCP, TLS over UDP, or TLS over SCTP 890 Issues: 891 None. 893 See Also: 895 3.3. Test Setup Parameters 897 3.3.1. Session Attempt Rate 899 Definition: 900 Configuration of the Emulated Agent for the number of sessions 901 that the Emulated Agent attempts to establish with the DUT/SUT 902 over a specified time interval. 904 Discussion: 905 The Session Attempt Rate can cause variation in performance 906 benchmark measurements. Since this is the number of sessions 907 configured on the Tester, some sessions may not be successfully 908 established on the DUT. A session may be either an IS or an NS. 910 Measurement Units: 911 Session attempts per second 913 Issues: 914 None. 916 See Also: 917 Session 918 Session Attempt 920 3.3.2. IS Media Attempt Rate 921 Definition: 922 Configuration on the Emulated Agent for number of ISs with 923 Associated Media to be established at the DUT per continuous one- 924 second time intervals. 926 Discussion: 927 Note that a Media Session MUST be associated with an IS. In this 928 document we assume that there is a one to one correspondence 929 between IS session attempts and Media Session attempts. By 930 including this definition we leave open the possibility that there 931 may be an IS that does not include a media description. Also note 932 that the IS Media Attempt Rate defines the number of media 933 sessions we are trying to create, not the number of media sessions 934 that are actually created. Variations in the Media Session 935 Attempt Rate might cause variations in performance benchmark 936 measurements. Some attempts might not result in successful 937 sessions established on the DUT. 939 Measurement Units: 940 session attempts per second (saps) 942 Issues: 943 None. 945 See Also: 946 IS 948 3.3.3. Establishment Threshold Time 950 Definition: 951 Configuration of the Emulated Agent for representing the amount of 952 time that an Emulated Agent will wait before declaring a Session 953 Attempt Failure. 955 Discussion: 956 This time duration is test dependent. 957 It is RECOMMENDED that the Establishment Threshold Time value be 958 set to Timer B (for ISs) or Timer F (for NSs) as specified in RFC 959 3261, Table 4 [RFC3261]. Following the default value of T1 960 (500ms) specified in the table and a constant multiplier of 64 961 gives a value of 32 seconds for this timer (i.e., 500ms * 64 = 962 32s). 964 Measurement Units: 966 seconds 968 Issues: 969 None. 971 See Also: 972 session establishment failure 974 3.3.4. Session Duration 976 Definition: 977 Configuration of the Emulated Agent that represents the amount of 978 time that the SIP dialog is intended to exist between the two EAs 979 associated with the test. 981 Discussion: 982 The time at which the BYE is sent will control the Session 983 Duration 984 Normally the Session Duration will be the same as the Media 985 Session Hold Time. However, it is possible that the dialog 986 established between the two EAs can support different media 987 sessions at different points in time. Providing both parameters 988 allows the testing agency to explore this possibility. 990 Measurement Units: 991 seconds 993 Issues: 994 None. 996 See Also: 997 Media Session Hold Time 999 3.3.5. Media Packet Size 1001 Definition: 1002 Configuration on the Emulated Agent for a fixed size of packets 1003 used for media streams. 1005 Discussion: 1006 For a single benchmark test, all sessions use the same size packet 1007 for media streams. The size of packets can cause variation in 1008 performance benchmark measurements. 1010 Measurement Units: 1011 bytes 1013 Issues: 1014 None. 1016 See Also: 1018 3.3.6. Media Offered Load 1020 Definition: 1021 Configuration of the Emulated Agent for the constant rate of 1022 Associated Media traffic offered by the Emulated Agent to the DUT/ 1023 SUT for one or more Established Sessions of type IS. 1025 Discussion: 1026 The Media Offered Load to be used for a test MUST be reported with 1027 three components: 1028 1. per Associated Media stream; 1029 2. per IS; 1030 3. aggregate. 1031 For a single benchmark test, all sessions use the same Media 1032 Offered Load per Media Stream. There may be multiple Associated 1033 Media streams per IS. The aggregate is the sum of all Associated 1034 Media for all IS. 1036 Measurement Units: 1037 packets per second (pps) 1039 Issues: 1040 None. 1042 See Also: 1043 Established Session 1044 Invite Initiated Session 1045 Associated Media 1047 3.3.7. Media Session Hold Time 1049 Definition: 1050 Parameter configured at the Emulated Agent, that represents the 1051 amount of time that the Associated Media for an Established 1052 Session of type IS will last. 1054 Discussion: 1055 The Associated Media streams may be bi-directional or uni- 1056 directional as indicated in the test methodology. 1057 Normally the Media Session Hold Time will be the same as the 1058 Session Duration. However, it is possible that the dialog 1059 established between the two EAs can support different media 1060 sessions at different points in time. Providing both parameters 1061 allows the testing agency to explore this possibility. 1063 Measurement Units: 1064 seconds 1066 Issues: 1067 None. 1069 See Also: 1070 Associated Media 1071 Established Session 1072 Invite-initiated Session (IS) 1074 3.3.8. Loop Detection Option 1076 Definition: 1077 An option that causes a Proxy to check for loops in the routing of 1078 a SIP request before forwarding the request. 1080 Discussion: 1081 This is an optional process that a SIP proxy may employ; the 1082 process is described under Proxy Behavior in RFC 3261 in Section 1083 16.3 Request Validation and that section also contains suggestions 1084 as to how the option could be implemented. Any procedure to 1085 detect loops will use processor cycles and hence could impact the 1086 performance of a proxy. 1088 Measurement Units: 1089 NA 1091 Issues: 1092 None. 1094 See Also: 1096 3.3.9. Forking Option 1097 Definition: 1098 An option that enables a Proxy to fork requests to more than one 1099 destination. 1101 Discussion: 1102 This is an process that a SIP proxy may employ to find the UAS. 1103 The option is described under Proxy Behavior in RFC 3261 in 1104 Section 16.1. A proxy that uses forking must maintain state 1105 information and this will use processor cycles and memory. Thus 1106 the use of this option could impact the performance of a proxy and 1107 different implementations could produce different impacts. 1108 SIP supports serial or parallel forking. When performing a test, 1109 the type of forking mode MUST be indicated. 1111 Measurement Units: 1112 The number of endpoints that will receive the forked invitation. 1113 A value of 1 indicates that the request is destined to only one 1114 endpoint, a value of 2 indicates that the request is forked to two 1115 endpoints, and so on. This is an integer value ranging between 1 1116 and N inclusive, where N is the maximum number of endpoints to 1117 which the invitation is sent. 1118 Type of forking used, namely parallel or serial. 1120 Issues: 1121 None. 1123 See Also: 1125 3.4. Benchmarks 1127 3.4.1. Registration Rate 1129 Definition: 1130 The maximum number of registrations that can be successfully 1131 completed by the DUT/SUT in a given time period. 1133 Discussion: 1134 This benchmark is obtained with zero failure in which 100% of the 1135 registrations attempted by the Emulated Agent are successfully 1136 completed by the DUT/SUT. The maximum value is obtained by 1137 testing to failure. This means that the registration rate 1138 provisioned on the EA is raised progressively until a registration 1139 attempt failure is observed. 1141 Measurement Units: 1142 registrations per second (rps) 1144 Issues: 1145 None. 1147 See Also: 1149 3.4.2. Session Establishment Rate 1151 Definition: 1152 The average maximum rate at which the DUT/SUT can successfully 1153 establish sessions. 1155 Discussion: 1156 This metric is an average of maxima. Each maximum is measured in 1157 a separate sample. The Session Establishment Rate is the average 1158 of the maximas established in each individual sample. In each 1159 sample, the maximum in question is the number of sessions 1160 successfully established in continuous one-second intervals with 1161 prior sessions remaining active. This maximum is designated in 1162 the equation below as "rate in sample i". The session 1163 establishment rate is calculated using the following equation (n = 1164 number of samples): 1166 n 1167 -- 1168 \ rate at sample i 1169 / 1170 -- 1171 i = 1 1172 --------------------- 1173 (n) 1175 In each sample, the maximum is obtained by testing to failure. 1176 With zero failure, 100% of the sessions introduced by the Emulated 1177 Agent are successfully established. The maximum value is obtained 1178 by testing to failure. This means that the Session Attempt Rate 1179 provisioned on the EA is raised progressively until a Session 1180 Attempt Failure is observed. The maximum rate is the rate 1181 acheived in the interval prior to the interval in which the 1182 failure is observed. Sessions may be IS or NS or a a mix of both 1183 and will be defined in the particular test. 1185 Measurement Units: 1186 sessions per second (sps) 1188 Issues: 1189 None. 1191 See Also: 1192 Invite-initiated Sessions 1193 Non-INVITE initiated Sessions 1194 Session Attempt Rate 1196 3.4.3. Session Capacity 1198 Definition: 1199 The maximum value of Standing Sessions Count achieved by the DUT/ 1200 SUT during the process of steadily increasing the number of 1201 Session Attempts per unit time, before the first Session Attempt 1202 Failure occurs. 1204 Discussion: 1205 When benchmarking Session Capacity for sessions with media it is 1206 required that these sessions be permanently established (i.e., 1207 they remain active for the duration of the test.) This can be 1208 achieved by causing the EA not to send a BYE for the duration of 1209 the testing. In the signaling plane, this requirement means that 1210 the dialog lasts as long as the test lasts. In order to test 1211 Session Capacity for sessions with media, the Media Session Hold 1212 Time MUST be set to infinity so that sessions remain established 1213 for the duration of the test. If the DUT/SUT is dialog-stateful, 1214 then we expect its performance will be impacted by setting Media 1215 Session Hold Time to infinity, since the DUT/SUT will need to 1216 allocate resources to process and store the state information. 1217 The Session Capacity must be reported with the Session Attempt 1218 Rate used to reach the maximum. Since Session Attempt Rate is a 1219 zero-loss measurement, there must be zero failures to achieve the 1220 Session Capacity. The maximum is indicated at the Emulated Agent 1221 by arrival of a SIP 4xx, 5xx, or 6xx response from the DUT/SUT. 1222 Sessions may be IS or NS. 1224 Measurement Units: 1225 sessions 1227 Issues: 1228 None. 1230 See Also: 1231 Established Session 1232 Session Attempt Rate 1233 Session Attempt Failure 1235 3.4.4. Session Overload Capacity 1237 Definition: 1238 The maximum number of Established Sessions that can exist 1239 simultaneously on the DUT/SUT until it stops responding to Session 1240 Attempts. 1242 Discussion: 1243 Session Overload Capacity is measured after the Session Capacity 1244 is measured. The Session Overload Capacity is greater than or 1245 equal to the Session Capacity. When benchmarking Session Overload 1246 Capacity, continue to offer Session Attempts to the DUT/SUT after 1247 the first Session Attempt Failure occurs and measure Established 1248 Sessions until no there is no SIP message response for the 1249 duration of the Establishment Threshold. It is worth noting that 1250 the Session Establishment Performance is expected to decrease 1251 after the first Session Attempt Failure occurs. 1253 Units: 1254 Sessions 1256 Issues: 1257 None. 1259 See Also: 1260 Overload 1261 Session Capacity 1262 Session Attempt Failure 1264 3.4.5. Session Establishment Performance 1266 Definition: 1267 The percent of Session Attempts that become Established Sessions 1268 over the duration of a benchmarking test. 1270 Discussion: 1271 Session Establishment Performance is a benchmark to indicate 1272 session establishment success for the duration of a test. The 1273 duration for measuring this benchmark is to be specified in the 1274 Methodology. The Session Duration SHOULD be configured to 1275 infinity so that sessions remain established for the entire test 1276 duration. 1278 Session Establishment Performance is calculated as shown in the 1279 following equation: 1281 Session Establishment = Total Established Sessions 1282 Performance -------------------------- 1283 Total Session Attempts 1285 Session Establishment Performance may be monitored real-time 1286 during a benchmarking test. However, the reporting benchmark MUST 1287 be based on the total measurements for the test duration. 1289 Measurement Units: 1290 Percent (%) 1292 Issues: 1293 None. 1295 See Also: 1296 Established Session 1297 Session Attempt 1299 3.4.6. Session Attempt Delay 1301 Definition: 1302 The average time measured at the Emulated Agent for a Session 1303 Attempt to result in an Established Session. 1305 Discussion: 1306 Time is measured from when the EA sends the first INVITE for the 1307 call-ID in the case of an IS. Time is measured from when the EA 1308 sends the first non-INVITE message in the case of an NS. Session 1309 Attempt Delay MUST be measured for every established session to 1310 calculate the average. Session Attempt Delay MUST be measured at 1311 the Maximum Session Establishment Rate. 1313 Measurement Units: 1314 Seconds 1316 Issues: 1317 None. 1319 See Also: 1320 Maximum Session Establishment Rate 1322 3.4.7. IM Rate 1323 Definition: 1324 Maximum number of IM messages completed by the DUT/SUT. 1326 Discussion: 1327 For a UAS, the definition of success is the receipt of an IM 1328 request and the subsequent sending of a final response. 1329 For a UAC, the definition of success is the sending of an IM 1330 request and the receipt of a final response to it. For a proxy, 1331 the definition of success is as follows: 1332 A. the number of IM requests it receives from the upstream client 1333 MUST be equal to the number of IM requests it sent to the 1334 downstream server; and 1335 B. the number of IM responses it receives from the downstream 1336 server MUST be equal to the number of IM requests sent to the 1337 downstream server; and 1338 C. the number of IM responses it sends to the upstream client 1339 MUST be equal to the number of IM requests it received from 1340 the upstream client. 1342 Measurement Units: 1343 IM messages per second 1345 Issues: 1346 None. 1348 See Also: 1350 4. IANA Considerations 1352 This document requires no IANA considerations. 1354 5. Security Considerations 1356 Documents of this type do not directly affect the security of 1357 Internet or corporate networks as long as benchmarking is not 1358 performed on devices or systems connected to production networks. 1359 Security threats and how to counter these in SIP and the media layer 1360 is discussed in RFC3261 [RFC3261], RFC 3550 [RFC3550], RFC3711 1361 [RFC3711] and various other drafts. This document attempts to 1362 formalize a set of common terminology for benchmarking SIP networks. 1363 Packets with unintended and/or unauthorized DSCP or IP precedence 1364 values may present security issues. Determining the security 1365 consequences of such packets is out of scope for this document. 1367 6. Acknowledgments 1369 The authors would like to thank Keith Drage, Cullen Jennings, Daryl 1370 Malas, Al Morton, and Henning Schulzrinne for invaluable 1371 contributions to this document. 1373 7. References 1375 7.1. Normative References 1377 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1378 Requirement Levels", BCP 14, RFC 2119, March 1997. 1380 [RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for 1381 Network Interconnect Devices", RFC 2544, March 1999. 1383 [RFC3261] Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston, 1384 A., Peterson, J., Sparks, R., Handley, M., and E. 1385 Schooler, "SIP: Session Initiation Protocol", RFC 3261, 1386 June 2002. 1388 [I-D.ietf-bmwg-sip-bench-meth] 1389 Poretsky, S., Gurbani, V., and C. Davids, "Methodology for 1390 Benchmarking SIP Networking Devices", 1391 draft-ietf-bmwg-sip-bench-meth-02 (work in progress), 1392 July 2010. 1394 7.2. Informational References 1396 [RFC2285] Mandeville, R., "Benchmarking Terminology for LAN 1397 Switching Devices", RFC 2285, February 1998. 1399 [RFC1242] Bradner, S., "Benchmarking terminology for network 1400 interconnection devices", RFC 1242, July 1991. 1402 [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. 1403 Jacobson, "RTP: A Transport Protocol for Real-Time 1404 Applications", STD 64, RFC 3550, July 2003. 1406 [RFC3711] Baugher, M., McGrew, D., Naslund, M., Carrara, E., and K. 1407 Norrman, "The Secure Real-time Transport Protocol (SRTP)", 1408 RFC 3711, March 2004. 1410 [I-D.ietf-sipping-overload-design] 1411 Hilt, V., Noel, E., Shen, C., and A. Abdelal, "Design 1412 Considerations for Session Initiation Protocol (SIP) 1413 Overload Control", draft-ietf-sipping-overload-design-02 1414 (work in progress), July 2009. 1416 [I-D.ietf-sipping-overload-reqs] 1417 Rosenberg, J., "Requirements for Management of Overload in 1418 the Session Initiation Protocol", 1419 draft-ietf-sipping-overload-reqs-05 (work in progress), 1420 July 2008. 1422 Appendix A. White Box Benchmarking Terminology 1424 Session Attempt Arrival Rate 1426 Definition: 1427 The number of Session Attempts received at the DUT/SUT over a 1428 specified time period. 1430 Discussion: 1431 Sessions Attempts are indicated by the arrival of SIP INVITES OR 1432 SUBSCRIBE NOTIFY messages. Session Attempts Arrival Rate 1433 distribution can be any model selected by the user of this 1434 document. It is important when comparing benchmarks of different 1435 devices that same distribution model was used. Common 1436 distributions are expected to be Uniform and Poisson. 1438 Measurement Units: 1439 Session attempts/sec 1441 Issues: 1442 None. 1444 See Also: 1445 Session Attempt 1447 Authors' Addresses 1449 Scott Poretsky 1450 Allot Communications 1451 300 TradeCenter, Suite 4680 1452 Woburn, MA 08101 1453 USA 1455 Phone: +1 508 309 2179 1456 Email: sporetsky@allot.com 1457 Vijay K. Gurbani 1458 Bell Laboratories, Alcatel-Lucent 1459 1960 Lucent Lane 1460 Rm 9C-533 1461 Naperville, IL 60566 1462 USA 1464 Phone: +1 630 224 0216 1465 Email: vkg@alcatel-lucent.com 1467 Carol Davids 1468 Illinois Institute of Technology 1469 201 East Loop Road 1470 Wheaton, IL 60187 1471 USA 1473 Phone: +1 630 682 6024 1474 Email: davids@iit.edu