idnits 2.17.1 draft-ietf-avt-rtptest-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing document type: Expected "INTERNET-DRAFT" in the upper left hand corner of the first page ** Missing expiration date. The document expiration date should appear on the first and last page. ** The document is more than 15 pages and seems to lack a Table of Contents. == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 6 longer pages, the longest (page 10) being 65 lines Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack a Security Considerations section. ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. ** There are 19 instances of too long lines in the document, the longest one being 4 characters in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- Couldn't find a document date in the document -- date freshness check skipped. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: '1' is defined on line 891, but no explicit reference was found in the text == Unused Reference: '2' is defined on line 894, but no explicit reference was found in the text ** Obsolete normative reference: RFC 1889 (ref. '2') (Obsoleted by RFC 3550) ** Downref: Normative reference to an Experimental RFC: RFC 2762 (ref. '3') Summary: 10 errors (**), 0 flaws (~~), 4 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Colin Perkins 2 USC/ISI 3 Jonathan Rosenberg 4 dynamicsoft 5 Henning Schulzrinne 6 Columbia University 8 RTP Testing Strategies 10 draft-ietf-avt-rtptest-03.txt 12 Status of this memo 14 This document is an Internet-Draft and is in full conformance with 15 all provisions of Section 10 of RFC2026. 17 Internet-Drafts are working documents of the Internet Engineering 18 Task Force (IETF), its areas, and its working groups. Note that 19 other groups may also distribute working documents as Internet-Drafts. 21 Internet-Drafts are draft documents valid for a maximum of six months 22 and may be updated, replaced, or obsoleted by other documents at 23 any time. It is inappropriate to use Internet-Drafts as reference 24 material or to cite them other than as work in progress. 26 The list of current Internet-Drafts can be accessed at 27 http://www.ietf.org/ietf/1id-abstracts.txt 29 The list of Internet-Draft Shadow Directories can be accessed at 30 http://www.ietf.org/shadow.html. 32 Distribution of this document is unlimited. 34 Comments are solicited and should be addressed to the author and/or the 35 IETF Audio/Video Transport working group's mailing list at rem-conf@es.net. 37 Abstract 39 This memo describes a possible testing strategy for RTP 40 implementations. It is intended as an aid to implementors 41 and does not specify a standard of any kind. 43 1 Introduction 45 This memo describes a possible testing strategy for RTP implementations. 46 The tests are intended to help demonstrate interoperability of multiple 47 implementations, and to illustrate common implementation errors. They 48 are not intended to be an exhaustive set of tests and passing these 49 tests does not necessarily imply conformance to the complete RTP 50 specification. 52 This memo is for information only and does not specify a standard 53 of any kind. 54 2 End systems 56 The architecture for testing RTP end systems is shown in figure 1. 58 +-----------------+ 59 +-------+ Test instrument +-----+ 60 | +-----------------+ | 61 | | 62 +-------+--------+ +-------+--------+ 63 | First RTP | | Second RTP | 64 | implementation | | implementation | 65 +----------------+ +----------------+ 67 Figure 1: Testing architecture 69 Both RTP implementations send packets to the test instrument, which 70 forwards packets from one implementation to the other. Unless otherwise 71 specified, packets are forwarded with no additional delay and without 72 loss. The test instrument is required to delay or discard packets 73 in some of the tests. The test instrument is invisible to the RTP 74 implementations - it merely simulates poor network conditions. 76 The test instrument is also capable of logging packet contents for 77 inspection of their correctness. 79 A typical test setup might comprise three machines on a single Ethernet 80 segment. Two of these machines run the RTP implementations, the 81 third runs the test instrument. The test instrument is an application 82 level packet forwarder. Both RTP implementations are instructed to 83 send unicast RTP packets to the test instrument, which forwards packets 84 between them. 86 2.1 Media transport 88 The aim of these tests is to show that basic media flows can be 89 exchanged between the two RTP implementations. The initial test is 90 for the first RTP implementation to transmit and the second to receive. 91 If this succeeds, the process is reversed, with the second implementation 92 sending and the first receiving. 94 The receiving application should be able to handle the following 95 edge cases, in addition to normal operation: 97 o Verify reception of packets which contain padding. 99 o Verify reception of packets which have the marker bit set 101 o Verify correct operation during sequence number wrap-around. 103 o Verify correct operation during timestamp wrap-around. 105 o Verify that the implementation correctly differentiates packets 106 according to the payload type field. 108 o Verify that the implementation ignores packets with unsupported 109 payload types 111 o Verify that the implementation can playout packets containing 112 a CSRC list and non-zero CC field (see section 4). 114 The sending application should be verified to correctly handle the 115 following edge cases: 117 o If padding is used, verify that the padding length indicator 118 (last octet of the packet) is correctly set and that the length 119 of the data section of the packet corresponds to that of this 120 particular payload plus the padding. 122 o Verify correct handling of the M bit, as defined by the profile. 124 o Verify that the SSRC is chosen randomly. 126 o Verify that the initial value of the sequence number is randomly 127 selected. 129 o Verify that the sequence number increments by one for each packet 130 sent. 132 o Verify correct operation during sequence number wrap-around. 134 o Verify that the initial value of the timestamp is randomly selected. 136 o Verify correct increment of timestamp (dependent on the payload 137 format). 139 o Verify correct operation during timestamp wrap-around. 141 o Verify correct choice of payload type according to the chosen 142 payload format, profile and any session level control protocol. 144 2.2 RTP Header Extension 146 An RTP implementation which does not use an extended header should 147 be able to process packets containing an extension header by ignoring 148 the extension. 150 If an implementation makes use of the header extension, it should 151 be verified that the profile specific field and the length field 152 of the extension are set correctly, and that the length of the packet 153 is consistent. 155 2.3 Basic RTCP 157 An RTP implementation is required to send RTCP control packets in 158 addition to data packets. 160 The first test requires both implementations to be run, but neither 161 sends data. It should be verified that RTCP packets are generated 162 by each implementation, and that those packets are correctly received 163 by the other implementation. It should also be verified that: 165 o all RTCP packets sent are compound packets 167 o all RTCP compound packets start with an empty RR packet 169 o all RTCP compound packets contain an SDES CNAME packet 171 The first implementation should then be made to transmit data packets. 172 It should be verified that that implementation now generates SR packets 173 in place of RR packets, and that the second application now generates 174 RR packets containing a single report block. It should be verified 175 that these SR and RR packets are correctly received. The following 176 features of the SR packets should also be verified: 178 o that the length field is consistent with both the length of 179 the packet and the RC field 181 o that the SSRC in SR packets is consistent with that in RTP data 182 packets 184 o that the NTP timestamp in a SR is sensible (matches the wall 185 clock time on the sending machine) 187 o that the RTP timestamp in a SR is consistent with that in the 188 RTP data packets 190 o that the packet and octet count fields in the SR packets are 191 consistent with the number of RTP data packets transmitted 193 In addition, the following features of the RR packets should also 194 be verified: 196 o that the SSRC in the report block is consistent with that in 197 the data packets being received 199 o that the fraction lost is zero 201 o that the cumulative number of packets lost is zero 203 o that the extended highest sequence number received is consistent 204 with the data packets being received (provided the round trip 205 time between test instrument and receiver is smaller than the 206 packet inter-arrival time, this can be directly checked by the 207 test instrument). 209 o that the interarrival jitter is small (a precise value cannot 210 be given, since it depends on the test instrument and network 211 conditions, but very little jitter should be present in this 212 scenario). 214 o that the last sender report timestamp is consistent with that 215 in the SR packets (ie: each RR passing through the test instrument 216 should contain the middle 32 bits from the 64 bit NTP timestamp 217 of the last SR packet which passed through the test instrument 218 in the opposite direction). 220 o that the delay since last SR field is sensible (an estimate 221 may be made by timing the passage of an SR and corresponding 222 RR through the test instrument, this should closely agree with 223 the DLSR field) 225 The next test is to show behaviour in the presence of packet loss. 226 The first implementation is made to transmit data packets, which 227 are received by the second implementation. This time, however, the 228 test instrument is made to randomly drop a small fraction (1% is 229 suggested) of the data packets. The second implementation should 230 be able to receive the data packets and process them in a normal 231 manner (with, of course, some quality degradation). The RR packets 232 should show a loss fraction corresponding to the drop rate of the 233 test instrument and should show an increasing cumulative number of 234 packets lost. 236 The loss rate in the test instrument is then returned to zero and 237 it is made to delay each packet by some random amount (the exact 238 amount depends on the media type, but a small fraction of the average 239 interarrival time is reasonable). The effect of this should be to 240 increase the reported interarrival jitter in the RR packets. 242 If these tests succeed, the process should be repeated with the second 243 implementation transmitting and the first receiving. 245 2.4 RTCP source description packets 247 Both implementations should be run, but neither is required to transmit 248 data packets. The RTCP packets should be observed and it should 249 be verified that each compound packet contains an SDES packet, that 250 that packet contains a CNAME item and that the CNAME is chosen according 251 to the rules in the RTP specification and profile (in many cases 252 the CNAME should be of the form `example@10.0.0.1' but this may be 253 overridden by a profile definition). 255 If an application supports additional SDES items then it should be 256 verified that they are sent in addition to the CNAME with some SDES 257 packets (the exact rate at which these additional items are included 258 is dependent on the application and profile). 260 It should be verified that an implementation can correctly receive NAME, 261 EMAIL, PHONE, LOC, NOTE, TOOL and PRIV items, even if it does not send 262 them. This is because it may reasonably be expected to interwork with 263 other implementations which support those items. Receiving and ignoring 264 such packets is valid behaviour. 266 It should be verified that an implementation correctly sets the length 267 fields in the SDES items it sends, and that the source count and 268 packet length fields are correct. 270 It should be verified that an implementation correctly receives SDES 271 items which do not terminate in a zero byte. 273 2.5 RTCP BYE packets 275 Both implementations should be run, but neither is required to transmit 276 data packets. The first implementation is then made to exit and 277 it should be verified that an RTCP BYE packet is sent. It should 278 be verified that the second implementation reacts to this BYE packet 279 and notes that the first implementation has left the session. 281 If the test succeeds, the implementations should be restarted and 282 the process repeated with the second implementation leaving the session. 284 It should be verified that implementations handle BYE packets containing 285 the optional reason for leaving text (ignoring the text is acceptable). 287 2.6 RTCP application defined packets 289 Tests for the correct response to application defined packets are 290 difficult to specify, since the response is clearly implementation 291 dependent. It should be verified that an implementation ignores APP 292 packets where the 4 octet name field is unrecognised. Implementations 293 which use APP packets should verify that they behave as expected. 295 2.7 RTCP transmission interval 297 The basic architecture for performing tests of the RTCP transmission 298 interval is shown in figure 2. 300 +--------------+ 301 | test | 302 | instrument | 303 +-----+--------+ 304 | 305 ------+----------+-------------- LAN 306 | 307 +-------+--------+ 308 | RTP | 309 | implementation | 310 +----------------+ 312 Figure 2: Testing architecture for RTCP 314 The test instrument is connected to the same LAN as the RTP implementation 315 being tested. It is assumed that the test instrument is preconfigured 316 with the addresses and ports used by the RTP implementation, and 317 is also aware of the RTCP bandwidth and sender/receiver fractions. 318 The tests can be conducted using either multicast or unicast. 320 The test instrument must be capable of sending arbitrarily crafted 321 RTP and RTCP packets to the RTP implementation. The test instrument 322 should also be capable of receiving packets sent by the RTP implementation, 323 parsing them, and computing metrics based on those packets. 325 It is furthermore assumed that a number of basic controls over the 326 RTP implementation exist. These controls are: 328 o the ability to force the implementation to send or not send 329 RTP packets at any desired point in time 331 o the ability to force the application to terminate its involvement 332 in the RTP session, and for this termination to be known immediately 333 to the test instrument 335 o the ability to set the session bandwidth and RTCP sender and 336 receiver fractions 338 The second of these is required only for the test of BYE reconsideration, 339 and is the only aspect of these tests not easily implementable by 340 pure automation. It will generally require manual intervention to 341 terminate the session from the RTP implementation and to convey this 342 to the test instrument through some non-RTP means. 344 2.7.1 Basic Behavior 346 The first test is to verify basic correctness of the implementation 347 of the RTCP transmission rules. This basic behavior consists of: 349 o periodic transmission of RTCP packets 351 o randomization of the interval for RTCP packet transmission 353 o correct implementation of the randomization interval computations, 354 with unconditional reconsideration 356 The RTP implementation acts as a receiver, and never sends any RTP 357 data packets. The implementation is configured with a large session 358 bandwidth, say 1 Mbit/s. This will cause the implementation to use 359 the minimal interval of 5s rather than the small interval based on 360 the session bandwidth and membership size. The implementation will 361 generate RTCP packets at this minimal interval, on average. The 362 test instrument generates no packets, but receives the RTCP packets 363 generated by the implementation. When an RTCP packet is received, 364 the time is noted by the test instrument. The difference in time 365 between each pair of subsequent packets (called the interval) is 366 computed. These intervals are stored, so that statistics based on 367 these intervals can be computed. It is recommended that this observation 368 process operate for at least 20 minutes. 370 An implementation passes this test if the intervals have the following 371 properties: 373 o the minimum interval is never less than 2 seconds or more than 374 2.5 seconds; 376 o the maximum interval is never more than 7 seconds or less than 377 5.5 seconds; 379 o the average interval is between 4.5 and 5.5 seconds; 381 o the number of intervals between x and x+500ms is less than the 382 number of intervals between x+500ms and x+1s, for any x. 384 In particular, an implementation fails if the packets are sent with 385 a constant interval. 387 2.7.2 Step join backoff 389 The main purpose of the reconsideration algorithm is to avoid a flood 390 of packets that might occur when a large number of users simultaneously 391 join an RTP session. Reconsideration therefore exhibits a backoff 392 behavior in sending of RTCP packets when group sizes increase. This 393 aspect of the algorithm can be tested in the following manner. 395 The implementation begins operation. The test instrument waits for 396 the arrival of the first RTCP packet. When it arrives, the test 397 instrument notes the time and then immediately sends 100 RTCP RR 398 packets to the implementation, each with a different SSRC and SDES 399 CNAME. The test instrument should ensure that each RTCP packet is 400 of the same length. The instrument should then wait until the next 401 RTCP packet is received from the implementation, and the time of 402 such reception is noted. 404 Without reconsideration, the next RTCP packet will arrive within a 405 short period of time. With reconsideration, transmission of this 406 packet will be delayed. The earliest it can arrive depends on the 407 RTCP session bandwidth, receiver fraction, and average RTCP packet 408 size. The RTP implementation should be using the exponential averaging 409 algorithm defined in the specification to compute the average RTCP 410 packet size. Since this is dominated by the received packets (the 411 implementation has only sent one itself), the average will be roughly 412 equal to the length of the RTCP packets sent by the test instrument. 413 Therefore, the minimum amount of time between the first and second 414 RTCP packets from the implementation is: 416 T > 101 * S / ( B * Fr * (e-1.5) * 2 ) 417 Where S is the size of the RTCP packets sent by the test instrument, 418 B is the RTCP bandwidth (normally five percent of the session bandwidth), 419 Fr is the fraction of RTCP bandwidth allocated to receivers (normally 420 75 percent), and e is the natural exponent. Without reconsideration, 421 this minimum interval Te would be much smaller: 423 Te > MAX( [ S / ( B * Fr * (e-1.5) * 2 ) ] , [ 2.5 / (e-1.5) ] ) 425 B should be chosen sufficiently small so that T is around 60 seconds. 426 Reasonable choices for these parameters are B = 950 bits per second, 427 and S = 1024 bits. An implementation passes this test if the interval 428 between packets is not less than T above, and not more than 3 times 429 T. 431 The test may be repeated for the case when the RTP implementation 432 is a sender. This is accomplished by having the implementation send 433 RTP packets at least once a second. In this case, the interval between 434 the first and second RTCP packets should be no less than: 436 T > S / ( B * Fs * (e-1.5) * 2 ) 438 Where Fs is the fraction of RTCP bandwidth allocated to senders, 439 usually 25%. Note that this value of T is significantly smaller 440 than the interval for receivers. 442 2.7.3 Steady State Behavior 444 In addition to the basic behavior in section 2.7.1, an implementation 445 should correctly implement a number of other, slightly more advanced 446 features: 448 o scale the RTCP interval with the group size; 450 o correctly divide bandwidth between senders and receivers; 452 o correctly compute the RTCP interval when the user is a sender 454 The implementation begins operation as a receiver. The test instrument 455 waits for the first RTCP packet from the implementation. When it 456 arrives, the test instrument notes the time, and immediately sends 457 50 RTCP RR packets and 50 RTCP SR packets to the implementation, 458 each with a different SSRC and SDES CNAME. The test instrument then 459 sends 50 RTP packets, using the 50 SSRC from the RTCP SR packets. 461 The test instrument should ensure that each RTCP packet is of the 462 same length. The instrument should then wait until the next RTCP 463 packet is received from the implementation, and the time of such 464 reception is noted. The difference between the reception of the 465 RTCP packet and the reception of the previous is computed and stored. 466 In addition, after every RTCP packet reception, the 100 RTCP and 467 50 RTP packets are retransmitted by the test instrument. This ensures 468 that the sender and member status of the 100 users does not time 469 out. The test instrument should collect the interval measurements 470 figures for at least 100 RTCP packets. 472 With 50 senders, the implementation should not try to divide the 473 RTCP bandwidth between senders and receivers, but rather group all 474 users together and divide the RTCP bandwidth equally. The test is 475 deemed successful if the average RTCP interval is within 5% of: 477 T = 101* S/B 479 Where S is the size of the RTCP packets sent by the test instrument, 480 and B is the RTCP bandwidth. B should be chosen sufficiently small 481 so that the value of T is on the order of tens of seconds or more. 482 Reasonable values are S=1024 bits and B=3.4 kb/s. 484 The previous test is repeated. However, the test instrument sends 485 10 RTP packets instead of 50, and 10 RTCP SR and 90 RTCP RR instead 486 of 50 of each. In addition, the implementation is made to send at 487 least one RTP packet between transmission of every one of its own 488 RTCP packets. 490 In this case, the average RTCP interval should be within 5% of: 492 T = 11 * S / (B * Fs) 494 Where S is the size of the RTCP packets sent by the test instrument, 495 B is the RTCP bandwidth, and Fs is the fraction of RTCP banwidth 496 allocated for senders (normally 25%). The values for B and S should 497 be chosen small enough so that T is on the order of tens of seconds. 498 Reasonable choices are S=1024 bits and B=1.5 kb/s. 500 2.7.4 Reverse Reconsideration 502 The reverse reconsideration algorithm is effectively the opposite 503 of the normal reconsideration algorithm. It causes the RTCP interval 504 to be reduced more rapidly in response to decreases in the group 505 membership. This is advantageous in that it keeps the RTCP information 506 as fresh as possible, and helps avoids some premature timeout problems. 508 In the first test, the implementation joins the session as a receiver. 509 As soon as the implementation sends its first RTCP packet, the test 510 instrument sends 100 RTCP RR packets, each of the same length S, 511 and a different SDES CNAME and SSRC in each. It then waits for the 512 implementation to send another RTCP packet. Once it does, the test 513 instrument sends 100 BYE packets, each one containing a different 514 SSRC, but matching an SSRC from one of the initial RTCP packets. 515 Each BYE should also be the same size as the RTCP packets sent by 516 the test instrument. This is easily accomplished by using a BYE 517 reason to pad out the length. The time of the next RTCP packet from 518 the implementation is then noted. The delay T between this (the 519 third RTCP packet) and the previous should be no more than: 521 T < 3 * S / (B * Fr * (e-1.5) * 2) 523 Where S is the size of the RTCP and BYE packets sent by the test 524 instrument, B is the RTCP bandwidth, Fr is the fraction of the RTCP 525 bandwidth allocated to receivers, and e is the natural exponent. 526 B should be chosen such that T is on the order of tens of seconds. 527 A reasonable choice is S=1024 bits and B=168 bits per second. 529 This test demonstrates basic correctness of implementation. An 530 implementation without reverse reconsideration will not send its next 531 RTCP packet for nearly 100 times as long as the above amount. 533 In the second test, the implementation joins the session as a receiver. 534 As soon as it sends its first RTCP packet, the test instrument sends 535 100 RTCP RR packets, each of the same length S, followed by 100 BYE 536 packets, also of length S. Each RTCP packet carries a different SDES 537 CNAME and SSRC, and is matched with precisely one BYE packet with 538 the same SSRC. This will cause the implementation to see a rapid 539 increase and then rapid drop in group membership. 541 The test is deemed succesful if the next RTCP packet shows up T seconds 542 after the first, and T is within: 544 2.5 / (e-1.5) < T < 7.5 / (e-1.5) 546 This tests correctness of the maintenance of the pmembers variable. 547 An incorrect implementation might try to execute reverse reconsideration 548 every time a BYE is received, as opposed to only when the group membership 549 drops below pmembers. If an implementation did this, it would end 550 up sending an RTCP packet immediately after receiving the stream 551 of BYE's. For this test to work, B must be chosen to be a large 552 value, around 1Mb/s. 554 2.7.5 BYE Reconsideration 556 The BYE reconsideration algorithm works in much the same fashion 557 as regular reconsideration, except applied to BYE packets. When a 558 user leaves the group, instead of sending a BYE immediately, it may 559 delay transmission of its BYE packet if others are sending BYE's. 561 The test for correctness of this algorithm is as follows. The RTP 562 implementation joins the group as a receiver. The test instrument 563 waits for the first RTCP packet. When the test instrument receives 564 this packet, the test instrument immediately sends 100 RTCP RR packets, 565 each of the same length S, and each containing a different SSRC and 566 SDES CNAME. Once the test instrument receives the next RTCP packet 567 from the implementation, the RTP implementation is made to leave 568 the RTP session, and this information is conveyed to the test instrument 569 through some non-RTP means. The test instrument then sends 100 BYE 570 packets, each with a different SSRC, and each matching an SSRC from 571 a previously transmitted RTCP packet. Each of these BYE packets 572 is also of size S. Immediately following the BYE packets, the test 573 instrument sends 100 RTCP RR packets, using the same SSRC/CNAMEs 574 as the original 100 RTCP packets. 576 The test is deemed successful if the implementation either never 577 sends a BYE, or if it does, the BYE is not received by the test 578 instrument earlier than T seconds after the implementation left the 579 session, where T is: 581 T = 100 * S / ( 2 * (e-1.5) * B * Fr) 583 S is the size of the RTCP and BYE packets, e is the natural exponent, 584 B is the RTCP bandwidth, and Fr is the RTCP bandwidth fraction for 585 receivers. S and B should be chosen so that T is on the order of 586 50 seconds. A reasonable choice is S=1024 bits and B=1.1 kb/s. 588 The transmission of the RTCP packets is meant to verify that the 589 implementation is ignoring non-BYE RTCP packets once it decides to 590 leave the group. 592 2.7.6 Timing out members 594 Active RTP participants are supposed to send periodic RTCP packets. 595 When a participant leaves the session, they may send a BYE, however 596 this is not required. Furthermore, BYE reconsideration may cause 597 a BYE to never be sent. As a result, participants must time out 598 other participants who have not sent an RTCP packet in a long time. 599 According to the specification, participants who have not sent an 600 RTCP packet in the last 5 intervals are timed out. This test verifies 601 that these timeouts are being performed correctly. 603 The RTP implementation joins a session as a receiver. The test instrument 604 waits for the first RTCP packet from the implementation. Once it 605 arrives, the test instrument sends 100 RTCP RR packets, each with 606 a different SDES and SSRC, and notes the time. This will cause the 607 implementation to believe that there are now 101 group participants, 608 causing it to increase its RTCP interval. The test instrument continues 609 to monitor the RTCP packets from the implementation. As each RTCP 610 packet is received, the time of its reception is noted, and the interval 611 between RTCP packets is stored. The 100 participants spoofed by 612 the test instrument should eventually time out at the RTP implementation. 613 This should cause the RTCP interval to be reduced to its minimum. 615 The test is deemed successful if the interval between RTCP packets 616 after the first is no less than: 618 Ti > 101 * S / ( 2 * (e-1.5) * B * Fr) 620 and this minimum interval is sustained no later than Td seconds after 621 the transmission of the 100 RR's, where Td is: 623 Td = 7 * 101 * S / ( B * Fr ) 625 and the interval between RTCP packets after this point is no less 626 than: 628 Tf > 2.5 / (e-1.5) 630 For this test to work, B and S must be chosen so Ti is on the order 631 of minutes. Recommended values are S = 1024 bits and B = 1.9 kbps. 633 2.7.7 Rapid SR's 635 The minimum interval for RTCP packets can be reduced for large session 636 bandwdiths. The reduction applies to senders only. The recommended 637 algorithm for computing this minimum interval is 360 divided by the 638 RTP session bandwidth, in kbps. For bandwidths larger than 72 kbps, 639 this interval is less than 5 seconds. 641 This test verifies the ability of an implementation to use a lower 642 RTCP minimum interval when it is a sender in a high bandwidth session. 643 The test can only be run on implementations that support this reduction, 644 since it is optional. 646 The RTP implementation is configured to join the session as a sender. 647 The session is configured to use 360 kbps. If the recommended algorithm 648 for computing the reduced minimum interval is used, the result is 649 a 1 second interval. If the RTP implementation uses a different 650 algorithm, the session bandwidth should be set in such a way to cause 651 the reduced minimum interval to be 1 second. 653 Once joining the session, the RTP implementation should begin to 654 send both RTP and RTCP packets. The interval between RTCP packets 655 is measured and stored until 100 intervals have been collected. 657 The test is deemed successful if the smallest interval is no less 658 than 1/2 a second, and the largest interval is no more than 1.5 seconds. 659 The average should be close to 1 second. 660 3 RTP translators 662 RTP translators should be tested in the same manner as end systems, 663 with the addition of the tests described in this section. 665 The architecture for testing RTP translators is shown in figure 3. 667 +-----------------+ 668 +-------+ RTP Translator +-----+ 669 | +-----------------+ | 670 | | 671 +-------+--------+ +-------+--------+ 672 | First RTP | | Second RTP | 673 | implementation | | implementation | 674 +----------------+ +----------------+ 676 Figure 3: Testing architecture for translators 678 The first RTP implementation is instructed to send data to the translator, 679 which forwards the packets to the other RTP implementation, after 680 translating then as desired. It should be verified that the second 681 implementation can playout the translated packets. 683 It should be verified that the packets received by the second implementation 684 have the same SSRC as those sent by the first implementation. The 685 CC should be zero and CSRC fields should not be present in the translated 686 packets. The other RTP header fields may be rewritten by the translator, 687 depending on the translation being performed, for example 689 o the payload type should change if the translator changes the 690 encoding of the data 692 o the timestamp may change if, for example, the encoding, packetisation 693 interval or framerate is changed 695 o the sequence number may change if the translator merges or splits 696 packets 698 o padding may be added or removed, in particular if the translator 699 is adding or removing encryption 701 o the marker bit may be rewritten 703 If the translator modifies the contents of the data packets it should 704 be verified that the corresponding change is made to the RTCP packets, 705 and that the receivers can correctly process the modified RTCP packets. 706 In particular 708 o the SSRC is unchanged by the translator 710 o if the translator changes the data encoding it should also change 711 the octet count field in the SR packets 713 o if the translator combines multiple data packets into one it 714 should also change the packet count field in SR packets 716 o if the translator changes the sampling frequency of the data 717 packets it should also change the RTP timestamp field in the 718 SR packets 720 o if the translator combines multiple data packets into one it 721 should also change the packet loss and extended highest sequence 722 number fields of RR packets flowing back from the receiver (it 723 is legal for the translator to strip the report blocks and send 724 empty SR/RR packets, but this should only be done if the transformation 725 of the data is such that the reception reports cannot sensibly 726 be translated) 728 o the translator should forward SDES CNAME packets 730 o the translator may forward other SDES packets 732 o the translator should forward BYE packets unchanged 734 o the translator should forward APP packets unchanged 736 When the translator exits it should be verified to send a BYE packet 737 to each receiver containing the SSRC of the other receiver. The 738 receivers should be verified to correctly process this BYE packet 739 (this is different to the BYE test in section 2.5 since multiple 740 SSRCs may be included in each BYE if the translator also sends its 741 own RTCP information). 742 4 RTP mixers 744 RTP mixers should be tested in the same manner as end systems, with 745 the addition of the tests described in this section. 747 The architecture for testing RTP mixers is shown in figure 4. 749 +----------------+ 750 | Second RTP | 751 | implementation | 752 +-------+--------+ 753 | 754 | +-----------+ 755 +-------+ RTP Mixer +-----+ 756 | +-----------+ | 757 | | 758 +-------+--------+ +-------+--------+ 759 | First RTP | | Third RTP | 760 | implementation | | implementation | 761 +----------------+ +----------------+ 763 Figure 4: Testing architecture for mixers 765 The first and second RTP implementations are instructed to send data 766 packets to the RTP mixer. The mixer combines those packets and sends 767 them to the third RTP implementation. The mixer should also process 768 RTCP packets from the other implementations, and should generate its 769 own RTCP reports. 771 It should be verified that the third RTP implementation can playout 772 the mixed packets. It should also be verified that 773 o the CC field in the RTP packets received by the third implementation 774 is set to 2 776 o the RTP packets received by the third implementation contain 777 2 CSRCs corresponding to the first and second RTP implementations 779 o the RTP packets received by the third implementation contain 780 an SSRC corresponding to that of the mixer 782 It should next be verified that the mixer generates SR and RR packets 783 for each cloud. The mixer should generate RR packets in the direction 784 of the first and second implementations, and SR packets in the direction 785 of the third implementation. 787 It should be verified that the SR packets sent to the third implementation 788 do not reference the first or second implementations, and vice versa. 790 It should be verified that SDES CNAME information is forwarded across 791 the mixer. Other SDES fields may optionally be forwarded. 793 Finally, one of the implementations should be quit, and it should 794 be verified that the other implementations see the BYE packet. This 795 implementation should then be restarted and the mixer should be quit. 796 It should be verified that the implementations see both the mixer 797 and the implementations on the other side of the mixer quit (illustrating 798 response to BYE packets containing multiple sources). 799 5 SSRC collision detection 801 RTP has provision for the resolution of SSRC collisions. These collisions 802 occur when two different session participants choose the same SSRC. 803 In this case, both participants are supposed to send a BYE, leave 804 the session, and rejoin with a different SSRC, but the same CNAME. 805 The purpose of this test is to verify that this function is present 806 in the implementation. 808 The test is straightforward. The RTP implementation is made to join 809 the multicast group as a receiver. A test instrument waits for the 810 first RTCP packet. Once it arrives, the test instrument notes the 811 CNAME and SSRC from the RTCP packet. The test instrument then generates 812 an RTCP receiver report. This receiver report contains an SDES chunk 813 with an SSRC matching that of the RTP implementation, but with a 814 different CNAME. At this point, the implementation should send a 815 BYE RTCP packet (containing an SDES chunk with the old SSRC and CNAME), 816 and then rejoin, causing it to send a receiver report containing 817 an SDES chunk, but with a new SSRC and the same CNAME. 819 The test is deemed succesful if the RTP implementation sends the 820 RTCP BYE and RTCP RR as described above within one minute of receiving 821 the colliding RR from the test instrument. 822 6 SSRC Randomization 824 According to the RTP specification, SSRC's are supposed to be chosen 825 randomly and uniformly over a 32 bit space. This randomization is 826 beneficial for several reasons: 828 o It reduces the probability of collisions in large groups. 830 o It simplifies the process of group sampling [3] which depends 831 on the uniform distribution of SSRC's across the 32 bit space. 833 Unfortunately, verifying that a random number has 32 bits of uniform 834 randomness requires a large number of samples. The procedure below 835 gives only a rough validation to the randomness used for generating 836 the SSRC. 838 The test runs as follows. The RTP implementation joins the group 839 as a receiver. The test instrument waits for the first RTCP packet. 840 It notes the SSRC in this RTCP packet. The test is repeated 2500 841 times, resulting in a collection of 2500 SSRC. 843 The are then placed into 25 bins. An SSRC with value X is placed 844 into bin FLOOR(X/(2**32 / 25)). The idea is to break the 32 bit 845 space into 25 regions, and compute the number of SSRC in each region. 846 Ideally, there should be 40 SSRC in each bin. Of course, the actual 847 number in each bin is a random variable whose expectation is 40. 848 With 2500 SSRC, the coefficient of variation of the number of SSRC 849 in a bin is 0.1, which means the number should be between 36 and 850 44. The test is thus deemed successful if each bin has no less than 851 30 and no more than 50 SSRC. 853 Running this test may require substantial amounts of time, particularly 854 if there is no automated way to have the implementation join the 855 session. In such a case, the test can be run fewer times. With 856 26 tests, half of the SSRC should be less than 2**31, and the other 857 half higher. The cofficient of variation in this case is 0.2, so 858 the test is successful if there are more than 8 SSRC less than 2**31, 859 and less than 26. 861 In general, if the SSRC is collected N times, and there are B bins, 862 the coefficient of variation of the number of SSRC in each bin is 863 given by: 865 coeff = SQRT( (B-1)/N ) 866 7 Authors' Address 868 Colin Perkins 869 USC Information Sciences Institute 870 4350 North Fairfax Drive 871 Suite 620 872 Arlington, VA 22203 873 Email: csp@isi.edu 875 Jonathan Rosenberg 876 dynamicsoft 877 72 Eagle Rock Ave. 878 First Floor 879 East Hanover, NJ 07936 880 Email: jdrosen@dynamicsoft.com 882 Henning Schulzrinne 883 Columbia University 884 M/S 0401 885 1214 Amsterdam Ave. 886 New York, NY 10027-7003 887 Email: schulzrinne@cs.columbia.edu 889 8 References 891 [1] S. Bradner, ``The Internet Standards Process -- Revision 3'', 892 RFC2026, Internet Engineering Task Force, October 1996. 894 [2] H. Schulzrinne, S. Casner, R. Frederick and V. Jacobson, ``RTP: 895 A Transport Protocol to Real-Time Applications'', RFC1889, Internet 896 Engineering Task Force, January 1996. 898 [3] J. Rosenberg and H. Schulzrinne, ``Sampling of the Group Membership 899 in RTP'', RFC2762, February 2000.