INTERNET-DRAFT 25 June 1999 Colin Perkins University College London RTP Testing Strategies draft-ietf-avt-rtptest-00.txt Status of this memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as work in progress. The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Distribution of this document is unlimited. Comments are solicited and should be addressed to the author and/or the IETF Audio/Video Transport working group's mailing list at rem-conf@es.net. Abstract This memo describes a possible testing strategy for RTP implementations and is intended as an aid to implementors. It does not specify a standard of any kind. 1 Introduction This memo describes a possible testing strategy for RTP implementations. The tests are intended to help demonstrate interoperability of multiple implementations, and to illustrate common implementation errors. They are not intended to be an exhaustive set of tests and passing these tests does not necessarily imply conformance to the complete RTP specification. This memo is for information only and does not specify a standard of any kind. Perkins Page 1 INTERNET-DRAFT 25 June 1999 2 End systems The architecture for testing RTP end systems is shown in figure 1. +-----------------+ +-------+ Test instrument +-----+ | +-----------------+ | | | +--------+-------+ +-------+--------+ | First RTP | | Second RTP | | implementation | | implementation | +----------------+ +----------------+ Figure 1: Testing architecture Both RTP implementations are instructed to send packets to the test instrument, which forwards packets from one implementation to the other. Unless otherwise specified, packets are forwarded with no additional delay or loss. The test instrument is also required to delay or discard packets in some of the tests. The test instrument is invisible to the RTP implementations - it merely simulates poor network conditions. The test instrument is also capable of logging packet contents for inspection of their correctness. A typical test setup might comprise three machines on a single Ethernet segment. Two of these machines run the RTP implementations, the third runs the test instrument. The test instrument is an application level UDP packet forwarder. Both RTP implementations are instructed to send unicast RTP/UDP packets to the test instrument, which forwards packets between them. 2.1 Media transport The aim of these tests is to show that basic media flows can be exchanged between the two RTP implementations. The initial test is for the first RTP implementation to transmit and the second to receive. If this succeeds, the process is reversed, with the second implementation sending and the first receiving. The receiving application should be able to handle the following edge cases, in addition to normal operation: o Verify reception of packets which contain padding. o Verify correct operation during sequence number wrap-around. o Verify correct operation during timestamp wrap-around. Perkins Page 2 INTERNET-DRAFT 25 June 1999 o Verify that the implementation ignores packets with unsupported payload types The sending application should be verified to correctly handle the following edge cases: o If padding is used, verify that the padding length indicator (last octet of the packet) is correctly set and that the length of the data section of the packet corresponds to that of this particular payload plus the padding. o Verify correct handling of the M bit, as defined by the profile. o Verify that the SSRC is chosen randomly. o Verify that the initial value of the sequence number is randomly selected. o Verify that the sequence number increments by one for each packet sent. o Verify correct operation during sequence number wrap-around. o Verify that the initial value of the timestamp is randomly selected. o Verify correct increment of timestamp (dependent on the payload format). o Verify correct operation during timestamp wrap-around. 2.2 RTP Header Extension An RTP implementation which does not use an extended header should be able to process packets containing an extension header by ignoring the extension. If an implementation makes use of the header extension, it should be verified that the profile specific field and the length field of the extension are set correctly, and that the length of the packet is consistent. 2.3 Basic RTCP An RTP implementation is required to send RTCP control packets in addition to data packets. The first test requires both implementations to be run, but neither sends data. It should be verified that RTCP packets are generated by each implementation, and that those packets are correctly received by the other implementation. It should also be verified that: Perkins Page 3 INTERNET-DRAFT 25 June 1999 o all RTCP packets sent are compound packets o all RTCP compound packets start with an empty RR packet o all RTCP compound packets contain an SDES CNAME packet The first implementation should then be made to transmit data packets. It should be verified that the first implementation now generates SR packets in place of RR packets, and that the second application now generates RR packets containing a single report block. It should be verified that these SR and RR packets are correctly received. The following features of the SR packets should also be verified: o that the length field is consistent with both the length of the packet and the RC field o that the SSRC in SR packets is consistent with that in RTP data packets o that the NTP timestamp in a SR is sensible o that the RTP timestamp in a SR is consistent with that in the RTP data packets o that the sender's packet and octet count fields are consistent with the RTP data packets transmitted In addition, the following features of the RR packets should also be verified: o that the SSRC in the report block is consistent with that in the data packets being received o that the fraction lost is zero o that the cumulative number of packets lost is zero o that the extended highest sequence number received is consistent with the data packets being received o that the interarrival jitter is small (a precise value cannot be given, since it depends on the test instrument, but very little jitter should be present in this scenario). o that the last sender report timestamp is consistent with that in the SR packets o that the delay since last SR field is sensible (an estimate may be made by timing the passage of an SR and corresponding RR through the test instrument) Perkins Page 4 INTERNET-DRAFT 25 June 1999 The next test is to show behaviour in the presence of packet loss. The first implementation is made to transmit data packets, which are received by the second implementation. This time, however, the test instrument is made to randomly drop a small fraction (1% is suggested) of the data packets. The second implementation should be able to receive the data packets and process them in a normal manner (with, of course, some quality degradation). The RR packets should show a loss fraction corresponding to the drop rate of the test instrument and should show an increasing cumulative number of packets lost. The loss rate in the test instrument is then returned to zero and it is made to delay each packet by some random amount (the exact amount depends on the media type, but a small fraction of the average interarrival time is reasonable). The effect of this should be to increase the reported interarrival jitter in the RR packets. If these tests succeed, the process should be repeated with the second implementation transmitting and the first receiving. 2.4 RTCP source description packets Both implementations should be run, but neither is required to transmit data packets. The RTCP packets should be observed and it should be verified that each compound packet contains an SDES packet and that that packet contains a CNAME item. If an application supports additional SDES items then it should be verified that they are sent in addition to the CNAME with some SDES packets (the exact rate at which these additional items are included is dependent on the application and profile). It should be verified that an implementation can correctly receive NAME, EMAIL, PHONE, LOC, NOTE, TOOL and PRIV items, even if it does not send them. This is because it may reasonably be expected to interwork with other implementations which support those items. Receiving and ignoring such packets is valid behaviour. It should be verified that an implementation correctly sets the length fields in the SDES items it sends, and that the source count and packet length fields are correct. It should be verified that an implementation correctly receives SDES items which do not terminate in a zero byte. 2.5 RTCP transmission interval See draft-ietf-avt-rtcptest-01.txt Perkins Page 5 INTERNET-DRAFT 25 June 1999 2.6 RTCP BYE packets Both implementations should be run, but neither is required to transmit data packets. The first implementation is then quit and it should be verified that an RTCP BYE packet is sent. If should be verified that the second implementation reacts to this BYE packet and notes that the first implementation has left the conference. If the test succeeds, the implementations should be restarted and the process repeated with the second implementation leaving the conference. It should be verified that implementations handle BYE packets containing the optional reason for leaving. 2.7 RTCP application defined packets Tests for the correct response to application defined packets are difficult to specify, since the response is clearly implementation dependent. It should be verified that an implementation ignores APP packets where the 4 octet name field is unrecognised. Implementations which use APP packets should verify that they behave as expected. 3 RTP translators RTP translators should be tested in the same manner as end systems, with the addition of the tests described in this section. The architecture for testing RTP translators is shown in figure 2. +-----------------+ +-------+ RTP Translator +-----+ | +-----------------+ | | | +--------+-------+ +-------+--------+ | First RTP | | Second RTP | | implementation | | implementation | +----------------+ +----------------+ Figure 2: Testing architecture for translators The first RTP implementation is instructed to send data to the translator, which forwards the packets to the other RTP implementation, after translating then as desired. It should be verified that the second implementation can playout the translated packets. It should be verified that the RTP packets sent to the second implementation have the same SSRC as those sent by the first implementation. The CC should be zero and CSRC fields should not be present in the translated Perkins Page 6 INTERNET-DRAFT 25 June 1999 packets. The other RTP header fields may be rewritten by the translator, depending on the translation being performed. If the translator modifies the contents of the data packets it should be verified that the corresponding change is made to the RTCP packets, and that both receivers can correctly process the modified RTCP packets. In particular o if the translator changes the data encoding it should also change the octet count field in the SR packets o if the translator combines multiple data packets into one it should also change the packet count field in SR packets o if the translator combines multiple data packets into one it should also change the packet loss and extended highest sequence number fields of RR packets flowing back from the receiver o if the translator changes the sampling frequency of the data packets it should also change the RTP timestamp field in the SR packets o the translator should forward SDES CNAME packets o the translator may forward other SDES packets o the translator should forward BYE packets unchanged o the translator should forward APP packets unchanged When the translator exits it should be verified to send a BYE packet to each receiver. That BYE packet should contain the SSRC of the translator and may contain the SSRC of the other receiver. The receivers should be verified to correctly process this BYE packet (this is different to the BYE test in section 2.6 since multiple SSRCs may be included in each BYE). 4 RTP mixers RTP mixers should be tested in the same manner as end systems, with the addition of the tests described in this section. The architecture for testing RTP mixers is shown in figure 3. The first and second RTP implementations are instructed to send data packets to the RTP mixer. The mixer combines those packets and sends them to the third RTP implementation. It should be verified that the third RTP implementation can playout the mixed packets. It should also be verified that Perkins Page 7 INTERNET-DRAFT 25 June 1999 +----------------+ | Second RTP | | implementation | +--------+-------+ | | +-----------+ +-------+ RTP Mixer +-----+ | +-----------+ | | | +--------+-------+ +-------+--------+ | First RTP | | Third RTP | | implementation | | implementation | +----------------+ +----------------+ Figure 3: Testing architecture for mixers o the CC field in the RTP packets received by the third implementation is set to 2 o the RTP packets received by the third implementation contain 2 CSRCs corresponding to the first and second RTP implementations o the RTP packets received by the third implementation contain an SSRC corresponding to that of the mixer It should next be verified that the mixer generates SR and RR packets for each cloud. The mixer should generate RR packets in the direction of the first and second implementations, and SR packets in the direction of the third implementation. It should be verified that the SR packets sent to the third implmentation do not reference the first or second implementations, and vice versa. It should be verified that SDES CNAME information is forwarded across the mixer. Other SDES fields may optionally be forwarded. Finally, one of the implementations should be quit, and it should be verified that the other implementations see the BYE packet. This implementation should then be restarted and the mixer should be quit. It should be verified that the implementations see both the mixer and the implementations on the other side of the mixer quit (illustrating response to BYE packets containing multiple sources). 5 SSRC collision detection See draft-ietf-avt-rtcptest-01.txt Perkins Page 8 INTERNET-DRAFT 25 June 1999 6 Loop detection To be completed. 7 Encrypted RTP To be completed. 8 Author's Address Colin Perkins Department of Computer Science University College London Gower Street London WC1E 6BT United Kingdom Email: c.perkins@cs.ucl.ac.uk 9 References [1] S. Bradner, ``The Internet Standards Process -- Revision 3'', RFC2026, Internet Engineering Task Force, October 1996. [2] H. Schulzrinne, S. Casner, R. Frederick and V. Jacobson, ``RTP: A Transport Protocol to Real-Time Applications'', RFC1889, Internet Engineering Task Force, January 1996. Perkins Page 9