idnits 2.17.1 draft-ietf-avt-rtp-midi-guidelines-15.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** It looks like you're using RFC 3978 boilerplate. You should update this to the boilerplate described in the IETF Trust License Policy document (see https://trustee.ietf.org/license-info), which is required now. -- Found old boilerplate from RFC 3978, Section 5.1 on line 15. -- Found old boilerplate from RFC 3978, Section 5.5 on line 1617. -- Found old boilerplate from RFC 3979, Section 5, paragraph 1 on line 1589. -- Found old boilerplate from RFC 3979, Section 5, paragraph 2 on line 1596. -- Found old boilerplate from RFC 3979, Section 5, paragraph 3 on line 1602. ** This document has an original RFC 3978 Section 5.4 Copyright Line, instead of the newer IETF Trust Copyright according to RFC 4748. ** This document has an original RFC 3978 Section 5.5 Disclaimer, instead of the newer disclaimer which includes the IETF Trust according to RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- == There are 2 instances of lines with non-RFC2606-compliant FQDNs in the document. -- The document has examples using IPv4 documentation addresses according to RFC6890, but does not use any IPv6 documentation addresses. Maybe there should be IPv6 examples, too? Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year == Line 697 has weird spacing: '...ed char uint8...' == Line 699 has weird spacing: '...ed long uint3...' == Line 1165 has weird spacing: '...ed char uint8...' == Line 1167 has weird spacing: '...ed long uint3...' -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- Couldn't find a document date in the document -- date freshness check skipped. -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Missing reference section? 'RTPMIDI' on line 1658 looks like a reference -- Missing reference section? 'RFC3550' on line 1519 looks like a reference -- Missing reference section? 'MIDI' on line 1526 looks like a reference -- Missing reference section? 'RFC3551' on line 1523 looks like a reference -- Missing reference section? 'RTPBOOK' on line 1559 looks like a reference -- Missing reference section? 'NMP' on line 1541 looks like a reference -- Missing reference section? 'SDP' on line 1529 looks like a reference -- Missing reference section? 'MPEGSA' on line 1532 looks like a reference -- Missing reference section? 'RFC3556' on line 1535 looks like a reference -- Missing reference section? 'STEVENS' on line 1648 looks like a reference -- Missing reference section? 'RFC3261' on line 1546 looks like a reference -- Missing reference section? 'UDPMAXSIZE' on line 431 looks like a reference -- Missing reference section? 'GRAME' on line 1550 looks like a reference -- Missing reference section? '2' on line 1643 looks like a reference -- Missing reference section? '274' on line 719 looks like a reference -- Missing reference section? '128' on line 1203 looks like a reference -- Missing reference section? '257' on line 736 looks like a reference -- Missing reference section? '3' on line 1653 looks like a reference -- Missing reference section? '16' on line 1245 looks like a reference -- Missing reference section? 'CCRMA' on line 1554 looks like a reference -- Missing reference section? '1' on line 1633 looks like a reference -- Missing reference section? '4' on line 1658 looks like a reference Summary: 3 errors (**), 0 flaws (~~), 7 warnings (==), 31 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 INTERNET-DRAFT J. Lazzaro 3 January 23, 2006 J. Wawrzynek 4 Expires: July 23, 2006 UC Berkeley 6 An Implementation Guide for RTP MIDI 8 10 Status of this Memo 12 By submitting this Internet-Draft, each author represents that any 13 applicable patent or other IPR claims of which he or she is aware have 14 been or will be disclosed, and any of which he or she becomes aware 15 will be disclosed, in accordance with Section 6 of BCP 79. 17 Internet-Drafts are working documents of the Internet Engineering 18 Task Force (IETF), its areas, and its working groups. Note that other 19 groups may also distribute working documents as Internet-Drafts. 21 Internet-Drafts are draft documents valid for a maximum of six months 22 and may be updated, replaced, or obsoleted by other documents at any 23 time. It is inappropriate to use Internet-Drafts as reference 24 material or to cite them other than as "work in progress." 26 The list of current Internet-Drafts can be accessed at 27 http://www.ietf.org/1id-abstracts.txt. 29 The list of Internet-Draft Shadow Directories can be accessed at 30 http://www.ietf.org/shadow.html. 32 This Internet-Draft will expire on July 23, 2006. 34 Abstract 36 This memo offers non-normative implementation guidance for the 37 RTP MIDI payload format. The memo presents its advice in the 38 context of a network musical performance application. In this 39 application two musicians, located in different physical locations, 40 interact over a network to perform as they would if located in the 41 same room. Underlying the performances are RTP MIDI sessions over 42 unicast UDP. Algorithms for sending and receiving recovery 43 journals (the resiliency structure for the payload format) are 44 described in detail. Although the memo focuses on network musical 45 performance, the presented implementation advice is relevant to 46 other RTP MIDI applications. 48 Table of Contents 50 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 4 51 2. Starting the Session . . . . . . . . . . . . . . . . . . . . . . 5 52 3. Session Management: Session Housekeeping . . . . . . . . . . . . 8 53 4. Sending Streams: General Considerations . . . . . . . . . . . . . 9 54 4.1 Queuing and Coding Incoming MIDI Data . . . . . . . . . . . 13 55 4.2 Sending Packets with Empty MIDI Lists . . . . . . . . . . . 14 56 4.3 Congestion Control and Bandwidth Management . . . . . . . . 16 57 5. Sending Streams: The Recovery Journal . . . . . . . . . . . . . . 16 58 5.1 Initializing the RJSS . . . . . . . . . . . . . . . . . . . 18 59 5.2 Traversing the RJSS . . . . . . . . . . . . . . . . . . . . 21 60 5.3 Updating the RJSS . . . . . . . . . . . . . . . . . . . . . 21 61 5.4 Trimming the RJSS . . . . . . . . . . . . . . . . . . . . . 22 62 5.5 Implementation Notes . . . . . . . . . . . . . . . . . . . 23 63 6. Receiving Streams: General Considerations . . . . . . . . . . . . 23 64 6.1 The NMP Receiver Design . . . . . . . . . . . . . . . . . . 24 65 6.2 High-Jitter Networks, Local Area Networks . . . . . . . . . 26 66 7. Receiving Streams: The Recovery Journal . . . . . . . . . . . . . 27 67 7.1 Chapter W: MIDI Pitch Wheel (0xE) . . . . . . . . . . . . . 32 68 7.2 Chapter N: MIDI NoteOn (0x8) and NoteOff (0x9) . . . . . . 32 69 7.3 Chapter C: MIDI Control Change (0xB) . . . . . . . . . . . 34 70 7.4 Chapter P: MIDI Program Change (0xC) . . . . . . . . . . . 36 71 8. Security Considerations . . . . . . . . . . . . . . . . . . . . . 37 72 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . . 37 73 A. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . 37 74 B. References . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 75 B.1 Normative References . . . . . . . . . . . . . . . . . . . 37 76 B.2 Informative References . . . . . . . . . . . . . . . . . . 38 77 C. Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 38 78 D. Intellectual Property Rights Statement . . . . . . . . . . . . . 39 79 E. Full Copyright Statement . . . . . . . . . . . . . . . . . . . . 39 80 F. Change Log . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 81 1. Introduction 83 [RTPMIDI] normatively defines a Real Time Protocol (RTP, [RFC3550]) 84 payload format for the MIDI command language [MIDI], for use under any 85 applicable RTP profile (such as the Audio/Visual Profile (AVP, 86 [RFC3551])). 88 However, [RTPMIDI] does not define algorithms for sending and receiving 89 MIDI streams. Implementors are free to use any sending or receiving 90 algorithm that conforms to the normative text in [RTPMIDI] [RFC3550] 91 [RFC3551] [MIDI]. 93 In this memo, we offer implementation guidance on sending and receiving 94 MIDI RTP streams. Unlike [RTPMIDI], this memo is not normative. 96 RTP is a mature protocol, and excellent RTP reference materials are 97 available [RTPBOOK]. This memo aims to complement the existing 98 literature, by focusing on issues that are specific to the MIDI payload 99 format. 101 The memo focuses on one application: two-party network musical 102 performance over wide-area networks, following the interoperability 103 guidelines in Appendix C.7.2 of [RTPMIDI]. Underlying the performances 104 are RTP MIDI sessions over unicast UDP transport. Resiliency is 105 provided by the recovery journal system [RTPMIDI]. The application also 106 uses the RTP control protocol (RTCP, [RFC3550]). 108 The application targets a network with a particular set of 109 characteristics: low nominal jitter, low packet loss, and occasional 110 outlier packets that arrive very late. However, in Section 6.2 of this 111 memo we discuss adapting the application to other network environments. 113 As defined in [NMP], a network musical performance occurs when a group 114 of musicians, located at different physical locations, interact over a 115 network to perform as they would if located in the same room. 117 Sections 2-3 of this memo describe session startup and maintenance. 118 Sections 4-5 cover sending MIDI streams, and Sections 6-7 cover 119 receiving MIDI streams. 121 2. Starting the Session 123 In this section, we describe how the application starts a two-player 124 session. We assume that the two parties have agreed on a session 125 configuration, embodied by a pair of Session Description Protocol (SDP, 126 [SDP]) session descriptions. 128 One session description (Figure 1) defines how the first party wishes to 129 receive its stream. The other session description (Figure 2) defines 130 how the second party wishes to receive its stream. 132 The session description in Figure 1 codes that the first party intends 133 to receive a MIDI stream on IP4 number 192.0.2.94 (coded in the c= line) 134 at UDP port 16112 (coded in the m= line). Implicit in the SDP m= line 135 syntax [SDP] is that the first party also intends to receive an RTCP 136 stream on 192.0.2.94 at UDP port 16113 (16112 + 1). The receiver 137 expects that the PT field of each RTP header in the received stream will 138 be set to 96 (coded in the m= line). 140 Likewise, the session description in Figure 2 codes that the second 141 party intends to receive a MIDI stream on IP4 number 192.0.2.105 at UDP 142 port 5004, and intends to receive an RTCP stream on 192.0.2.105 at UDP 143 port 5005 (5004 + 1). The second party expects that the PT RTP header 144 field of received stream will be set to 101. 146 v=0 147 o=first 2520644554 2838152170 IN IP4 first.example.net 148 s=Example 149 t=0 0 150 c=IN IP4 192.0.2.94 151 m=audio 16112 RTP/AVP 96 152 b=AS:20 153 b=RS:0 154 b=RR:400 155 a=rtpmap:96 mpeg4-generic/44100 156 a=fmtp:96 streamtype=5; mode=rtp-midi; config=""; profile-level-id=12; 157 cm_unused=ABFGHJKMQTVXYZ; cm_unused=C120-127; ch_never=ADEFMQTVX; 158 tsmode=buffer; linerate=320000; octpos=last; mperiod=44; rtp_ptime=0; 159 rtp_maxptime=0; guardtime=44100; render=synthetic; rinit="audio/asc"; 160 url="http://example.net/sa.asc"; 161 cid="xjflsoeiurvpa09itnvlduihgnvet98pa3w9utnuighbuk" 163 (The a=fmtp line has been wrapped to fit the page to accommodate 164 memo formatting restrictions; it comprises a single line in SDP) 166 Figure 1 -- Session description for first participant. 168 v=0 169 o=second 2520644554 2838152170 IN IP4 second.example.net 170 s=Example 171 t=0 0 172 c=IN IP4 192.0.2.105 173 m=audio 5004 RTP/AVP 101 174 b=AS:20 175 b=RS:0 176 b=RR:400 177 a=rtpmap:101 mpeg4-generic/44100 178 a=fmtp:101 streamtype=5; mode=rtp-midi; config=""; profile-level-id=12; 179 cm_unused=ABFGHJKMQTVXYZ; cm_unused=C120-127; ch_never=ADEFMQTVX; 180 tsmode=buffer; linerate=320000;octpos=last;mperiod=44; guardtime=44100; 181 rtp_ptime=0; rtp_maxptime=0; render=synthetic; rinit="audio/asc"; 182 url="http://example.net/sa.asc"; 183 cid="xjflsoeiurvpa09itnvlduihgnvet98pa3w9utnuighbuk" 185 (The a=fmtp line has been wrapped to fit the page to accommodate 186 memo formatting restrictions; it comprises a single line in SDP) 188 Figure 2 -- Session description for second participant. 190 The session descriptions use the mpeg4-generic media type (coded in the 191 a=rtpmap line) to specify the use of the MPEG 4 Structured Audio 192 renderer [MPEGSA]. The session descriptions also use parameters to 193 customize the stream (Appendix C of [RTPMIDI]). The parameter values 194 are identical for both parties, yielding identical rendering 195 environments for the two client hosts. 197 The bandwidth (b=) AS parameter [SDP] [RFC3550] indicates that the total 198 RTP session bandwidth is 20 kbs. This value assumes that the two 199 players send 10 kbs streams concurrently. To derive the 10 kbs value, 200 we begin with the analysis of RTP MIDI payload bandwidth in Appendix A.4 201 of [NMP], and add in RTP and IP4 packet overhead and a small safety 202 factor. 204 The bandwidth RR parameter [RFC3556] indicates that the shared RTCP 205 session bandwidth for the two parties is 400 bps. We set the bandwidth 206 SR parameter to 0 bps, to signal that sending parties and non-sending 207 parties equally share the 400 bps of RTCP bandwidth (note that in this 208 particular example, the guardtime parameter value of 44100 ensures that 209 both parties are sending for the duration of the session). The 400 bps 210 RTCP bandwidth value supports one RTCP packet per 5 seconds from each 211 party, containing a Sender Report and CNAME information [RFC3550]. 213 We now show example code that implements the actions the parties take 214 during the session. The code is written in C, and uses standard network 215 programming techniques described in [STEVENS]. We show code for the 216 first party (the second party takes a symmetric set of actions). 218 Figure 3 shows how the first party initializes a pair of socket 219 descriptors (rtp_fd and rtcp_fd) to send and receive UDP packets. After 220 the code in Figure 3 runs, the first party may check for new RTP or RTCP 221 packets by calling recv() on rtp_fd or rtcp_fd. 223 Applications may use recv() to receive UDP packets on a socket using one 224 of two general methods: "blocking" or "non-blocking". 226 A call to recv() on a blocking UDP socket puts the calling thread to 227 sleep until a new packet arrives. 229 A call to recv() on a non-blocking socket acts to poll the device: the 230 recv() call returns immediately, with a return value that indicates the 231 polling result. In this case, a positive return value signals the size 232 of a new received packet, and a negative return value (coupled with an 233 errno value of EAGAIN) indicates no new packet was available. 235 The choice of blocking or non-blocking sockets is a critical application 236 choice. Blocking sockets offer the lowest potential latency (as the OS 237 wakes the caller as soon as a packet has arrived). However, audio 238 applications that use blocking sockets must adopt a multi-threaded 239 program architecture, so that audio samples may be generated on a 240 "rendering thread" while the "network thread" sleeps while awaiting the 241 next packet. The architecture must also support a thread communication 242 mechanism, so that the network thread has a mechanism to send MIDI 243 commands the rendering thread. 245 In contrast, audio applications that use non-blocking sockets may be 246 coded using a single thread, that alternatives between audio sample 247 generation and network polling. This architecture trades off increased 248 network latency (as a packet may arrive between polls) for a simpler 249 program architecture. For simplicity, our example uses non-blocking 250 sockets and presumes a single run loop. Figure 4 shows how the example 251 configures its sockets to be non-blocking. 253 Figure 5 shows how to use recv() to check a non-blocking socket for new 254 packets. 256 The first party also uses rtp_fd and rtcp_fd to send RTP and RTCP 257 packets to the second party. In Figure 6, we show how to initialize 258 socket structures that address the second party. In Figure 7, we show 259 how to use one of these structures in a sendto() call to send an RTP 260 packet to the second party. 262 Note that the code shown in Figures 3-7 assumes a clear network path 263 between the participants. The code may not work if firewalls or Network 264 Address Translation (NAT) devices are present in the network path. 266 3. Session Management: Session Housekeeping 268 After the two-party interactive session is set up, the parties begin to 269 send and receive RTP packets. In Sections 4-7, we discuss RTP MIDI 270 sending and receiving algorithms. In this section, we describe session 271 "housekeeping" tasks that the participants also perform. 273 One housekeeping task is the maintenance of the 32-bit SSRC value that 274 uniquely identifies each party. Section 8 of [RFC3550] describes SSRC 275 issues in detail, as does Section 2.1 in [RTPMIDI]. Another 276 housekeeping task is the sending and receiving of RTCP. Section 6 of 277 [RFC3550] describes RTCP in detail. 279 Another housekeeping task concerns security. As detailed in Appendix G 280 of [RTPMIDI], per-packet authentication is strongly recommended for use 281 with MIDI streams, because the acceptance of rogue packets may lead to 282 the execution of arbitrary MIDI commands. 284 A final housekeeping task concerns the termination of the session. In 285 our two-party example, the session terminates upon the exit of one of 286 the participants. A clean termination may require active effort by a 287 receiver, as a MIDI stream stopped at an arbitrary point may cause stuck 288 notes and other indefinite artifacts in the MIDI renderer. 290 The exit of a party may be signalled in several ways. Session 291 management tools may offer a reliable signal for termination (such as 292 the SIP BYE method [RFC3261]). The (unreliable) RTCP BYE packet 293 [RFC3550] may also signal the exit of a party. Receivers may also sense 294 the lack of RTCP activity and timeout a party, or may use transport 295 methods to detect an exit. 297 4. Sending Streams: General Considerations 299 In this section, we discuss sender implementation issues. 301 The sender is a real-time data-driven entity. On an on-going basis, the 302 sender checks to see if the local player has generated new MIDI data. 303 At any time, the sender may transmit a new RTP packet to the remote 304 player, for the reasons described below: 306 1. New MIDI data has been generated by the local player, and the 307 sender decides it is time to issue a packet coding the data. 309 2. The local player has not generated new MIDI data, but the 310 sender decides too much time has elapsed since the last 311 RTP packet transmission. The sender transmits a packet in 312 order to relay updated header and recovery journal data. 314 In both cases, the sender generates a packet that consists of an RTP 315 header, a MIDI command section, and a recovery journal. In the first 316 case, the MIDI list of the MIDI command section codes the new MIDI data. 317 In the second case, the MIDI list is empty. 319 #include 320 #include 321 #include 323 int rtp_fd, rtcp_fd; /* socket descriptors */ 324 struct sockaddr_in addr; /* for bind address */ 326 /*********************************/ 327 /* create the socket descriptors */ 328 /*********************************/ 330 if ((rtp_fd = socket(AF_INET, SOCK_DGRAM, 0)) < 0) 331 ERROR_RETURN("Couldn't create Internet RTP socket"); 333 if ((rtcp_fd = socket(AF_INET, SOCK_DGRAM, 0)) < 0) 334 ERROR_RETURN("Couldn't create Internet RTCP socket"); 336 /**********************************/ 337 /* bind the RTP socket descriptor */ 338 /**********************************/ 340 memset(&(addr.sin_zero), 0, 8); 341 addr.sin_family = AF_INET; 342 addr.sin_addr.s_addr = htonl(INADDR_ANY); 343 addr.sin_port = htons(16112); /* port 16112, from SDP */ 345 if (bind(rtp_fd, (struct sockaddr *)&addr, 346 sizeof(struct sockaddr)) < 0) 347 ERROR_RETURN("Couldn't bind Internet RTP socket"); 349 /***********************************/ 350 /* bind the RTCP socket descriptor */ 351 /***********************************/ 353 memset(&(addr.sin_zero), 0, 8); 354 addr.sin_family = AF_INET; 355 addr.sin_addr.s_addr = htonl(INADDR_ANY); 356 addr.sin_port = htons(16113); /* port 16113, from SDP */ 358 if (bind(rtcp_fd, (struct sockaddr *)&addr, 359 sizeof(struct sockaddr)) < 0) 360 ERROR_RETURN("Couldn't bind Internet RTCP socket"); 362 Figure 3 -- Setup code for listening for RTP/RTCP packets. 364 #include 365 #include 367 /***************************/ 368 /* set non-blocking status */ 369 /***************************/ 371 if (fcntl(rtp_fd, F_SETFL, O_NONBLOCK)) 372 ERROR_RETURN("Couldn't unblock Internet RTP socket"); 374 if (fcntl(rtcp_fd, F_SETFL, O_NONBLOCK)) 375 ERROR_RETURN("Couldn't unblock Internet RTCP socket"); 377 Figure 4 -- Code to set socket descriptors to be non-blocking. 379 #include 380 #define UDPMAXSIZE 1472 /* based on Ethernet MTU of 1500 */ 382 unsigned char packet[UDPMAXSIZE+1]; 383 int len, normal; 385 while ((len = recv(rtp_fd, packet, UDPMAXSIZE + 1, 0)) > 0) 386 { 387 /* process packet[]. If (len == UDPMAXSIZE + 1), recv() 388 * may be returning a truncated packet -- process with care 389 */ 390 } 392 /* line below sets "normal" to 1 if the recv() return */ 393 /* status indicates no packets are left to process */ 395 normal = (len < 0) && (errno == EAGAIN); 397 if (!normal) 398 { 399 /* 400 * recv() return status indicates an empty UDP payload 401 * (len == 0) or an error condition (coded by (len < 0) 402 * and (errno != EAGAIN)). Examine len and errno, and 403 * take appropriate recovery action. 404 */ 405 } 407 Figure 5 -- Code to check rtp_fd for new RTP packets. 409 #include 410 #include 412 struct sockaddr_in * rtp_addr; /* RTP destination IP/port */ 413 struct sockaddr_in * rtcp_addr; /* RTCP destination IP/port */ 415 /* set RTP address, as coded in Figure 2's SDP */ 417 rtp_addr = calloc(1, sizeof(struct sockaddr_in)); 418 rtp_addr->sin_family = AF_INET; 419 rtp_addr->sin_port = htons(5004); 420 rtp_addr->sin_addr.s_addr = inet_addr("192.0.2.105"); 422 /* set RTCP address, as coded in Figure 2's SDP */ 424 rtcp_addr = calloc(1, sizeof(struct sockaddr_in)); 425 rtcp_addr->sin_family = AF_INET; 426 rtcp_addr->sin_port = htons(5005); /* 5004 + 1 */ 427 rtcp_addr->sin_addr.s_addr = rtp_addr->sin_addr.s_addr; 429 Figure 6 -- Initializing destination addresses for RTP and RTCP. 431 unsigned char packet[UDPMAXSIZE]; /* RTP packet to send */ 432 int size; /* length of RTP packet */ 434 /* first fill packet[] and set size ... then: */ 436 if (sendto(rtp_fd, packet, size, 0, rtp_addr, 437 sizeof(struct sockaddr)) == -1) 438 { 439 /* 440 * try again later if errno == EAGAIN or EINTR 441 * 442 * other errno values --> an operational error 443 */ 444 } 446 Figure 7 -- Using sendto() to send an RTP packet. 448 Figure 8 shows the 5 steps a sender takes to issue a packet. This 449 algorithm corresponds to the code fragment for sending RTP packets shown 450 in Figure 7 of Section 2. Steps 1, 2, and 3 occur before the sendto() 451 call in the code fragment. Step 4 corresponds to the sendto() call 452 itself. Step 5 may occur once Step 3 completes. 454 Algorithm for Sending a Packet: 456 1. Generate the RTP header for the new packet. See Section 2.1 457 of [RTPMIDI] for details. 459 2. Generate the MIDI command section for the new packet. See 460 Section 3 of [RTPMIDI] for details. 462 3. Generate the recovery journal for the new packet. We discuss 463 this process in Section 5.2. The generation algorithm examines 464 the Recovery Journal Sending Structure (RJSS), a stateful 465 coding of a history of the stream. 467 4. Send the new packet to the receiver. 469 5. Update the RJSS to include the data coded in the MIDI command 470 section of the packet sent in step 4. We discuss the update 471 procedure in Section 5.3. 473 Figure 8 -- A 5 step algorithm for sending a packet. 475 In the sections that follow, we discuss specific sender implementation 476 issues in detail. 478 4.1 Queuing and Coding Incoming MIDI Data 480 Simple senders transmit a new packet as soon as the local player 481 generates a complete MIDI command. The system described in [NMP] uses 482 this algorithm. This algorithm minimizes the sender queuing latency, as 483 the sender never delays the transmission of a new MIDI command. 485 In a relative sense, this algorithm uses bandwidth inefficiently, as it 486 does not amortize the overhead of a packet over several commands. This 487 inefficiency may be acceptable for sparse MIDI streams (see Appendix A.4 488 of [NMP]). More sophisticated sending algorithms [GRAME] improve 489 efficiency by coding small groups of commands into a single packet, at 490 the expense of increasing the sender queuing latency. 492 Senders assign a timestamp value to each command issued by the local 493 player (Appendix C.3 of [RTPMIDI]). Senders may code the timestamp 494 value of the first MIDI list command in two ways. The most efficient 495 method is to set the RTP timestamp of the packet to the timestamp value 496 of the first command. In this method, the Z bit of the MIDI command 497 section header (Figure 2 of [RTPMIDI]) is set to 0, and the RTP 498 timestamps increment at a non-uniform rate. 500 However, in some applications, senders may wish to generate a stream 501 whose RTP timestamps increment at a uniform rate. To do so, senders may 502 use the Delta Time MIDI list field to code a timestamp for the first 503 command in the list. In this case, the Z bit of the MIDI command 504 section header is set to 1. 506 Senders should strive to maintain a constant relationship between the 507 RTP packet timestamp and the packet sending time: if two packets have 508 RTP timestamps that differ by 1 second, the second packet should be sent 509 1 second after the first packet. To the receiver, variance in this 510 relationship is indistinguishable from network jitter. Latency issues 511 are discussed in detail in Section 6. 513 Senders may alter the running status coding of the first command in the 514 MIDI list, in order to comply with the coding rules defined in Section 515 3.2 of [RTPMIDI]. The P header bit (Figure 2 of [RTPMIDI]) codes this 516 alteration of the source command stream. 518 4.2 Sending Packets with Empty MIDI Lists 520 During a session, musicians might refrain from generating MIDI data for 521 extended periods of time (seconds or even minutes). If an RTP stream 522 followed the dynamics of a silent MIDI source, and stopped sending RTP 523 packets, system behavior might be degraded in the following ways: 525 o The receiver's model of network performance may fall out 526 of date. 528 o Network middleboxes (such as Network Address Translators) 529 may "time-out" the silent stream and drop the port and IP 530 association state. 532 o If the session does not use RTCP, receivers may misinterpret 533 the silent stream as a dropped network connection. 535 Senders avoid these problems by sending "keep-alive" RTP packets during 536 periods of network inactivity. Keep-alive packets have empty MIDI 537 lists. 539 Session participants may specify the frequency of keep-alive packets 540 during session configuration with the MIME parameter "guardtime" 541 (Appendix C.4.2 of [RTPMIDI]). The session descriptions shown in 542 Figures 1-2 use guardtime to specify a keep-alive sending interval of 1 543 second. 545 Senders may also send empty packets to improve the performance of the 546 recovery journal system. As we describe in Section 6, the recovery 547 process begins when a receiver detects a break in the RTP sequence 548 number pattern of the stream. The receiver uses the recovery journal of 549 the break packet to guide corrective rendering actions, such as ending 550 stuck notes and updating out-of-date controller values. 552 Consider the situation where the local player produces a MIDI NoteOff 553 command (which the sender promptly transmits in a packet), but then 5 554 seconds pass before the player produces another MIDI command (which the 555 sender transmits in a second packet). If the packet coding the NoteOff 556 is lost, the receiver is not be aware of the packet loss incident for 5 557 seconds, and the rendered MIDI performance contains a note that sounds 558 for 5 seconds too long. 560 To handle this situation, senders may transmit empty packets to "guard" 561 the stream during silent sections. The guard packet algorithm defined 562 in Section 7.3 of [NMP], as applied to the situation described above, 563 sends a guard packet after 100 ms of player inactivity, and sends a 564 second guard packet 100 ms later. Subsequent guard packets are sent 565 with an exponential backoff, with a limiting period of 1 second (set by 566 the "guardtime" parameter in Figures 1-2). The algorithm terminates 567 once MIDI activity resumes, or once RTCP receiver reports indicate that 568 the receiver is up to date. 570 The perceptual quality of guard packet sending algorithms is a quality 571 of implementation issue for RTP MIDI applications. Sophisticated 572 implementations may tailor the guard packet sending rate to the nature 573 of the MIDI commands recently sent in the stream, to minimize the 574 perceptual impact of moderate packet loss. 576 As an example of this sort of specialization, the guard packet algorithm 577 described in [NMP] protects against the transient artifacts that occur 578 when NoteOn commands are lost. The algorithm sends a guard packet 1 ms 579 after every packet whose MIDI list contains a NoteOn command. The Y bit 580 in Chapter N note logs (Appendix A.6 of [RTPMIDI]) supports this use of 581 guard packets. 583 Congestion control and bandwidth management are key issues in guard 584 packet algorithms. We discuss these issues in the next section. 586 4.3 Congestion Control and Bandwidth Management 588 The congestion control section of [RTPMIDI] discusses the importance of 589 congestion control for RTP MIDI streams, and references the normative 590 text in [RFC3550] and [RFC3551] that concerns congestion control. To 591 comply the requirements described in those normative documents, RTP MIDI 592 senders may use several methods to control the sending rate: 594 o As described in Section 4.1, senders may pack several MIDI 595 commands into a single packet, thereby reducing stream bandwidth 596 (at the expense of increasing sender queuing latency). 598 o Guard packet algorithms (Section 4.2) may be designed in 599 a parametric way, so that the tradeoff between artifact 600 reduction and stream bandwidth may be tuned dynamically. 602 o The recovery journal size may be reduced, by adapting the 603 techniques described in Section 5 of this memo. Note that 604 in all cases, the recovery journal sender must conform to 605 the normative text in Section 4 of [RTPMIDI]. 607 o The incoming MIDI stream may be modified, to reduce the 608 number of MIDI commands without significantly altering the 609 performance. Lossy "MIDI filtering" algorithms are well 610 developed in the MIDI community, and may be directly applied 611 to RTP MIDI rate management. 613 RTP MIDI senders incorporate these rate control methods into feedback 614 systems to implement congestion control and bandwidth management. 615 Sections 10 and 6.4.4 of [RFC3550] and Section 2 in [RFC3551] describe 616 feedback systems for congestion control in RTP, and Section 6 of [SDP] 617 describes bandwidth management in media sessions. 619 5. Sending Streams: The Recovery Journal 621 In this section, we describe how senders implement the recovery journal 622 system. The implementation we describe uses the default "closed-loop" 623 recovery journal semantics (Appendix C.2.2.2 of [RTPMIDI]). 625 We begin by describing the Recovery Journal Sending Structure (RJSS). 626 Senders use the RJSS to generate the recovery journal section for RTP 627 MIDI packets. 629 The RJSS is a hierarchical representation of the checkpoint history of 630 the stream. The checkpoint history holds the MIDI commands that are at 631 risk to packet loss (Appendix A.1 of [RTPMIDI] precisely defines the 632 checkpoint history). The layout of the RJSS mirrors the hierarchical 633 structure of the recovery journal bitfields. 635 Figure 9 shows a RJSS implementation for a simple sender. The leaf 636 level of the RJSS hierarchy (the jsend_chapter structures) corresponds 637 to channel chapters (Appendices A.2-9 in [RTPMIDI]). The second level 638 of the hierarchy (jsend_channel) corresponds to the channel journal 639 header (Figure 9 in [RTPMIDI]). The top level of the hierarchy 640 (jsend_journal) corresponds to the recovery journal header (Figure 8 in 641 [RTPMIDI]). 643 Each RJSS data structure may code several items: 645 1. The current contents of the recovery journal bitfield 646 associated with the RJSS structure (jheader[], cheader[], 647 or a chapter bitfield). 649 2. A seqnum variable. Seqnum codes the extended RTP sequence 650 number of the most recent packet that added information to the 651 RJSS structure. If the seqnum of a structure is updated, the 652 seqnums of all structures above it in the recovery journal 653 hierarchy are also updated. Thus, a packet that caused an 654 update to a specific jsend_chapter structure would update the 655 seqnum values of this structure and of the jsend_channel and 656 jsend_journal structures that contain it. 658 3. Ancillary variables used by the sending algorithm. 660 A seqnum variable is set to zero if the checkpoint history contains no 661 information at the level or at any level below it. This coding scheme 662 assumes that the first sequence number of a stream is normalized to 1, 663 and limits the total number of stream packets to 2^32 - 1. 665 The cm_unused and ch_never parameters in Figures 1-2 define the subset 666 of MIDI commands supported by the sender (see Appendix C.2.3 of 667 [RTPMIDI] for details). The sender transmits most voice commands, but 668 does not transmit system commands. The sender assumes the MIDI source 669 uses note commands in the typical way, and does not use the Chapter E 670 note resiliency tools (Appendix A.7 of [RTPMIDI]). The sender does not 671 support Control Change commands for controller numbers with All Notes 672 Off (123-127), All Sound Off (120), and Reset All Controllers (121) 673 semantics, and does not support enhanced Chapter C encoding (Appendix 674 A.3.3 of [RTPMIDI]). 676 We chose this subset of MIDI commands to simplify the example. In 677 particular, the command restrictions ensure that all commands are 678 active, all note commands are N-active, and all Control Change commands 679 are C-active (see Appendix A.1 of [RTPMIDI] for definitions of active, 680 N-active, and C-active). 682 In the sections that follow, we describe the tasks a sender performs to 683 manage the recovery journal system. 685 5.1 Initializing the RJSS 687 At the start of a stream, the sender initializes the RJSS. All seqnum 688 variables are set to zero, including all elements of note_seqnum[] and 689 control_seqnum[]. 691 The sender initializes jheader[] to form a recovery journal header that 692 codes an empty journal. The S bit of the header is set to 1, and the A, 693 Y, R, and TOTCHAN header fields are set to zero. The checkpoint packet 694 sequence number field is set to the sequence number of the upcoming 695 first RTP packet (per Appendix A.1 of [RTPMIDI]). 697 typedef unsigned char uint8; /* must be 1 octet */ 698 typedef unsigned short uint16; /* must be 2 octet */ 699 typedef unsigned long uint32; /* must be 4 octets */ 701 /**************************************************************/ 702 /* leaf level hierarchy: Chapter W, Appendix A.5 of [RTPMIDI] */ 703 /**************************************************************/ 705 typedef struct jsend_chapterw { /* Pitch Wheel (0xE) */ 706 uint8 chapterw[2]; /* bitfield Figure A.5.1 [RTPMIDI] */ 707 uint32 seqnum; /* extended sequence number, or 0 */ 708 } jsend_chapterw; 710 /**************************************************************/ 711 /* leaf level hierarchy: Chapter N, Appendix A.6 of [RTPMIDI] */ 712 /**************************************************************/ 714 typedef struct jsend_chaptern { /* Note commands (0x8, 0x9) */ 716 /* chapter N maximum size is 274 octets: a 2 octet header, */ 717 /* and a maximum of 128 2-octet logs and 16 OFFBIT octets */ 719 uint8 chaptern[274]; /* bitfield Figure A.6.1 [RTPMIDI] */ 720 uint16 size; /* actual size of chaptern[] */ 721 uint32 seqnum; /* extended seq number, or 0 */ 722 uint32 note_seqnum[128]; /* most recent note seqnum, or 0 */ 723 uint32 note_tstamp[128]; /* NoteOn execution timestamp */ 724 uint32 bitfield_ptr[128]; /* points to a chapter log, or 0 */ 725 } jsend_chaptern; 727 /**************************************************************/ 728 /* leaf level hierarchy: Chapter C, Appendix A.3 of [RTPMIDI] */ 729 /**************************************************************/ 731 typedef struct jsend_chapterc { /* Control Change (0xB) */ 733 /* chapter C maximum size is 257 octets: a 1 octet header */ 734 /* and a maximum of 128 2-octet logs */ 736 uint8 chapterc[257]; /* bitfield Figure A.3.1 [RTPMIDI] */ 737 uint16 size; /* actual size of chapterc[] */ 738 uint32 seqnum; /* extended sequence number, or 0 */ 739 uint32 control_seqnum[128]; /* most recent seqnum, or 0 */ 740 uint32 bitfield_ptr[128]; /* points to a chapter log, or 0 */ 741 } jsend_chapterc; 743 Figure 9 -- Recovery Journal Sending Structure (part 1) 745 /**************************************************************/ 746 /* leaf level hierarchy: Chapter P, Appendix A.2 of [RTPMIDI] */ 747 /**************************************************************/ 749 typedef struct jsend_chapterp { /* MIDI Program Change (0xC) */ 751 uint8 chapterp[3]; /* bitfield Figure A.2.1 [RTPMIDI] */ 752 uint32 seqnum; /* extended sequence number, or 0 */ 754 } jsend_chapterp; 756 /***************************************************/ 757 /* second-level of hierarchy, for channel journals */ 758 /***************************************************/ 760 typedef struct jsend_channel { 762 uint8 cheader[3]; /* header Figure 9 [RTPMIDI]) */ 763 uint32 seqnum; /* extended sequence number, or 0 */ 765 jsend_chapterp chapterp; /* chapter P info */ 766 jsend_chapterc chapterc; /* chapter C info */ 767 jsend_chapterw chapterw; /* chapter W info */ 768 jsend_chaptern chaptern; /* chapter N info */ 770 } jsend_channel; 772 /*******************************************************/ 773 /* top level of hierarchy, for recovery journal header */ 774 /*******************************************************/ 776 typedef struct jsend_journal { 778 uint8 jheader[3]; /* header Figure 8, [RTPMIDI] */ 779 /* Note: Empty journal has a header */ 781 uint32 seqnum; /* extended sequence number, or 0 */ 782 /* seqnum = 0 codes empty journal */ 784 jsend_channel channels[16]; /* channel journal state */ 785 /* index is MIDI channel */ 787 } jsend_journal; 789 Figure 9 (continued) -- Recovery Journal Sending Structure 791 In jsend_chaptern, elements of note_tstamp[] are set to zero. In 792 jsend_chaptern and jsend_chapterc, elements of bitfield_ptr[] are set to 793 the null pointer index value (bitfield_ptr[] is an array whose elements 794 point to the first octet of the note or control log associated with the 795 array index). 797 5.2 Traversing the RJSS 799 Whenever an RTP packet is created (Step 3 in the algorithm defined in 800 Figure 8), the sender traverses the RJSS to create the recovery journal 801 for the packet. The traversal begins at the top level of the RJSS. The 802 sender copies jheader[] into the packet, and then sets the S bit of 803 jheader[] to 1. 805 The traversal continues depth-first, visiting every jsend_channel whose 806 seqnum variable is non-zero. The sender copies the cheader[] array into 807 the packet, and then sets the S bit of cheader[] to 1. After each 808 cheader[] copy, the sender visits each leaf-level chapter, in order of 809 its appearance in the chapter journal Table of Contents (first P, then 810 C, then W, then N, as shown in Figure 9 of [RTPMIDI]). 812 If a chapter has a non-zero seqnum, the sender copies the chapter 813 bitfield array into the packet, and then sets the S bit of the RJSS 814 array to 1. For chaptern[], the B bit is also set to 1. For the 815 variable-length chapters (chaptern[] and chapterc[]), the sender checks 816 the size variable to determine the bitfield length. 818 Before copying chaptern[], the sender updates the Y bit of each note log 819 to code the onset of the associated NoteOn command (Figure A.6.3 in 820 [RTPMIDI]). To determine the Y bit value, the sender checks the 821 note_tstamp[] array for note timing information. 823 5.3 Updating the RJSS 825 After an RTP packet is sent, the sender updates the RJSS to refresh the 826 checkpoint history (Step 5 in the sending algorithm defined in Figure 827 8). For each command in the MIDI list of the sent packet, the sender 828 performs the update procedure we now describe. 830 The update procedure begins at the leaf level. The sender generates a 831 new bitfield array for the chapter associated with the MIDI command, 832 using the chapter-specific semantics defined in Appendix A of [RTPMIDI]. 834 For Chapter N and Chapter C, the sender uses the bitfield_ptr[] array to 835 locate and update an existing log for a note or controller. If a log 836 does not exist, the sender adds a log to the end of the chaptern[] or 837 chapterc[] bitfield, and changes the bitfield_ptr[] value to point to 838 the log. For Chapter N, the sender also updates note_tstamp[]. 840 The sender also clears the S bit of the chapterp[], chapterw[], or 841 chapterc[] bitfield. For chaptern[], the sender clears the S bit or the 842 B bit of the bitfield, as described in Appendix A.6 of [RTPMIDI]. 844 Next, the sender refreshes the upper levels of the RJSS hierarchy. At 845 the second-level, the sender updates the cheader[] bitfield of the 846 channel associated with the command. The sender sets the S bit of 847 cheader[] to 0. If the new command forced the addition of a new chapter 848 or channel journal, the sender may also update other cheader[] fields. 849 At the top-level, the sender updates the top-level jheader[] bitfield in 850 a similar manner. 852 Finally, the sender updates the seqnum variables associated with the 853 changed bitfield arrays. The sender sets the seqnum variables to the 854 extended sequence number of the packet. 856 5.4 Trimming the RJSS 858 At regular intervals, receivers send RTCP receiver reports to the sender 859 (as described in Section 6.4.2 of [RFC3550]). These reports include the 860 extended highest sequence number received (EHSNR) field. This field 861 codes the highest sequence number that the receiver has observed from 862 the sender, extended to disambiguate sequence number rollover. 864 When the sender receives an RTCP receiver report, it runs the RJSS 865 trimming algorithm. The trimming algorithm uses the EHSNR to trim away 866 parts of the RJSS, and thus reduce the size of recovery journals sent in 867 subsequent RTP packets. The algorithm conforms to the closed-loop 868 sending policy defined in Appendix C.2.2.2 of [RTPMIDI]. 870 The trimming algorithm relies on the following observation: if the EHSNR 871 indicates that a packet with sequence number K has been received, MIDI 872 commands sent in packets with sequence numbers J <= K may be removed 873 from the RJSS without violating the closed-loop policy. 875 To begin the trimming algorithm, the sender extracts the EHSNR field 876 from the receiver report, and adjusts the EHSNR to reflect the sequence 877 number extension prefix of the sender. Then, the sender compares the 878 adjusted EHSNR value with seqnum fields at each level of the RJSS, 879 starting at the top level. 881 Levels whose seqnum is less than or equal to the adjusted EHSNR are 882 trimmed, by setting the seqnum to zero. If necessary, the jheader[] and 883 cheader[] arrays above the trimmed level are adjusted to match the new 884 journal layout. The checkpoint packet sequence number field of 885 jheader[] is updated to match the EHSNR. 887 At the leaf level, the sender trims the size of the variable-length 888 chaptern[] and chapterc[] bitfields. The sender loops through the 889 note_seqnum[] or control_seqnum[] array, and removes chaptern[] or 890 chapterc[] logs whose seqnum value is less than or equal to the adjusted 891 EHSNR. The sender sets the associated bitfield_ptr[] to null, and 892 updates the LENGTH field of the associated cheader[] bitfield. 894 Note that the trimming algorithm does not add information to the 895 checkpoint history. As a consequence, the trimming algorithm does not 896 clear the S bit (and for chaptern[], the B bit) of any recovery journal 897 bitfield. As a second consequence, the trimming algorithm does not set 898 RJSS seqnum variables to the EHSNR value. 900 5.5 Implementation Notes 902 For pedagogical purposes, the recovery journal sender we describe has 903 been simplified in several ways. In practice, an implementation would 904 use enhanced versions of the traversing, updating, and trimming 905 algorithms presented in Sections 5.2-4. 907 6. Receiving Streams: General Considerations 909 In this section, we discuss receiver implementation issues. 911 To begin, we imagine that an ideal network carries the RTP stream. 912 Packets are never lost or reordered, and the end-to-end latency is 913 constant. In addition, we assume that all commands coded in the MIDI 914 list of a packet share the same timestamp (an assumption coded by the 915 "rtp_ptime" and "rtp_maxptime" values in Figures 1-2; see Appendix C.4.1 916 of [RTPMIDI] for details). 918 Under these conditions, a simple algorithm may be used to render a high- 919 quality performance. Upon the receipt of an RTP packet, the receiver 920 immediately executes the commands coded in the MIDI command section of 921 the payload. Commands are executed in order of their appearance in the 922 MIDI list. The command timestamps are ignored. 924 Unfortunately, this simple algorithm breaks down once we relax our 925 assumptions about the network and the MIDI list: 927 1. If we permit lost and reordered packets to occur in the 928 network, the algorithm may produce unrecoverable rendering 929 artifacts, violating the mandate defined in Section 4 of [RTPMIDI]. 931 2. If we permit the network to exhibit variable latency, the 932 algorithm modulates the network jitter onto rendered MIDI 933 command stream. 935 3. If we permit a MIDI list to code commands with different 936 timestamps, the algorithm adds temporal jitter to the 937 rendered performance, as it ignores MIDI list timestamps. 939 In this section, we discuss interactive receiver design techniques under 940 these relaxed assumptions. Section 6.1 describes a receiver design for 941 high-performance Wide Area Networks (WANs), and Section 6.2 discusses 942 design issues for other types of networks. 944 6.1 The NMP Receiver Design 946 The Network Musical Performance (NMP) system [NMP] is an interactive 947 performance application that uses an early version of the RTP MIDI 948 payload format. NMP is designed for use between universities within the 949 State of California, using the high-performance CalREN2 network. 951 In the NMP system, network artifacts may affect how a musician hears the 952 performances of remote players. However, the network does not affect 953 how a musician hears his own performance. 955 Several aspects of CalREN2 network behavior (as measured in 2001 956 timeframe, as documented in [NMP]) guided the NMP system design: 958 o The median symmetric latency (1/2 the round-trip time) 959 of packets sent between network sites is comparable to the 960 acoustic latency between two musicians located in the same 961 room. For example, the latency between Berkeley and Stanford 962 is 2.1 ms, corresponding to an acoustic distance of 2.4 feet 963 (0.72 meters). These campuses are 40 miles (64 km) apart. 964 Preserving the benefits of the underlying network latency 965 at the application level was a key NMP design goal. 967 o For most times of day, the nominal temporal jitter is 968 quite short. For Berkeley-Stanford, the standard deviation 969 of the round-trip time was under 200 microseconds. 971 o For most times of day, a few percent (0-4%) of the packets 972 sent arrive significantly late (> 40 ms), probably due 973 to a queuing transient somewhere in the network path. 974 More rarely (< 0.1%), a packet is lost during the transient. 976 o At predictable times during the day (before lunchtime, 977 at the end of the workday, etc), network performance 978 deteriorates (10-20% late packets) in a manner that makes 979 the network unsuitable for low-latency interactive use. 981 o CalREN2 has deeply over-provisioned bandwidth, relative to 982 MIDI bandwidth usage. 984 The NMP sender freely uses network bandwidth to improve the performance 985 experience. As soon as a musician generates a MIDI command, an RTP 986 packet coding the command is sent to the other players. This sending 987 algorithm reduces latency at the cost of bandwidth. In addition, guard 988 packets (described in Section 4.2) are sent at frequent intervals, to 989 minimize the impact of packet loss. 991 The NMP receiver maintains a model of the stream, and uses this model as 992 the basis of its resiliency system. Upon the receipt of a packet, the 993 receiver predicts the RTP sequence number and the RTP timestamp (with 994 error bars) of the packet. Under normal network conditions, about 95% 995 of received packets fit the predictions [NMP]. In this common case, the 996 receiver immediately executes the MIDI command coded in the packet. 998 Note that the NMP receiver does not use a playout buffer; the design is 999 optimized for lowest latency at the expense of command jitter. Thus, 1000 the NMP receiver design does not completely satisfy the interoperability 1001 text in Appendix C.7.2 of [RTPMIDI], which requires that receivers in 1002 network musical performance applications be capable of using a playout 1003 buffer. 1005 Occasionally, an incoming packet fits the sequence number prediction, 1006 but falls outside the timestamp prediction error bars (see Appendix B of 1007 [NMP] for timestamp model details). In most cases, the receiver still 1008 executes the command coded in the packet. However, the receiver 1009 discards NoteOn commands with non-zero velocity. By discarding late 1010 commands that sound notes, the receiver prevents "straggler notes" from 1011 disturbing a performance. By executing all other late commands, the 1012 receiver quiets "soft stuck notes" immediately, and updates the state of 1013 the MIDI system. 1015 More rarely, an incoming packet does not fit the sequence number 1016 prediction. The receiver keeps track of the highest sequence number 1017 received in the stream, and predicts that an incoming packet will have a 1018 sequence number one greater than this value. If the sequence number of 1019 an incoming packet is greater than the prediction, a packet loss has 1020 occurred. If the sequence number of the received packet is less than 1021 the prediction, the packet has been received out of order. All sequence 1022 number calculations are modulo 2^16, and use standard methods (described 1023 in [RFC3550]) to avoid tracking errors during rollover. 1025 If a packet loss has occurred, the receiver examines the journal section 1026 of the received packet, and uses it to gracefully recover from the loss 1027 episode. We describe this recovery procedure in Section 7 of this memo. 1028 The recovery process may result in the execution of one or more MIDI 1029 commands. After executing the recovery commands, the receiver processes 1030 the MIDI command encoded in the packet, using the timestamp model test 1031 described above. 1033 If a packet is received out of order, the receiver ignores the packet. 1034 The receiver takes this action because a packet received out of order is 1035 always preceded by a packet that signalled a loss event. This loss 1036 event triggered the recovery process, which may have executed recovery 1037 commands. The MIDI command coded in the out-of-order packet might, if 1038 executed, duplicate these recovery commands, and this duplication might 1039 endanger the integrity of the stream. Thus, ignoring the out-of-order 1040 packet is the safe approach. 1042 6.2 High-Jitter Networks, Local Area Networks 1044 The NMP receiver targets a network with a particular set of 1045 characteristics: low nominal jitter, low packet loss, and occasional 1046 outlier packets that arrive very late. In this section, we consider how 1047 networks with different characteristics impact receiver design. 1049 Networks with significant nominal jitter cannot use the buffer-free 1050 receiver design described in Section 6.1. For example, the NMP system 1051 performs poorly for musicians that use dial-up modem connections, 1052 because the buffer-free receiver design modulates modem jitter onto the 1053 performances. Receivers designed for high-jitter networks should use a 1054 substantial playout buffer. References [GRAME] and [CCRMA] describe how 1055 to use playout buffers in latency-critical applications. 1057 Receivers intended for use on Local Area Networks (LANs) face a 1058 different set of issues. A dedicated LAN fabric built with modern 1059 hardware is in many ways a predictable environment. The network 1060 problems addressed by the NMP receiver design (packet loss and outlier 1061 late packets) might only occur under extreme network overload 1062 conditions. 1064 Systems designed for this environment may choose to configure streams 1065 without the recovery journal system (Appendix C.2.1 of [RTPMIDI]). 1066 Receivers may also wish to forego, or simplify, the detection of outlier 1067 late packets. Receivers should monitor the RTP sequence numbers of 1068 incoming packets, to detect network unreliability. 1070 However, in some respects, LAN applications may be more demanding than 1071 WAN applications. In LAN applications, musicians may be receiving 1072 performance feedback from audio that is rendered from the stream. The 1073 tolerance a musician has for latency and jitter in this context may be 1074 quite low. 1076 To reduce the perceived jitter, receivers may use a small playout buffer 1077 (in the range of 100us to 2ms). The buffer does add a small amount of 1078 latency to the system, that may be annoying to some players. Receiver 1079 designs should include buffer tuning parameters, to let musicians adjust 1080 the tradeoff between latency and jitter. 1082 7. Receiving Streams: The Recovery Journal 1084 In this section, we describe the recovery algorithm used by the NMP 1085 receiver [NMP]. In most ways, the recovery techniques we describe are 1086 generally applicable to interactive receiver design. However, a few 1087 aspects of the design are specialized for the NMP system: 1089 o The recovery algorithm covers a subset of the MIDI command 1090 set. MIDI Systems (0xF), Poly Aftertouch (0xA), and Channel 1091 Aftertouch (0xD) commands are not protected, and Control 1092 Change (0xB) commands protection is simplified. Note commands 1093 for a particular note number are assumed to follow the typical 1094 NoteOn->NoteOff->NoteOn->NoteOff pattern. The cm_unused and 1095 ch_never parameters in Figures 1-2 specify this coverage. 1097 o The NMP system does not use a playout buffer. Therefore, the 1098 recovery algorithm does not address interactions with a 1099 playout buffer. 1101 At a high level, the receiver algorithm works as follows. Upon the 1102 detection of a packet loss, the receiver examines the recovery journal 1103 of the packet that ends the loss event. If necessary, the receiver 1104 executes one or more MIDI commands to recover from the loss. 1106 To prepare for recovery, a receiver maintains a data structure, the 1107 Recovery Journal Receiver Structure (RJRS). The RJRS codes information 1108 about the MIDI commands the receiver executes (both incoming stream 1109 commands and self-generated recovery commands). At the start of the 1110 stream, the RJRS is initialized to code that no commands have been 1111 executed. Immediately after executing a MIDI command, the receiver 1112 updates the RJRS with information about the command. 1114 We now describe the recovery algorithm in detail. We begin with two 1115 definitions that classify loss events. These definitions assume that 1116 the packet that ends the loss event has RTP sequence number I. 1118 o Single-packet loss. A single-packet loss occurs if the last 1119 packet received before the loss event (excluding out-of-order 1120 packets) has the sequence number I-2 (modulo 2^16). 1122 o Multi-packet loss. A multi-packet loss occurs if the last 1123 packet received before the loss event (excluding out-of-order 1124 packets) has a sequence number less than I-2 (modulo 2^16). 1126 Upon the detection of a packet loss, the recovery algorithm examines the 1127 recovery journal header (Figure 8 of [RTPMIDI]) to check for special 1128 cases: 1130 o If the header field A is 0, the recovery journal has no channel 1131 journals, and so no action is taken. 1133 o If a single-packet loss has occurred, and the header S bit is 1134 1, the lost packet has a MIDI command section with an empty 1135 MIDI list. No action is taken. 1137 If these checks fail, the algorithm parses the recovery journal body. 1138 For each channel journal (Figure 9 in [RTPMIDI]) in the recovery 1139 journal, the receiver compares the data in each chapter journal 1140 (Appendix A of [RTPMIDI]) to the RJRS data for the chapter. If the data 1141 are inconsistent, the algorithm infers that MIDI command(s) related to 1142 the chapter journal have been lost. The recovery algorithm executes 1143 MIDI commands to repair this loss, and updates the RJRS to reflect the 1144 repair. 1146 For single-packet losses, the receiver skips channel and chapter 1147 journals whose S bits are set to 1. For multi-packet losses, the 1148 receiver parses each channel and chapter journal and checks for 1149 inconsistency. 1151 In the sections that follow, we describe the recovery steps that are 1152 specific to each chapter journal. We cover 4 chapter journal types: P 1153 (Program Change, 0xC), C (Control Change, 0xB), W (Pitch Wheel, 0xE), 1154 and N (Note, 0x8 and 0x9). Chapters are parsed in the order of their 1155 appearance in the channel journal (P, then W, then N, then C). 1157 The sections below reference the C implementation of the RJRS shown in 1158 Figure 10. This structure is hierarchical, reflecting the recovery 1159 journal architecture. At the leaf level, specialized data structures 1160 (jrec_chapterw, jrec_chaptern, jrec_chapterc, and jrec_chapterp) code 1161 state variables for a single chapter journal type. A mid-level 1162 structure (jrec_channel) represents a single MIDI channel, and a top- 1163 level structure (jrec_stream) represents the entire MIDI stream. 1165 typedef unsigned char uint8; /* must be 1 octet */ 1166 typedef unsigned short uint16; /* must be 2 octets */ 1167 typedef unsigned long uint32; /* must be 4 octets */ 1169 /*****************************************************************/ 1170 /* leaf level of hierarchy: Chapter W, Appendix A.5 of [RTPMIDI] */ 1171 /*****************************************************************/ 1173 typedef struct jrec_chapterw { /* MIDI Pitch Wheel (0xE) */ 1175 uint16 val; /* most recent 14-bit wheel value */ 1177 } jrec_chapterw; 1179 /*****************************************************************/ 1180 /* leaf level of hierarchy: Chapter N, Appendix A.6 of [RTPMIDI] */ 1181 /*****************************************************************/ 1183 typedef struct jrec_chaptern { /* Note commands (0x8, 0x9) */ 1185 /* arrays of length 128 --> one for each MIDI Note number */ 1187 uint32 time[128]; /* exec time of most recent NoteOn */ 1188 uint32 extseq[128]; /* extended seqnum for that NoteOn */ 1189 uint8 vel[128]; /* NoteOn velocity (0 for NoteOff) */ 1191 } jrec_chaptern; 1193 /*****************************************************************/ 1194 /* leaf level of hierarchy: Chapter C, Appendix A.3 of [RTPMIDI] */ 1195 /*****************************************************************/ 1197 typedef struct jrec_chapterc { /* Control Change (0xB) */ 1199 /* array of length 128 --> one for each controller number */ 1201 uint8 value[128]; /* Chapter C value tool state */ 1202 uint8 count[128]; /* Chapter C count tool state */ 1203 uint8 toggle[128]; /* Chapter C toggle tool state */ 1205 } jrec_chapterc; 1207 Figure 10 -- Recovery Journal Receiving Structure (part 1) 1209 /*****************************************************************/ 1210 /* leaf level of hierarchy: Chapter P, Appendix A.2 of [RTPMIDI] */ 1211 /*****************************************************************/ 1213 typedef struct jrec_chapterp { /* MIDI Program Change (0xC) */ 1215 uint8 prognum; /* most recent 7-bit program value */ 1216 uint8 prognum_qual; /* 1 once first 0xC command arrives */ 1218 uint8 bank_msb; /* most recent Bank Select MSB value */ 1219 uint8 bank_msb_qual; /* 1 once first 0xBn 0x00 arrives */ 1221 uint8 bank_lsb; /* most recent Bank Select LSB value */ 1222 uint8 bank_lsb_qual; /* 1 once first 0xBn 0x20 arrives */ 1224 } jrec_chapterp; 1226 /***************************************************/ 1227 /* second-level of hierarchy, for MIDI channels */ 1228 /***************************************************/ 1230 typedef struct jrec_channel { 1232 jrec_chapterp chapterp; /* Program Change (0xC) info */ 1233 jrec_chapterc chapterc; /* Control Change (0xB) info */ 1234 jrec_chapterw chapterw; /* Pitch Wheel (0xE) info */ 1235 jrec_chaptern chaptern; /* Note (0x8, 0x9) info */ 1237 } jrec_channel; 1239 /***********************************************/ 1240 /* top level of hierarchy, for the MIDI stream */ 1241 /***********************************************/ 1243 typedef struct jrec_stream { 1245 jrec_channel channels[16]; /* index is MIDI channel */ 1247 } jrec_stream; 1249 Figure 10 (continued) -- Recovery Journal Receiving Structure 1251 7.1 Chapter W: MIDI Pitch Wheel (0xE) 1253 Chapter W of the recovery journal protects against the loss of MIDI 1254 Pitch Wheel (0xE) commands. A common use of the Pitch Wheel command is 1255 to transmit the current position of a rotary "pitch wheel" controller 1256 placed on the side of MIDI piano controllers. Players use the pitch 1257 wheel to dynamically alter the pitch of all depressed keys. 1259 The NMP receiver maintains the jrec_chapterw structure (Figure 10) for 1260 each voice channel in jrec_stream, to code pitch wheel state 1261 information. In jrec_chapterw, val holds the 14-bit data value of the 1262 most recent Pitch Wheel command that has arrived on a channel. At the 1263 start of the stream, val is initialized to the default pitch wheel value 1264 (0x2000). 1266 At the end of a loss event, a receiver may find a Chapter W (Appendix 1267 A.5 in [RTPMIDI]) bitfield in a channel journal. This chapter codes the 1268 14-bit data value of the most recent MIDI Pitch Wheel command in the 1269 checkpoint history. If the Chapter W and jrec_chapterw pitch wheel 1270 values do not match, one or more commands have been lost. 1272 To recover from this loss, the NMP receiver immediately executes a MIDI 1273 Pitch Wheel command on the channel, using the data value coded in the 1274 recovery journal. The receiver then updates the jrec_chapterw variables 1275 to reflect the executed command. 1277 7.2 Chapter N: MIDI NoteOn (0x8) and NoteOff (0x9) 1279 Chapter N of the recovery journal protects against the loss of MIDI 1280 NoteOn (0x9) and NoteOff (0x8) commands. If a NoteOn command is lost, a 1281 note is skipped. If a NoteOff command is lost, a note may sound 1282 indefinitely. Recall that NoteOn commands with a velocity value of 0 1283 have the semantics of NoteOff commands. 1285 The recovery algorithms in this section only work for MIDI sources that 1286 produce NoteOn->NoteOff->NoteOn->NoteOff patterns for a note number. 1287 Piano keyboard and drum pad controllers produce these patterns. MIDI 1288 sources that use NoteOn->NoteOn->NoteOff->NoteOff patterns for legato 1289 repeated notes, such as guitar and wind controllers, require more 1290 sophisticated recovery strategies. Chapter E (not used in this example) 1291 supports recovery algorithms for atypical note command patterns (see 1292 Appendix A.7 of [RTPMIDI] for details). 1294 The NMP receiver maintains a jrec_chaptern structure (Figure 10) for 1295 each voice channel in jrec_stream, to code note-related state 1296 information. State is kept for each of the 128 note numbers on a 1297 channel, using three arrays of length 128 (vel[], seq[], and time[]). 1298 The arrays are initialized to zero at the start of a stream. 1300 The vel[n] array element holds information about the most recent note 1301 command for note number n. If this command is a NoteOn command, vel[n] 1302 holds the velocity data for the command. If this command is a NoteOff 1303 command, vel[n] is set to 0. 1305 The time[n] and extseq[n] array elements code information about the most 1306 recently executed NoteOn command. The time[n] element holds the 1307 execution time of the command, referenced to the local timebase of the 1308 receiver. The extseq[n] element holds the RTP extended sequence number 1309 of the packet associated with the command. For incoming stream 1310 commands, extseq[n] codes the packet of the associated MIDI list. For 1311 commands executed to perform loss recovery, extseq[n] codes the packet 1312 of the associated recovery journal. 1314 The Chapter N recovery journal bitfield (Figure A.6.1 in [RTPMIDI]) 1315 consists of two data structures: a bit array coding recently-sent 1316 NoteOff commands that are vulnerable to packet loss, and a note log list 1317 coding recently-sent NoteOn commands that are vulnerable to packet loss. 1319 At the end of a loss event, Chapter N recovery processing begins with 1320 the NoteOff bit array. For each set bit in the array, the receiver 1321 checks the corresponding vel[n] element in jrec_chaptern. If vel[n] is 1322 non-zero, a NoteOff command, or a NoteOff->NoteOn->NoteOff command 1323 sequence, has been lost. To recover from this loss, the receiver 1324 immediately executes a NoteOff command for the note number on the 1325 channel, and sets vel[n] to 0. 1327 The receiver then parses the note log list, using the S bit to skip over 1328 "safe" logs in the single-packet loss case. For each at-risk note log, 1329 the receiver checks the corresponding vel[n] element. 1331 If vel[n] is zero, a NoteOn command, or a NoteOn->NoteOff->NoteOn 1332 command sequence, has been lost. The receiver may execute the most 1333 recent lost NoteOn (to play the note) or may take no action (to skip the 1334 note), based on criteria we describe at the end of this section. 1335 Whether the note is played or skipped, the receiver updates the vel[n], 1336 time[n], and extseq[n] elements as if the NoteOn executed. 1338 If vel[n] is non-zero, the receiver performs several checks to test if a 1339 NoteOff->NoteOn sequence has been lost. 1341 o If vel[n] does not match the note log velocity, the note log 1342 must code a different NoteOn command, and thus a NoteOff->NoteOn 1343 sequence has been lost. 1345 o If extseq[n] is less than the (extended) checkpoint packet 1346 sequence numbed coded in the recovery journal header (Figure 8 1347 of [RTPMIDI]), the vel[n] NoteOn command is not in the checkpoint 1348 history, and thus a NoteOff->NoteOn sequence has been lost. 1350 o If the Y bit is set to 1, the NoteOn is musically "simultaneous" 1351 with the RTP timestamp of the packet. If time[n] codes a time 1352 value that is clearly not recent, a NoteOff->NoteOn sequence has 1353 been lost. 1355 If these tests indicate a lost NoteOff->NoteOn sequence, the receiver 1356 immediately executes a NoteOff command. The receiver decides if the 1357 most graceful action is to play or to skip the lost NoteOn, using the 1358 criteria we describe at the end of this section. Whether or not the 1359 receiver issues a NoteOn command, the vel[n], time[n], and extseq[n] 1360 arrays are updated as if it did. 1362 Note that the tests above do not catch all lost NoteOff->NoteOn 1363 commands. If a fast NoteOn->NoteOff->NoteOn sequence occurs on a note 1364 number, with identical velocity values for both NoteOn commands, a lost 1365 NoteOff->NoteOn does not result in the recovery algorithm generating a 1366 NoteOff command. Instead, the first NoteOn continues to sound, to be 1367 terminated by the future NoteOff command. In practice, this (rare) 1368 outcome is not musically objectionable. 1370 The number of tests in this resiliency algorithm may seem excessive. 1371 However, in some common cases, a subset of the tests are not useful. 1372 For example, MIDI streams that assigns the same velocity value to all 1373 note events are often produced by inexpensive keyboards. The vel[n] 1374 tests are not useful for these streams. 1376 Finally, we discuss how the receiver decides whether to play or to skip 1377 a lost NoteOn command. The note log Y bit is set if the NoteOn is 1378 "simultaneous" with the RTP timestamp of the packet holding the note 1379 log. If Y is 0, the receiver does not execute a NoteOn command. If Y 1380 is 1, and if the packet has not arrived late, the receiver immediately 1381 executes a NoteOn command for the note number, using the velocity coded 1382 in the note log. 1384 7.3 Chapter C: MIDI Control Change (0xB) 1386 Chapter C (Appendix A.3 in [RTPMIDI]) protects against the loss of MIDI 1387 Control Change commands. A Control Change command alters the 7-bit 1388 value of one of the 128 MIDI controllers. 1390 Chapter C offers three tools for protecting a Control Change command: 1391 the value tool (for graded controllers such as sliders) the toggle tool 1392 (for on/off switches) and the count tool (for momentary-contact 1393 switches). Senders choose a tool to encode recovery information for a 1394 controller, and encode the tool type along with the data in the journal 1395 (Figures A.3.2 and A.3.3 in [RTPMIDI]). 1397 A few uses of Control Change commands are not solely protected by 1398 Chapter C. The protection of controllers 0 and 32 (Bank Select MSB and 1399 Bank Select LSB) is shared between Chapter C and Chapter P (Section 1400 7.4). 1402 Chapter M (Appendix A.4 of [RTPMIDI]) also protects the Control Change 1403 command. However, the NMP system does not use this chapter, because 1404 MPEG 4 Structured Audio [MPEGSA] does not use the controllers protected 1405 by this chapter. 1407 The Chapter C bitfield consists of a list of controller logs. Each log 1408 codes the controller number, the tool type, and the state value for the 1409 tool. 1411 The NMP receiver maintains the jrec_chapterc structure (Figure 10) for 1412 each voice channel in jrec_stream, to code Control Change state 1413 information. The value[] array holds the most recent data values for 1414 each controller number. At the start of the stream, value[] is 1415 initialized to the default controller data values specified in [MPEGSA]. 1417 The count[] and toggle[] arrays hold the count tool and toggle tool 1418 state values. At the start of a stream, these arrays are initialized to 1419 zero. Whenever a Control Command executes, the receiver updates the 1420 count[] and toggle[] state values, using the algorithms defined in 1421 Appendix A.3 of [RTPMIDI]. 1423 At the end of a loss event, the receiver parses the Chapter C controller 1424 log list, using the S bit to skip over "safe" logs in the single-packet 1425 loss case. For each at-risk controller number n, the receiver 1426 determines the tool type in use (value, toggle, or count), and compares 1427 the data in the log to the associated jrec_chapterc array element 1428 (value[n], toggle[n], or count[n]). If the data do not match, one or 1429 more Control Change commands have been lost. 1431 The method the receiver uses to recover from this loss depends on the 1432 tool type and the controller number. For graded controllers protected 1433 by the value tool, the receiver executes a Control Change command using 1434 the new data value. 1436 For the toggle and count tools, the recovery action is more complex. 1437 For example, the Damper Pedal (Sustain) controller (number 64) is 1438 typically used as a sustain pedal for piano-like sounds, and is 1439 typically coded using the toggle tool. If Damper Pedal (Sustain) 1440 Control Change command(s) are lost, the receiver takes different actions 1441 depending on the starting and ending state of the lost sequence, to 1442 ensure "ringing" piano notes are "damped" to silence. 1444 After recovering from the loss, the receiver updates the value[], 1445 toggle[], and count[] arrays to reflect the Chapter C data and the 1446 executed commands. 1448 7.4 Chapter P: MIDI Program Change (0xC) 1450 Chapter P of the recovery journal protects against the loss of MIDI 1451 Program Change (0xC) commands. 1453 The 7-bit data value of the Program Change command selects one of 128 1454 possible timbres for the channel. To increase the number of possible 1455 timbres, Control Change (0xB) commands may be issued prior to the 1456 Program Change command, to select a "program bank". The Bank Select MSB 1457 (number 0) and Bank Select LSB (number 32) controllers specify the 1458 14-bit bank number that subsequent Program Change commands reference. 1460 The NMP receiver maintains the jrec_chapterp structure (Figure 10) for 1461 each voice channel in jrec_stream, to code Program Change state 1462 information. 1464 The prognum variable of jrec_chapterp holds the data value for the most 1465 recent Program Change command that has arrived on the stream. The 1466 bank_msb and bank_lsb variables of jrec_chapterp code the Bank Select 1467 MSB and Bank Select LSB controller data values that were in effect when 1468 that Program Change command arrived. The prognum_qual, bank_msb_qual 1469 and bank_lsb_qual variables are initialized to 0, and are set to 1 to 1470 qualify the associated data values. 1472 Chapter P fields code the data value for the most recent Program Change 1473 command, and the MSB and LSB bank values in effect for that command. 1475 At the end of a loss event, the receiver checks Chapter P to see if the 1476 recovery journal fields match the data stored in jrec_chapterp. If 1477 these checks fail, one or more Program Change commands have been lost. 1479 To recover from this loss, the receiver takes the following steps. If 1480 the B bit in Chapter P is set (Figure A.2.1 in [RTPMIDI]), Control 1481 Change bank command(s) have preceded the Program Change command. The 1482 receiver compares the bank data coded by Chapter P with the current bank 1483 data for the channel (coded in jrec_channelc). 1485 If the bank data do not agree, the receiver issues Control Change 1486 command(s) to align the stream with Chapter P. The receiver then 1487 updates jrec_channelp and jrec_channelc variables to reflect the 1488 executed command(s). Finally, the receiver issues a Program Change 1489 command that reflects the data in Chapter P, and updates the prognum and 1490 qual_prognum fields in jrec_channelp. 1492 Note that this method relies on Chapter P recovery to precede Chapter C 1493 recovery during channel journal processing. This ordering ensures that 1494 lost Bank Select Control Change commands that occur after a lost Program 1495 Change command in a stream are handled correctly. 1497 8. Security Considerations 1499 Security considerations for the RTP MIDI payload format are discussed in 1500 the Security Considerations section of [RTPMIDI]. 1502 9. IANA Considerations 1504 IANA considerations for the RTP MIDI payload format are discussed in the 1505 IANA Considerations section of [RTPMIDI]. 1507 A. Acknowledgments 1509 This memo was written in conjunction with [RTPMIDI], and the 1510 Acknowledgments section of [RTPMIDI] also applies to this memo. 1512 B. References 1514 B.1 Normative References 1516 [RTPMIDI] Lazzaro, J., and J. Wawrzynek. "RTP Payload Format for MIDI", 1517 work in progress, draft-ietf-avt-rtp-midi-format-15.txt. 1519 [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson. 1520 "RTP: A transport protocol for real-time applications", RFC 3550, July 1521 2003. 1523 [RFC3551] Schulzrinne, H., and S. Casner. "RTP Profile for Audio and 1524 Video Conferences with Minimal Control", RFC 3551, July 2003. 1526 [MIDI] MIDI Manufacturers Association. "The Complete MIDI 1.0 1527 Detailed Specification", 1996. 1529 [SDP] Handley, M., Jacobson, V., and C. Perkins. "SDP: Session 1530 Description Protocol", draft-ietf-mmusic-sdp-new-25.txt. 1532 [MPEGSA] International Standards Organization. "ISO/IEC 14496 1533 MPEG-4", Part 3 (Audio), Subpart 5 (Structured Audio), 2001. 1535 [RFC3556] S. Casner. "Session Description Protocol (SDP) Bandwidth 1536 Modifiers for RTP Control Protocol (RTCP) Bandwidth", RFC 3556, July 1537 2003. 1539 B.2 Informative References 1541 [NMP] Lazzaro, J. and J. Wawrzynek. "A Case for Network Musical 1542 Performance", 11th International Workshop on Network and Operating 1543 Systems Support for Digital Audio and Video (NOSSDAV 2001) June 25-26, 1544 2001, Port Jefferson, New York. 1546 [RFC3261] Rosenberg, J, Schulzrinne, H., Camarillo, G., Johnston, A., 1547 Peterson, J., Sparks, R., Handley, M., and E. Schooler. "SIP: Session 1548 Initiation Protocol", RFC 3261, June 2002. 1550 [GRAME] Fober, D., Orlarey, Y. and S. Letz. "Real Time Musical Events 1551 Streaming over Internet", Proceedings of the International Conference 1552 on WEB Delivering of Music 2001, pages 147-154. 1554 [CCRMA] Chafe C., Wilson S., Leistikow R., Chisholm D., and G. Scavone. 1555 "A simplified approach to high quality music and sound over IP", 1556 COST-G6 Conference on Digital Audio Effects (DAFx-00), Verona, Italy, 1557 December 2000. 1559 [RTPBOOK] Perkins, C. "RTP: Audio and Video for the Internet", 1560 Addison-Wesley, ISBN 0-672-32249-8, 2003. 1562 [STEVENS] Stevens, R. W, Fenner, B., and A. Rudoff. "Unix Network 1563 Programming: The Sockets Networking API", Addison-Wesley, 2003. 1565 C. Authors' Addresses 1567 John Lazzaro (corresponding author) 1568 UC Berkeley 1569 CS Division 1570 315 Soda Hall 1571 Berkeley CA 94720-1776 1572 Email: lazzaro@cs.berkeley.edu 1574 John Wawrzynek 1575 UC Berkeley 1576 CS Division 1577 631 Soda Hall 1578 Berkeley CA 94720-1776 1579 Email: johnw@cs.berkeley.edu 1580 D. Intellectual Property Rights Statement 1582 The IETF takes no position regarding the validity or scope of any 1583 Intellectual Property Rights or other rights that might be claimed to 1584 pertain to the implementation or use of the technology described in this 1585 document or the extent to which any license under such rights might or 1586 might not be available; nor does it represent that it has made any 1587 independent effort to identify any such rights. Information on the 1588 procedures with respect to rights in RFC documents can be found in BCP 1589 78 and BCP 79. 1591 Copies of IPR disclosures made to the IETF Secretariat and any 1592 assurances of licenses to be made available, or the result of an attempt 1593 made to obtain a general license or permission for the use of such 1594 proprietary rights by implementers or users of this specification can be 1595 obtained from the IETF on-line IPR repository at 1596 http://www.ietf.org/ipr. 1598 The IETF invites any interested party to bring to its attention any 1599 copyrights, patents or patent applications, or other proprietary rights 1600 that may cover technology that may be required to implement this 1601 standard. Please address the information to the IETF at ietf- 1602 ipr@ietf.org. 1604 E. Full Copyright Statement 1606 Copyright (C) The Internet Society (2006). This document is subject to 1607 the rights, licenses and restrictions contained in BCP 78, and except as 1608 set forth therein, the authors retain all their rights. 1610 This document and the information contained herein are provided 1611 on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE 1612 REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND 1613 THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, 1614 EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT 1615 THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR 1616 ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A 1617 PARTICULAR PURPOSE. 1619 Acknowledgement 1621 Funding for the RFC Editor function is currently provided by the 1622 Internet Society. 1624 F. Change Log 1626 [Note to RFC Editors: this Appendix, and its Table of Contents listing, 1627 should be removed from the final version of the memo] 1629 The following changes were made to the document: 1631 -- 1633 [1] In the Introduction, after the the paragraph that begins "The memo 1634 focuses on one application" we add the paragraph: 1636 The application targets a network with a particular set of 1637 characteristics: low nominal jitter, low packet loss, and occasional 1638 outlier packets that arrive very late. However, in Section 6.2 of this 1639 memo we discuss adapting the application to other network environments. 1641 -- 1643 [2] In Section 2, the phrase "uses the sockets API and other POSIX 1644 systems calls" was replaced by "uses standard network programming 1645 techniques described in [STEVENS]", where [STEVENS] is the informative 1646 reference: 1648 [STEVENS] Stevens, R. W, Fenner, B., and A. Rudoff. "Unix Network 1649 Programming: The Sockets Networking API", Addison-Wesley, 2003. 1651 -- 1653 [3] RTCP is now defined as "RTP control protocol", not "Real Time 1654 Control Protocol. 1656 -- 1658 [4] The [RTPMIDI] reference was updated to -15.txt.