idnits 2.17.1 draft-ietf-avt-fec-08.txt: ** The Abstract section seems to be numbered Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity -- however, there's a paragraph with a matching beginning. Boilerplate error? ** The document is more than 15 pages and seems to lack a Table of Contents. == No 'Intended status' indicated for this document; assuming Proposed Standard == It seems as if not all pages are separated by form feeds - found 0 form feeds but 24 pages Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** There are 9 instances of too long lines in the document, the longest one being 7 characters in excess of 72. == There are 1 instance of lines with non-RFC6890-compliant IPv4 addresses in the document. If these are example addresses, they should be changed. == There are 3 instances of lines with multicast IPv4 addresses in the document. If these are generic example addresses, they should be changed to use the 233.252.0.x range defined in RFC 5771 Miscellaneous warnings: ---------------------------------------------------------------------------- -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (August 16, 1999) is 8991 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Missing reference section? '1' on line 1027 looks like a reference -- Missing reference section? '2' on line 1032 looks like a reference -- Missing reference section? '3' on line 1036 looks like a reference -- Missing reference section? '4' on line 1040 looks like a reference -- Missing reference section? '5' on line 1044 looks like a reference -- Missing reference section? '24' on line 530 looks like a reference -- Missing reference section? '65535' on line 537 looks like a reference -- Missing reference section? '6' on line 1049 looks like a reference -- Missing reference section? '7' on line 1053 looks like a reference Summary: 6 errors (**), 0 flaws (~~), 4 warnings (==), 12 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Internet Engineering Task Force Audio Video Transport WG 2 Internet Draft J.Rosenberg,H.Schulzrinne 3 draft-ietf-avt-fec-08.txt Bell Laboratories,Columbia U. 4 August 16, 1999 5 Expires: February 2000 7 An RTP Payload Format for Generic Forward Error Correction 9 STATUS OF THIS MEMO 11 This document is an Internet-Draft and is in full conformance with 12 all provisions of Section 10 of RFC2026. 14 Internet-Drafts are working documents of the Internet Engineering 15 Task Force (IETF), its areas, and its working groups. Note that 16 other groups may also distribute working documents as Internet- 17 Drafts. 19 Internet-Drafts are draft documents valid for a maximum of six months 20 and may be updated, replaced, or obsoleted by other documents at any 21 time. It is inappropriate to use Internet- Drafts as reference 22 material or to cite them other than as work in progress. 24 The list of current Internet-Drafts can be accessed at 25 http://www.ietf.org/ietf/1id-abstracts.txt 27 The list of Internet-Draft Shadow Directories can be accessed at 28 http://www.ietf.org/shadow.html. 30 1 Abstract 32 This document specifies a payload format for generic forward error 33 correction of media encapsulated in RTP. It is engineered for FEC 34 algorithms based on the exclusive-or (parity) operation. The payload 35 format allows end systems to transmit using arbitrary block lengths 36 and parity schemes. It also allows for the recovery of both the 37 payload and critical RTP header fields. Since FEC is sent as a 38 separate stream, it is backwards compatible with non-FEC capable 39 hosts, so that receivers which do not wish to implement FEC can just 40 ignore the extensions. 42 2 Introduction 44 The quality of packet voice on the Internet has been mediocre due, in 45 part, to high packet loss rates. This is especially true on wide-area 46 connections. Unfortunately, the strict delay requirements of real- 47 time multimedia usually eliminate the possibility of retransmissions. 49 It is for this reason that forward error correction (FEC) has been 50 proposed to compensate for packet loss in the Internet [1] [2]. In 51 particular, the use of traditional error correcting codes, such as 52 parity, Reed-Solomon, and Hamming codes, has attracted attention. To 53 support these mechanisms, protocol support is required. 55 This document defines a payload format for RTP [3] which allows for 56 generic forward error correction of real time media. In this context, 57 generic means that the FEC protocol is (1) independent of the nature 58 of the media being protected, be it audio, video, or otherwise, (2) 59 flexible enough to support a wide variety of FEC mechanisms, (3) 60 designed for adaptivity so that the FEC technique can be modified 61 easily without out of band signaling, and (4) supportive of a number 62 of different mechanisms for transporting the FEC packets. 64 3 Terminology 66 The following terms are used throughout this document: 68 Media Payload: is a piece of raw, un-protected user data which 69 is to be transmitted from the sender. The media payload is 70 placed inside of an RTP packet. 72 Media Header: is the RTP header for the packet containing the 73 media payload. 75 Media Packet: The combination of a media payload and media 76 header is called a media packet. 78 FEC Packet: The forward error correction algorithms at the 79 transmitter take the media packets as an input. They output 80 both the media packets that they are passed, and new 81 packets called FEC packets. The FEC packets are formatted 82 according to the rules specified in this document. 84 FEC Header: The FEC header is the header information contained 85 in an FEC packet. 87 FEC Payload: The FEC payload is the payload in an FEC packet. 89 Associated: An FEC packet is said to be "associated" with one or 90 more media packets when those media packets are used to 91 generate the FEC packet (by use of the exclusive or 92 operation). 94 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 95 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 96 document are to be interpreted as described in RFC 2119 [4]. 98 4 Basic Operation 100 The payload format described here is used whenever a participant in 101 an RTP session would like to protect a media stream it is sending 102 with forward error correction (FEC). The FEC supported by the format 103 are those codes based on simple exclusive or (xor) parities. The 104 sender takes some set of packets from the media stream, and applies 105 an xor operation across the payloads. The sender also applies the xor 106 operation over components of the RTP headers. Based on the procedures 107 defined here, the result is an RTP packet containing FEC information. 108 This packet can be used at the receiver to recover any one of the 109 packets used to generate the FEC packet. This document does not 110 mandate the particular set of media packets combined to generate an 111 FEC packet (such a set referred to as a code). Use of differing sets 112 results in a tradeoff between overhead, delay, and recoverability. 113 Section 5 outlines some possible combinations. 115 The payload format contains information that allows the sender to 116 tell the receiver exactly which media packets have been used to 117 generate the FEC. Specifically, each FEC packet contains a bitmask, 118 called the offset mask, containing 24 bits. If bit i in the mask is 119 set to 1, the media packet with sequence number N + i was used to 120 generate this FEC packet. N is called the sequence number base, and 121 is sent in the FEC packet as well. The offset mask and payload type 122 are sufficient to signal arbitrary parity based forward error 123 correction schemes with little overhead. 125 This document also describes procedures that allow the receiver to 126 make use of the FEC without having to know the details of specific 127 codes. This allows the sender much flexibility; it can adapt the code 128 in use based on network conditions, and be certain the receivers can 129 still make use of the FEC for recovery. 131 As the sender generates FEC packets, they are sent to the receivers. 132 The sender still usually sends the original media stream, as if there 133 were no FEC. This allows the media stream to still be used by 134 receivers who are not FEC capable. However, some FEC codes do not 135 require the original media to be sent; the FEC stream is sufficient 136 for recovery. These codes have the drawback that all receivers must 137 be FEC capable. However, they are supported by this format. 139 The FEC packets are not sent in the same RTP stream as the media 140 packets. They can be sent as a separate stream, or as a secondary 141 codec in the redundant codec payload format [5]. When sent as a 142 separate stream, the FEC packets have their own sequence number 143 space. Although the timestamps for the FEC packets are derived from 144 the media packets, they increment monotonically. FEC packet streams 145 thus work well with any header compression mechanism which requires 146 fixed deltas between fields in the packet header. 148 This document does not prescribe the definition of "separate 149 streams", but leaves this to applications and higher level protocols 150 to define. For multicast, the separate stream may be implemented by 151 separate multicast groups, different ports in the same group, or by a 152 different SSRC within the same group/port. For unicast, different 153 ports or different SSRC may be used. Each of these approaches has 154 drawbacks and benefits which depend on the application. 156 At the receiver, the FEC and original media are received. If no media 157 packets are lost, the FEC can be ignored. In the event of loss, the 158 FEC packets can be combined with other media and FEC packets that 159 have been received, resulting in recovery of missing media packets. 160 The recovery is exact; the payload is perfectly reconstructed, along 161 with most components of the header. 163 RTP packets which contain data formatted according to this 164 specification (i.e., FEC packets) are signaled using dynamic RTP 165 payload types. 167 5 Parity Codes 169 For brevity, we define the function f(x,y,..) to be the XOR (parity) 170 operator applied to the packets x,y,... The output of this function 171 is another packet, called the parity packet. For simplicity, we 172 assume here that the parity packet is computed as the bitwise XOR of 173 the input packets. The exact procedure is specified in section 7. 175 Recovery of data packets using parity codes is accomplished by 176 generating one or more parity packets over a group of data packets. 177 To be effective, the parity packets must be generated by linearly 178 independent combinations of data packets. The particular combination 179 is called a parity code. One class of codes takes a group of k data 180 packets, and generates n-k parity packets. There are a large number 181 of possible parity codes for a given n,k. The payload format does not 182 mandate a particular code. 184 For example, consider a parity code which generates a single parity 185 packet over two data packets. If the original media packets are 186 a,b,c,d, the packets generated by the sender are: 188 a b c d <-- media stream 189 f(a,b) f(c,d) <-- FEC stream 191 where time increases to the right. In this example, the error 192 correction scheme (we use the terms scheme and code interchangeably) 193 introduces a 50% overhead. But if b is lost, a and f(a,b) can be used 194 to recover b. 196 Some additional codes are listed below. In each, the original media 197 stream consists of packets a,b,c,d and so on. 199 Scheme 1 200 -------- 202 This scheme is the similar to the one in the example above. However, 203 instead of sending b, followed by f(a,b), f(a,b) is sent before 204 b. Doing this clearly requires additional delay at the 205 sender. However, if allows some bursts of two consecutive packet 206 losses to be recovered. The packets generated by the sender look like: 208 a b c d e <-- media stream 209 f(a,b) f(b,c) f(c,d) f(d,e) <-- FEC stream 211 Scheme 2 212 -------- 214 It is not strictly necessary for the original media stream to be 215 transmitted. In this scheme, only FEC packets are transmitted. This 216 scheme allows for recovery of all single packet losses and some 217 consecutive packet losses, but with slightly less overhead than scheme 218 1. The packets generated by the sender look like: 220 f(a,b) f(a,c) f(a,b,c) f(c,d) f(c,e) f(c,d,e) <-- FEC stream 222 Scheme 3 223 -------- 225 This scheme requires the receiver to wait an additional four packet 226 intervals to recover the original media packets. However, it can 227 recover from one, two or three consecutive packet losses. The packets 228 generated by the sender look like: 230 a b c d <-- media stream 231 f(a,b,c) f(a,c,d) f(a,b,d) <-- FEC stream 233 6 RTP Media Packet Structure 234 The formatting of the media packets are unaffected by FEC. If the FEC 235 is sent as a separate stream, the media packets are sent as if there 236 was no FEC. If the FEC is being sent as a redundant codec, the media 237 packets are sent as the main codec as defined in RFC2198 [5]. 239 This lends to a very efficient encoding. When little (or no) FEC is 240 used, there are mostly media packets being sent. This means that the 241 overhead (present in FEC packets only) tracks the amount of FEC in 242 use. 244 7 FEC Packet Structure 246 An FEC packet is constructed by placing an FEC header and FEC payload 247 in the RTP payload, as shown in Figure 1: 249 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 250 | RTP Header | 251 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 252 | FEC Header | 253 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 254 | FEC Payload | 255 | | 256 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 258 Figure 1: FEC Packet Structure 260 7.1 RTP Header of FEC Packets 262 The version field is set to 2. The padding bit is computed via the 263 protection operation, defined below. The extension bit is also 264 computed via the protection operation. The SSRC value will generally 265 be the same as the SSRC value of the media stream it protects. It MAY 266 be different if the FEC stream is being demultiplexed via the SSRC 267 value. The CC value is computed via the protection operation. The 268 CSRC list is never present, independent of the value of the CC field. 269 The extension is never present, independent of the value of the X 270 bit. The marker bit is computed via the protection operation. 272 The sequence number has the standard definition: it MUST be one 273 higher than the sequence number in the previously transmitted FEC 274 packet. The timestamp MUST be set to the value of the media RTP clock 275 at the instant the FEC packet is transmitted. This results in the TS 276 value in FEC packets to be monotonically increasing, independent of 277 the FEC scheme. 279 The payload type for the FEC packet is determined through dynamic, 280 out of band means. According to RFC1889 [3], RTP participants which 281 cannot recognize a payload type must discard it. This provides 282 backwards compatibility. The FEC mechanisms can then be used in a 283 multicast group with mixed FEC-capable and FEC-incapable receivers. 285 7.2 FEC Header 287 This header is 12 bytes. The format of the header is shown in Figure 288 2, and consists of an SN base field, length recovery field, E field, 289 PT recovery field, mask field and TS recovery field. 291 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 292 | SN base | length recovery | 293 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 294 |E| PT recovery | mask | 295 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 296 | TS recovery | 297 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 299 Figure 2: Parity Header Format 301 The length recovery field is used to determine the length of any 302 recovered packets. It is computed via the protection operation 303 applied to the unsigned network-ordered 16 bit representation of the 304 sums of the lengths (in bytes) of the media payload, CSRC list, 305 extension and padding of media packets associated with this FEC 306 packet (in other words, the CSRC list, extension, and padding, if 307 present, are "counted" as part of the payload). This allows the FEC 308 procedure to be applied even when the lengths of the media packets 309 are not identical. For example, assume an FEC packet is being 310 generated by xor'ing two media packets together. The length of the 311 two media packets are 3 (0b011) and 5 (0b101) bytes, respectively. 312 The length recovery field is then encoded as 0b011 xor 0b101 = 0b110. 314 The E bit indicates a header extension. Implementations conforming to 315 this version of the specification MUST set this bit to zero. 317 The PT recovery field is obtained via the protection operation 318 applied to the payload type values of the media packets associated 319 with the FEC packet. 321 The mask field is 24 bits. If bit i in the mask is set to 1, then the 322 media packet with sequence number N + i is associated with this FEC 323 packet, where N is the SN Base field in the FEC packet header. The 324 least significant bit corresponds to i=0, and the most significant to 325 i=23. 327 The SN base field MUST be set to the minimum sequence number of those 328 media packets protected by FEC. This allows for the FEC operation to 329 extend over any string of at most 24 packets. 331 The TS recovery field is computed via the protection operation 332 applied to the timestamps of the media packets associated with this 333 FEC packet. This allows the timestamp to be completely recovered. 335 The payload of the FEC packet is the protection operation applied to 336 the concatenation of the CSRC list, RTP extension, media payload, and 337 padding of the media packets associated with the FEC packet. 339 Note that it's possible for the FEC packet to be slightly larger than 340 the media packets it protects (due to the presence of the FEC 341 header). This could cause difficulties if this results in the FEC 342 packet exceeding the Maximum Transmission Unit size for the path 343 along which it is sent. 345 8 Protection Operation 347 The protection operation involves concatenating specific fields from 348 the RTP header of the media packet, appending the payload, padding 349 with zeroes, and then computing the xor across the resulting bit 350 strings. The resulting bit string is used to generate the FEC packet. 352 The following procedure MAY be followed for the protection operation. 353 Other procedures MAY be followed, but the end result MUST be 354 identical to the one described here. For each media packet to be 355 protected, a bit string is generated by concatenating the following 356 fields together in the order specifed: 358 o Padding Bit (1 bit) 360 o Extension Bit (1 bit) 362 o CC bits (4 bits) 364 o Marker bit (1 bit) 366 o Payload Type (7 bits) 368 o Timestamp (32 bits) 370 o Unsigned network-ordered 16 bit representation of the sum of 371 the lengths of the CSRC List, length of the padding, length of 372 the extension, and length of the media packet (16 bits) 374 o if CC is nonzero, the CSRC List (variable length) 376 o if X is 1, the Header Extension (variable length) 378 o the payload (variable length) 380 o Padding, if present (variable length) 382 Note that the Padding Bit (first entry above) forms the most 383 significant bit of the bit string. 385 If the lengths of the bit strings are not equal, each bit string that 386 is shorter than the length of the longest, MUST be padded to the 387 length of the longest. Any value for the pad may be used. The pad 388 MUST be added at the end of the bit string. 390 The parity operation is then applied across the bit strings. The 391 result is the bit string used to build the FEC packet. Call this the 392 FEC bit string. 394 The first (most significant) bit in the FEC bit string is written 395 into the Padding Bit of the FEC packet. The second bit in the FEC bit 396 string is written into the Extension bit of the FEC packet. The next 397 four bits of the FEC bit string are written into the CC field of the 398 FEC packet. The next bit of the FEC bit string is written into the 399 marker bit of the FEC packet. The next 7 bits of the FEC bit string 400 are written into the PT recovery field in the FEC packet header. The 401 next 32 bits of the FEC bit string are written into the TS recovery 402 field in the packet header. The next 16 bits are written into the 403 length recovery field in the FEC packet header. The remaining bits 404 are set to be the payload of the FEC packet. 406 9 Recovery Procedures 408 The FEC packets allow end systems to recover from the loss of media 409 packets. All of the header fields of the missing packets, including 410 CSRC lists, extensions, padding bits, marker and payload type, are 411 recoverable. This section describes the procedure for performing 412 this recovery. 414 Recovery requires two distinct operations. The first determines which 415 packets (media and FEC) must be combined in order to recover a 416 missing packet. Once this is done, the second step is to actually 417 reconstruct the data. The second step MUST be performed as described 418 below. The first step MAY be based on any algorithm chosen by the 419 implementor. Different algorithms result in a tradeoff between 420 complexity and the ability to recover missing packets if at all 421 possible. 423 9.1 Reconstruction 425 Let T be the list of packets (FEC and media) which can be combined to 426 recover some media packet xi. The procedure is as follows: 428 1. For the media packets in T, compute the bit string as 429 described in the protection operation of the previous 430 section. 432 2. For the FEC packet in T, compute the bit string in the same 433 fashion, except always set the CSRC list, extension, and 434 padding to null. 436 3. If any of the bit strings generated from the media packets 437 are shorter than the bit string generated from the FEC 438 packet, pad them to be the same length as the bit string 439 generated from the FEC. The padding MUST be added at the 440 end of the bit string, and MAY be of any value. 442 4. Perform the exclusive or (parity) operation across the bit 443 strings, resulting in a recovery bit string. 445 5. Create a new packet with the standard 12 byte RTP header 446 and no payload. 448 6. Set the version of the new packet to 2. 450 7. Set the Padding bit in the new packet to the first bit in 451 the recovery bit string. 453 8. Set the Extension bit in the new packet to the second bit 454 in the recovery bit string. 456 9. Set the CC field to the next four bits in the recovery bit 457 string. 459 10. Set the marker bit in the new packet to the next bit in the 460 recovery bit string. 462 11. Set the payload type in the new packet to the next 7 bits 463 in the recovery bit string. 465 12. Set the SN field in the new packet to xi. 467 13. Set the TS field in the new packet to the next 32 bits in 468 the recovery bit string. 470 14. Take the next 16 bits of the recovery bit string. Whatever 471 unsigned integer this represents (assuming network-order), 472 take that many bytes from the recovery bit string and 473 append them to the new packet. This represents the CSRC 474 list, extension, payload, and padding. 476 15. Set the SSRC of the new packet to the SSRC of the media 477 stream it's protecting. 479 This procedure will completely recover both the header and payload of 480 an RTP packet. 482 9.2 Determination of When to Recover 484 The previous section discussed how to recover a media packet with 485 sequence number xi when all of the packets needed to recover it were 486 available. The decision about whether to attempt recovery of some 487 media packet xi, and how to determine if sufficient data is available 488 to recover it, is left to the implementor. However, this section 489 provides a simple algorithm which MAY be used for this purpose. 491 The algorithm is described below in C code. The code assumes that 492 several functions exist. recover_packet() takes the sequence number 493 of a packet, and an FEC packet. Using the FEC packet and data packets 494 received previously, the data packet with the given sequence number 495 is recovered. add_fec_to_pending_list() adds the given FEC packet to 496 a linked list of FEC packets which have not yet been used for 497 recovery. wait_for_packet() waits for a packet, FEC or data, from the 498 network. remove_from_pending_list() removes the FEC packet from the 499 pending list. The structure packet contains a boolean variable fec 500 which is true when the packet is FEC, false if it's media. When its 501 an FEC packet, the mask and snbase field contain those values from 502 the FEC packet header. When it's a media packet, the sn variable 503 contains the sequence number of the packet. The global array A 504 indicates which media packets have been received, and which have not. 505 It is indexed by the sequence number of the packet. 507 The function fec_recovery implements the algorithm. It waits for 508 packets, and when it receives an FEC packet, calls recover_with_fec() 509 to attempt to use it to recover. If no recovery is possible, the FEC 510 packet is stored for later attempts. If the received packet was a 511 media packet, its presence is noted, and any old FEC packets are 512 checked to see if recovery is now possible. Recovered packets are 513 treated as if they were received, triggering further attempts at 514 recovery. 516 A real implementation will need to use a circular buffer instead of 517 the simple array (A in the code) in order to avoid running off the 518 end of the buffer. In addition, the code below does not attempt to 519 free up FEC packets that are old and were never used. Normally, such 520 discarding is done based on time constraints introduced by the 521 playout buffer. If an FEC data protects packets whose play time has 522 elapsed, the FEC is no longer needed. 524 typedef struct packet_s { 526 BOOLEAN fec; /* FEC or media */ 528 int sn; /* SN of the packet, for media only */ 530 BOOLEAN mask[24]; /* Mask, FEC only */ 531 int snbase; /* SN Base, FEC only */ 533 struct packet_s *next; 535 } packet; 537 BOOLEAN A[65535]; 538 packet *pending_list; 540 packet *recover_with_fec(packet *fec_pkt) { 542 packet *data_pkt; 543 int pkts_present, /* number of packets from the mask that are present */ 544 pkts_needed, /* number of packets needed is the number of ones in 545 the mask minus 1 */ 546 pkt_to_recover, /* sn of the packet we are recovering */ 547 i; 549 pkts_present = 0; 551 /* The number of packets needed is the number of ones in the mask minus 1. 552 The code below increments pkts_needed by the number of ones in the 553 mask, so we initialize this to -1 so that the final count is correct */ 555 pkts_needed = -1; 557 /* Go through all 24 bits in the mask, and check if we have 558 all but one of the media packets */ 560 for(i = 0; i < 24; i++) { 561 /* If the packet is here and in the mask, increment counter */ 563 if(A[i+fec_pkt->snbase] && fec_pkt->mask[i]) pkts_present++; 565 /* Count the number of packets needed as well */ 566 if(fec_pkt->mask[i]) pkts_needed++; 568 /* The packet to recover is the one with a bit in the 569 mask that's not here yet */ 570 if(!A[i+fec_pkt->snbase] && fec_pkt->mask[i]) 571 pkt_to_recover = i+fec_pkt->snbase; 572 } 574 /* If we can recover, do so. Otherwise, return NULL */ 576 if(pkts_present == pkts_needed) { 577 data_pkt = recover_packet(pkt_to_recover, fec_pkt); 578 } else { 579 data_pkt = NULL; 580 } 582 return(data_pkt); 583 } 585 void fec_recovery() { 587 packet *p, /* packet received or regenerated */ 588 *fecp, /* fec packet from pending list */ 589 *pnew; /* new packets recovered */ 591 while(1) { 593 p = wait_for_packet(); /* get packet from network */ 595 while(p) { 597 /* if it's an FEC packet, try to recover with it. If we can't, 598 store it for later potential use. If we can recover, act as 599 if the recovered packet is received and try to recover some 600 more. Otherwise, if it's a data packet, mark it as received, 601 and check if we can now recover a data packet with the list 602 of pending FEC packets */ 604 if(p->fec == TRUE) { 605 pnew = recover_with_fec(p); 607 if(pnew) 608 A[pnew->sn] = TRUE; 609 else 610 add_fec_to_pending_list(p); 612 /* We assign pnew to p since the while loop will continue 613 to recover based on p not being NULL */ 614 p = pnew; 616 } else { 618 /* Mark this data packet as here */ 619 A[p->sn] = TRUE; 621 free(p); 622 p = NULL; 624 /* Go through pending list. Try and recover a packet using 625 each FEC. If we are successful, add the data packet to 626 the list of received packets, remove the FEC packet from the 627 pending list, since we've used it, and then try to recover 628 some more */ 630 for(fecp = pending_list; fecp != NULL; fecp = fecp->next) { 631 pnew = recover_with_fec(fecp); 632 if(pnew) { 634 /* The packet is now here, as we've recovered it */ 635 A[pnew->sn] = TRUE; 637 /* One FEC packet can only be used once to recover, 638 so remove it from the pending list */ 640 remove_fec_from_pending_list(fecp); 642 p = pnew; 644 break; 645 } 647 } /*for*/ 649 } /*p->fec was false */ 651 } /* while p*/ 653 } /* while 1 */ 655 } 657 10 Example 659 Consider 2 media packets to be sent, x and y, from SSRC 2. Their 660 sequence numbers are 8 and 9, respectively, with timestamps of 3 and 661 5, respectively. Packet x uses payload type 11, and packet y uses 662 payload type 18. Packet x is has 10 bytes of payload, and packet y 663 11. Packet y has its marker bit set. The RTP headers for packets x 664 and y are shown in Figures 3 and 4 respectively. 666 Media Packet x 668 0 1 2 3 669 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 670 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 671 |1 0|0|0|0 0 0 0|0|0 0 0 1 0 1 1|0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0| 672 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 673 |0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1| 674 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 675 |0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0| 676 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 678 Version: 2 679 Padding: 0 680 Extension: 0 681 Marker: 0 682 PTI: 11 683 SN: 8 684 TS: 3 685 SSRC: 2 687 Figure 3: RTP Header for Media Packet X 689 An FEC packet is generated from these two. We assume that payload 690 type 127 is used to indicate an FEC packet. The resulting RTP header 691 is shown in Figure 5. 693 The FEC header in the FEC packet is shown in Figure 6. 695 11 Use with Redundant Encodings 697 One can consider an FEC packet as a "redundant coding" of the media. 699 Media Packet y 701 0 1 2 3 702 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 703 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 704 |1 0|0|0|0 0 0 0|1|0 0 1 0 0 1 0|0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 1| 705 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 706 |0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1| 707 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 708 |0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0| 709 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 711 Version: 2 712 Padding: 0 713 Extension: 0 714 Marker: 1 715 PTI: 18 716 SN: 9 717 TS: 5 718 SSRC: 2 720 Figure 4: RTP Header for Media Packet Y 722 Because of this, the payload format for encoding of redundant audio 723 data [5] can be used to carry the FEC data along with the media. The 724 procedure for this is as follows. 726 The FEC operation defined above acts on a stream of RTP media 727 packets. The stream which is operated on is the stream before the 728 encapsulation defined in RFC2198 [5]. In other words, the media 729 stream to be protected is encapsulated in standard RTP media packets. 730 The FEC operation above is performed (with one minor change), 731 generating a stream of FEC packets. The change to the procedure above 732 is that if the RTP packets being protected contain an RTP extension, 733 padding, or a CSRC list, these MUST be removed from the packets, and 734 the CC field, Padding Bit, and Extension but MUST be set to zero, 735 before the FEC operation is applied. These modified packets are used 736 in the procedure above. Note that the sender MUST still send the 737 original packets (with the CSRC list, padding, and extension in tact) 738 as the primary encoding in RFC2198. The removal of these fields only 739 applies to the protection operation. 741 Once the FEC packets have been generated, the media payload is 742 extracted from the media packets. This payload is used as the primary 743 0 1 2 3 744 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 745 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 746 |1 0|0|0|0 0 0 0|1|1 1 1 1 1 1 1|0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1| 747 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 748 |0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1| 749 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 750 |0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0| 751 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 753 Version: 2 754 Padding: 0 755 Extension: 0 756 Marker: 1 757 PTI: 127 758 SN: 1 759 TS: 5 760 SSRC: 2 762 Figure 5: RTP Header of FEC for Packets X and Y 764 encoding as defined in rfc2198. Then, the FEC header and payload of 765 the FEC packets is extracted, and treated as a redundant encoding. 766 Additional redundant encodings, besides FEC, MAY be added to the 767 packet as well. These encodings will not be protected by FEC, 768 however. 770 The redundant encodings header for the primary codec is set as 771 defined in RFC2198. The redundant encodings header for the FEC data 772 is set as follows. The block PT is set to the dynamic PT associated 773 with the FEC format. The block length is set to the sum of the 774 lengths of the FEC header and payload. The timestamp offset SHOULD be 775 set to zero. The secondary coder payload includes the FEC header and 776 FEC payload. 778 At the receiver, the primary codec and all secondary codecs are 779 extracted as separate RTP packets. This is done by copying the 780 sequence number, SSRC, marker bit, CC field, RTP version, and 781 extension bit from the RTP header of the redundant encodings packet 782 to the RTP header of each extracted packet. If the secondary codec 783 contains FEC, the CC field, Extension Bit, and Padding Bit in the RTP 784 header of the FEC packet MUST be set to zero instead. The payload 785 type identifier in the extracted packet is copied from the block PT 786 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 787 |0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0|0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1| 788 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 789 |0|0 0 1 1 0 0 1|0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1| 790 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 791 |0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0| 792 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 794 SN base: 8 [min(8,9)] 795 len. rec.: 1 [8 xor 9] 796 E: 0 797 PTI rec.: 25 [11 xor 18] 798 mask: 3 799 TS rec.: 6 [3 xor 5] 801 The payload length is 11 bytes. 803 Figure 6: FEC Header of Result 805 of the redundant encodings header. The timestamp of the extracted 806 packet is the difference between the timestamp in the RTP header and 807 the offset in the block header. The payload of the extracted packet 808 is the data block. This will result in the FEC stream and media 809 stream being extracted. 811 To use the FEC and media packets for recovery, the CSRC list, 812 extension, and padding MUST be removed from the media packets, if 813 present, and the CC field, Extension Bit, and Padding Bit MUST be set 814 to zero. These modified media packets, along with the FEC packets, 815 are thn used to recover based on the procedures in section 9. The 816 recovered media packets will always have no extension, padding, or 817 CSRC list. An implementation MAY copy these fields into the recovered 818 packet from another media packet, if available. 820 Using the redundant encodings payload format also implies that the 821 marker bit may not be recovered correctly. Applications MUST set the 822 marker bit to zero in media packets reconstructed using FEC 823 encapsulated in RFC2198 redundancy. 825 An advantage of this approach is a reduction in the overhead for 826 sending FEC packets. 828 12 Indicating FEC Usage in SDP 829 FEC packets contain RTP packets with dynamic payload type values. In 830 addition, the FEC packets can be sent on separate multicast groups or 831 separate ports from the media. The FEC can even be carried in packets 832 containing media, using the redundant encodings payload format [5]. 833 These configuration options must be indicated out of band. This 834 section describes how this can be accomplished using the Session 835 Description Protocol (SDP), specified in RFC2327 [6]. 837 12.1 FEC as a Separate Stream 839 In the first case, the FEC packets are sent as a separate stream. 840 This can mean they are sent on a different port and/or multicast 841 group from the media. When this is done, several pieces of 842 information must be conveyed: 844 o The address and port where the FEC is being sent to 846 o The payload type number for the FEC 848 o Which media stream the FEC is protecting 850 The payload type number for the FEC is conveyed in the m line of the 851 media it is protecting, listed as if it were another valid encoding 852 for the stream. There is no static payload type assignment for FEC, 853 so dynamic payload type numbers MUST be used. The binding to the 854 number is indicated by an rtpmap attribute. The name used in this 855 binding is "parityfec". 857 The presence of the payload type number in the m line of the media it 858 is protecting does not mean the FEC is sent to the same address and 859 port as the media. Instead, this information is conveyed through an 860 fmtp attribute line. The presence of the FEC payload type on the m 861 line of the media serves only to indicate which stream the FEC is 862 protecting. 864 The format for the fmtp line for FEC is: 866 a=fmtp: 869 where 'number' is the payload type number present in the m line. Port 870 is the port number where the FEC is sent to. The remaining three 871 items - network type, address type, and connection address - have the 872 same syntax and semantics as the c line from SDP. This allows the 873 fmtp line to be partially parsed by the same parser used on the c 874 lines. Note that since FEC cannot be hierarchically encoded, the 875 parameter MUST NOT appear in the connection 876 address. 878 The following is an example SDP for FEC: 880 v=0 881 o=hamming 2890844526 2890842807 IN IP4 126.16.64.4 882 s=FEC Seminar 883 c=IN IP4 224.2.17.12/127 884 t=0 0 885 m=audio 49170 RTP/AVP 0 78 886 a=rtpmap:78 parityfec/8000 887 a=fmtp:78 49172 IN IP4 224.2.17.12/127 888 m=video 51372 RTP/AVP 31 79 889 a=rtpmap:79 parityfec/8000 890 a=fmtp:79 51372 IN IP4 224.2.17.13/127 892 The presence of two m lines in this SDP indicates that there are two 893 media streams - one audio and one video. The media format of 0 894 indicates that the audio uses PCM, and is protected by FEC with 895 payload type number 78. The FEC is sent to the same multicast group 896 and TTL as the audio, but on a port number two higher (49172). The 897 video is protected by FEC with payload type numer 79. The FEC appears 898 on the same port as the video (51372), but on a different multicast 899 address. 901 12.2 Use with Redundant Encodings 903 When the FEC stream is being sent as a secondary codec in the 904 redundant encodings format, this must be signaled through SDP. To do 905 this, the procedures defined in RFC2198 are used to signal the use of 906 redundant encodings. The FEC payload type is indicated in the same 907 fashion as any other secondary codec. An rtpmap attribute MUST be 908 used to indicate a dynamic payload type number for the FEC packets. 909 The FEC MUST protect only the main codec. In this case, the fmtp 910 attribute for the FEC MUST NOT be present. 912 For example: 914 m=audio 12345 RTP/AVP 121 0 5 100 915 a=rtpmap:121 red/8000/1 916 a=rtpmap:100 parityfec/8000 917 a=fmtp:121 0/5/100 918 This SDP indicates that there is a single audio stream, which can 919 consist of PCM (media format 0) , DVI (media format 5), the redundant 920 encodings (indicated by media format 121, which is bound to red 921 through the rtpmap attribute), or FEC (media format 100, which is 922 bound to parityfec through the rtpmap attribute). Although the FEC 923 format is specified as a possible coding for this stream, the FEC 924 MUST NOT be sent by itself for this stream. Its presence in the m 925 line is required only because non-primary codecs must be listed here 926 according to RFC2198. The fmtp attribute indicates that the redundant 927 encodings format can be used, with DVI as a secondary coding and FEC 928 as a tertiary encoding. 930 12.3 Usage with RTSP 932 RTSP [7] can be used to request FEC packets to be sent as a separate 933 stream. When SDP is used with RTSP, the Session Description does not 934 include a connection address and port number for each stream. 935 Instead, RTSP uses the concept of a "Control URL". Control URLs are 936 used in SDP in two distinct ways. 938 1. There is a single control URL for all streams. This is 939 referred to as "aggregate control". In this case, the fmtp 940 line for the FEC stream is omitted. 942 2. There is a Control URL assigned to each stream. This is 943 referred to as "non-aggregate control". In this case, the 944 fmtp line specifies the Control URL for the stream of FEC 945 packets. The URL may be used in a SETUP command by an RTSP 946 client. 948 The format for the fmtp line for FEC with RTSP and non-aggregate 949 control is: 951 a=fmtp: 953 where 'number' is the payload type number present in the m line. 954 Control URL is the URL used to control the stream of FEC packets. 955 Note that the Control URL does not need to be an absolute URL. The 956 rules for converting a relative Control URL to an absolute URL are 957 given in RFC-2326, Section C.1.1. 959 13 Security Considerations 961 The use of FEC has implications on the usage and changing of keys for 962 encryption. As the FEC packets do consist of a separate stream, there 963 are a number of permutations on the usage of encryption. In 964 particular: 966 o The FEC stream may be encrypted, while the media stream is 967 not. 969 o The media stream may be encrypted, while the FEC stream is 970 not. 972 o The media stream and FEC stream are both encrypted, but using 973 different keys. 975 o The media stream and FEC stream are both encrypted, but using 976 the same key. 978 The first three of these would require any application level 979 signaling protocols to be aware of the usage of FEC, and to thus 980 exchange keys for it and negotiate its usage on the media and FEC 981 streams separately. In the final case, no such additional mechanisms 982 are needed. The first two cases present a layering violation, as FEC 983 packets should really be treated no differently than other RTP 984 packets. Encrypting just one may also make certain known-plaintext 985 attacks possible. For these reasonse, applications utilizing 986 encryption SHOULD encrypt both streams. 988 However, the changing of keys becomes problematic. For example, if 989 two packets a and b are sent, and FEC packet f(a,b) is sent, and the 990 keys used for a and b are different, which key should be used to 991 decode f(a,b)? In general, old keys will likely need to be cached, so 992 that when the keys change for the media stream, the old key is kept, 993 and used, until it is determined that the key has changed on the FEC 994 packets as well. 996 Another issue with the use of FEC is its impact on network 997 congestion. Adding FEC in the face of increasing network losses is a 998 bad idea, as it can lead to increased congestion and eventual 999 congestion collapse if done on a widespread basis. As a result, 1000 implementors MUST NOT substantially increase the amount of FEC in use 1001 as network losses increase. 1003 14 Acknowledgments 1005 This work is based on an earlier draft on FEC, submitted by Budge and 1006 Mackenzie in 1997. We would also like to thank Steve Casner, Mark 1007 Handley, Orion Hodson and Colin Perkins for their comments. Thanks to 1008 Anders Klemets who wrote the section on usage with RTSP. 1010 15 Author's Addresses 1011 Jonathan Rosenberg 1012 Lucent Technologies, Bell Laboratories 1013 101 Crawfords Corner Rd. 1014 Holmdel, NJ 07733 1015 Rm. 4C-526 1016 email: jdrosen@bell-labs.com 1018 Henning Schulzrinne 1019 Columbia University 1020 M/S 0401 1021 1214 Amsterdam Ave. 1022 New York, NY 10027-7003 1023 email: schulzrinne@cs.columbia.edu 1025 16 Bibliography 1027 [1] J.-C. Bolot and A. V. Garcia, "Control mechanisms for packet 1028 audio in the internet," in Proceedings of the Conference on Computer 1029 Communications (IEEE Infocom) , (San Fransisco, Californialifornia), 1030 Mar. 1996. 1032 [2] C. Perkins and O. Hodson, "Options for repair of streaming 1033 media," Request for Comments (Informational) 2354, Internet 1034 Engineering Task Force, June 1998. 1036 [3] H. Schulzrinne, S. Casner, R. Frederick, and V. Jacobson, "RTP: a 1037 transport protocol for real-time applications," Request for Comments 1038 (Proposed Standard) 1889, Internet Engineering Task Force, Jan. 1996. 1040 [4] S. Bradner, "Key words for use in RFCs to indicate requirement 1041 levels," Request for Comments (Best Current Practice) 2119, Internet 1042 Engineering Task Force, Mar. 1997. 1044 [5] C. Perkins, I. Kouvelas, O. Hodson, V. Hardman, M. Handley, J. C. 1045 Bolot, A. Vega-Garcia, and S. Fosse-Parisis, "RTP payload for 1046 redundant audio data," Request for Comments (Proposed Standard) 2198, 1047 Internet Engineering Task Force, Sept. 1997. 1049 [6] M. Handley and V. Jacobson, "SDP: session description protocol," 1050 Request for Comments (Proposed Standard) 2327, Internet Engineering 1051 Task Force, Apr. 1998. 1053 [7] H. Schulzrinne, A. Rao, and R. Lanphier, "Real time streaming 1054 protocol (RTSP)," Request for Comments (Proposed Standard) 2326, 1055 Internet Engineering Task Force, Apr. 1998. 1057 Table of Contents 1059 1 Abstract ............................................ 1 1060 2 Introduction ........................................ 1 1061 3 Terminology ......................................... 2 1062 4 Basic Operation ..................................... 3 1063 5 Parity Codes ........................................ 4 1064 6 RTP Media Packet Structure .......................... 5 1065 7 FEC Packet Structure ................................ 6 1066 7.1 RTP Header of FEC Packets ........................... 6 1067 7.2 FEC Header .......................................... 7 1068 8 Protection Operation ................................ 8 1069 9 Recovery Procedures ................................. 9 1070 9.1 Reconstruction ...................................... 10 1071 9.2 Determination of When to Recover .................... 11 1072 10 Example ............................................. 15 1073 11 Use with Redundant Encodings ........................ 15 1074 12 Indicating FEC Usage in SDP ......................... 18 1075 12.1 FEC as a Separate Stream ............................ 19 1076 12.2 Use with Redundant Encodings ........................ 20 1077 12.3 Usage with RTSP ..................................... 21 1078 13 Security Considerations ............................. 21 1079 14 Acknowledgments ..................................... 22 1080 15 Author's Addresses .................................. 22 1081 16 Bibliography ........................................ 23