idnits 2.17.1 draft-ietf-avtcore-multiplex-guidelines-12.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 282 has weird spacing: '... vv vv ...' -- The document date (June 16, 2020) is 1404 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-13) exists of draft-ietf-perc-srtp-ekt-diet-11 -- Obsolete informational reference (is this intentional?): RFC 4566 (Obsoleted by RFC 8866) -- Obsolete informational reference (is this intentional?): RFC 5389 (Obsoleted by RFC 8489) Summary: 0 errors (**), 0 flaws (~~), 3 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group M. Westerlund 3 Internet-Draft B. Burman 4 Intended status: Informational Ericsson 5 Expires: December 18, 2020 C. Perkins 6 University of Glasgow 7 H. Alvestrand 8 Google 9 R. Even 10 June 16, 2020 12 Guidelines for using the Multiplexing Features of RTP to Support 13 Multiple Media Streams 14 draft-ietf-avtcore-multiplex-guidelines-12 16 Abstract 18 The Real-time Transport Protocol (RTP) is a flexible protocol that 19 can be used in a wide range of applications, networks, and system 20 topologies. That flexibility makes for wide applicability, but can 21 complicate the application design process. One particular design 22 question that has received much attention is how to support multiple 23 media streams in RTP. This memo discusses the available options and 24 design trade-offs, and provides guidelines on how to use the 25 multiplexing features of RTP to support multiple media streams. 27 Status of This Memo 29 This Internet-Draft is submitted in full conformance with the 30 provisions of BCP 78 and BCP 79. 32 Internet-Drafts are working documents of the Internet Engineering 33 Task Force (IETF). Note that other groups may also distribute 34 working documents as Internet-Drafts. The list of current Internet- 35 Drafts is at https://datatracker.ietf.org/drafts/current/. 37 Internet-Drafts are draft documents valid for a maximum of six months 38 and may be updated, replaced, or obsoleted by other documents at any 39 time. It is inappropriate to use Internet-Drafts as reference 40 material or to cite them other than as "work in progress." 42 This Internet-Draft will expire on December 18, 2020. 44 Copyright Notice 46 Copyright (c) 2020 IETF Trust and the persons identified as the 47 document authors. All rights reserved. 49 This document is subject to BCP 78 and the IETF Trust's Legal 50 Provisions Relating to IETF Documents 51 (https://trustee.ietf.org/license-info) in effect on the date of 52 publication of this document. Please review these documents 53 carefully, as they describe your rights and restrictions with respect 54 to this document. Code Components extracted from this document must 55 include Simplified BSD License text as described in Section 4.e of 56 the Trust Legal Provisions and are provided without warranty as 57 described in the Simplified BSD License. 59 Table of Contents 61 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 62 2. Definitions . . . . . . . . . . . . . . . . . . . . . . . . . 4 63 2.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 4 64 2.2. Subjects Out of Scope . . . . . . . . . . . . . . . . . . 5 65 3. RTP Multiplexing Overview . . . . . . . . . . . . . . . . . . 5 66 3.1. Reasons for Multiplexing and Grouping RTP Streams . . . . 5 67 3.2. RTP Multiplexing Points . . . . . . . . . . . . . . . . . 6 68 3.2.1. RTP Session . . . . . . . . . . . . . . . . . . . . . 7 69 3.2.2. Synchronisation Source (SSRC) . . . . . . . . . . . . 8 70 3.2.3. Contributing Source (CSRC) . . . . . . . . . . . . . 10 71 3.2.4. RTP Payload Type . . . . . . . . . . . . . . . . . . 11 72 3.3. Issues Related to RTP Topologies . . . . . . . . . . . . 12 73 3.4. Issues Related to RTP and RTCP Protocol . . . . . . . . . 13 74 3.4.1. The RTP Specification . . . . . . . . . . . . . . . . 13 75 3.4.2. Multiple SSRCs in a Session . . . . . . . . . . . . . 15 76 3.4.3. Binding Related Sources . . . . . . . . . . . . . . . 15 77 3.4.4. Forward Error Correction . . . . . . . . . . . . . . 17 78 4. Considerations for RTP Multiplexing . . . . . . . . . . . . . 17 79 4.1. Interworking Considerations . . . . . . . . . . . . . . . 17 80 4.1.1. Application Interworking . . . . . . . . . . . . . . 17 81 4.1.2. RTP Translator Interworking . . . . . . . . . . . . . 18 82 4.1.3. Gateway Interworking . . . . . . . . . . . . . . . . 19 83 4.1.4. Multiple SSRC Legacy Considerations . . . . . . . . . 20 84 4.2. Network Considerations . . . . . . . . . . . . . . . . . 20 85 4.2.1. Quality of Service . . . . . . . . . . . . . . . . . 20 86 4.2.2. NAT and Firewall Traversal . . . . . . . . . . . . . 21 87 4.2.3. Multicast . . . . . . . . . . . . . . . . . . . . . . 23 88 4.3. Security and Key Management Considerations . . . . . . . 24 89 4.3.1. Security Context Scope . . . . . . . . . . . . . . . 24 90 4.3.2. Key Management for Multi-party Sessions . . . . . . . 25 91 4.3.3. Complexity Implications . . . . . . . . . . . . . . . 26 92 5. RTP Multiplexing Design Choices . . . . . . . . . . . . . . . 26 93 5.1. Multiple Media Types in One Session . . . . . . . . . . . 26 94 5.2. Multiple SSRCs of the Same Media Type . . . . . . . . . . 28 95 5.3. Multiple Sessions for One Media Type . . . . . . . . . . 29 96 5.4. Single SSRC per Endpoint . . . . . . . . . . . . . . . . 30 97 5.5. Summary . . . . . . . . . . . . . . . . . . . . . . . . . 32 98 6. Guidelines . . . . . . . . . . . . . . . . . . . . . . . . . 32 99 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 33 100 8. Security Considerations . . . . . . . . . . . . . . . . . . . 33 101 9. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 34 102 10. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 34 103 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 34 104 11.1. Normative References . . . . . . . . . . . . . . . . . . 34 105 11.2. Informative References . . . . . . . . . . . . . . . . . 35 106 Appendix A. Dismissing Payload Type Multiplexing . . . . . . . . 39 107 Appendix B. Signalling Considerations . . . . . . . . . . . . . 40 108 B.1. Session Oriented Properties . . . . . . . . . . . . . . . 41 109 B.2. SDP Prevents Multiple Media Types . . . . . . . . . . . . 42 110 B.3. Signalling RTP Stream Usage . . . . . . . . . . . . . . . 42 111 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 43 113 1. Introduction 115 The Real-time Transport Protocol (RTP) [RFC3550] is a commonly used 116 protocol for real-time media transport. It is a protocol that 117 provides great flexibility and can support a large set of different 118 applications. RTP was from the beginning designed for multiple 119 participants in a communication session. It supports many topology 120 paradigms and usages, as defined in [RFC7667]. RTP has several 121 multiplexing points designed for different purposes. These enable 122 support of multiple RTP streams and switching between different 123 encoding or packetization of the media. By using multiple RTP 124 sessions, sets of RTP streams can be structured for efficient 125 processing or identification. Thus, an RTP application designer 126 needs to understand how to best use the RTP session, the RTP stream 127 identifier (SSRC), and the RTP payload type to meet the application's 128 needs. 130 There have been increased interest in more advanced usage of RTP. 131 For example, multiple RTP streams can be used when a single endpoint 132 has multiple media sources (like multiple cameras or microphones) 133 that need to be sent simultaneously. Consequently, questions are 134 raised regarding the most appropriate RTP usage. The limitations in 135 some implementations, RTP/RTCP extensions, and signalling have also 136 been exposed. This document aims to clarify the usefulness of some 137 functionalities in RTP which will hopefully result in more complete 138 implementations in the future. 140 The purpose of this document is to provide clear information about 141 the possibilities of RTP when it comes to multiplexing. The RTP 142 application designer needs to understand the implications arising 143 from a particular usage of the RTP multiplexing points. The document 144 will provide some guidelines and recommend against some usages as 145 being unsuitable, in general or for particular purposes. 147 The document starts with some definitions and then goes into the 148 existing RTP functionalities around multiplexing. Both the desired 149 behaviour and the implications of a particular behaviour depend on 150 which topologies are used, which requires some consideration. This 151 is followed by a discussion of some choices in multiplexing behaviour 152 and their impacts. Some designs of RTP usage are discussed. 153 Finally, some guidelines and examples are provided. 155 2. Definitions 157 2.1. Terminology 159 The definitions in Section 3 of [RFC3550] are referenced normatively. 161 The taxonomy defined in [RFC7656] is referenced normatively. 163 The following terms and abbreviations are used in this document: 165 Multiparty: A communication situation including multiple endpoints. 166 In this document, it will be used to refer to situations where 167 more than two endpoints communicate. 169 Multiplexing: The operation of taking multiple entities as input, 170 aggregating them onto some common resource while keeping the 171 individual entities addressable such that they can later be fully 172 and unambiguously separated (de-multiplexed) again. 174 RTP Receiver: An Endpoint or Middlebox receiving RTP streams and 175 RTCP messages. It uses at least one SSRC to send RTCP messages. 176 An RTP Receiver may also be an RTP Sender. 178 RTP Sender: An Endpoint sending one or more RTP streams, but also 179 sending RTCP messages. 181 RTP Session Group: One or more RTP sessions that are used together 182 to perform some function. Examples are multiple RTP sessions used 183 to carry different layers of a layered encoding. In an RTP 184 Session Group, CNAMEs are assumed to be valid across all RTP 185 sessions, and designate synchronisation contexts that can cross 186 RTP sessions; i.e. SSRCs that map to a common CNAME can be assumed 187 to have RTCP Sender Report (SR) timing information derived from a 188 common clock such that they can be synchronised for playout. 190 Signalling: The process of configuring endpoints to participate in 191 one or more RTP sessions. 193 Note: The above definitions of RTP Receiver and RTP Sender are 194 consistent with the usage in [RFC3550]. 196 2.2. Subjects Out of Scope 198 This document is focused on issues that affect RTP. Thus, issues 199 that involve signalling protocols, such as whether SIP [RFC3261], 200 Jingle [JINGLE] or some other protocol is in use for session 201 configuration, the particular syntaxes used to define RTP session 202 properties, or the constraints imposed by particular choices in the 203 signalling protocols, are mentioned only as examples in order to 204 describe the RTP issues more precisely. 206 This document assumes the applications will use RTCP. While there 207 are applications that don't send RTCP, they do not conform to the RTP 208 specification, and thus can be regarded as reusing the RTP packet 209 format but not implementing the RTP protocol. 211 3. RTP Multiplexing Overview 213 3.1. Reasons for Multiplexing and Grouping RTP Streams 215 There are several reasons why an endpoint might choose to send 216 multiple media streams. In the below discussion, please keep in mind 217 that the reasons for having multiple RTP streams vary and include but 218 are not limited to the following: 220 o Multiple media sources 222 o Multiple RTP streams might be needed to represent one media source 223 for instance: 225 * To carry different layers of an scalable encoding of a media 226 source 228 * Alternative encodings during simulcast, for instance using 229 different codecs for the same audio stream 231 * Alternative formats during simulcast, for instance multiple 232 resolutions of the same video stream 234 o A retransmission stream might repeat some parts of the content of 235 another RTP stream 237 o A Forward Error Correction (FEC) stream might provide material 238 that can be used to repair another RTP stream 240 For each of these reasons, it is necessary to decide if each 241 additional RTP stream is sent within the same RTP session as the 242 other RTP streams, or if it is necessary to use additional RTP 243 sessions to group the RTP streams. The choice suitable for one 244 situation, might not be the choice suitable in another situation or 245 combination of reasons. The clearest understanding is associated 246 with multiplexing multiple media sources of the same media type. 247 However, all reasons warrant discussion and clarification on how to 248 deal with them. As the discussion below will show, in reality we 249 cannot choose a single one of SSRC or RTP session multiplexing 250 solutions for all purposes. To utilise RTP well and as efficiently 251 as possible, both are needed. The real issue is finding the right 252 guidance on when to create additional RTP sessions and when 253 additional RTP streams in the same RTP session is the right choice. 255 3.2. RTP Multiplexing Points 257 This section describes the multiplexing points present in the RTP 258 protocol that can be used to distinguish RTP streams and groups of 259 RTP streams. Figure 1 outlines the process of demultiplexing 260 incoming RTP streams starting already at the socket representing 261 reception of one or more transport flows, e.g. based on the UDP 262 destination port. It also demultiplexes RTP/RTCP from any other 263 protocols, such as STUN [RFC5389] and DTLS-SRTP [RFC5764] on the same 264 transport as described in [RFC7983]. The Processing and Buffering 265 (PB) step of Figure 1 terminates the RTP/RTCP protocol and prepares 266 the RTP payload for input to the decoder. 268 | | | 269 | | | packets 270 +-- v v v 271 | +------------+ 272 | | Socket(s) | Transport Protocol Demultiplexing 273 | +------------+ 274 | || || 275 RTP | RTP/ || |+-----> DTLS (SRTP Keying, SCTP, etc) 276 Session | RTCP || +------> STUN (multiplexed using same port) 277 +-- || 278 +-- || 279 | ++(split by SSRC)-++---> Identify SSRC collision 280 | || || || || 281 | (associate with signalling by MID/RID) 282 | vv vv vv vv 283 RTP | +--+ +--+ +--+ +--+ Jitter buffer, 284 Streams | |PB| |PB| |PB| |PB| process RTCP, etc. 285 | +--+ +--+ +--+ +--+ 286 +-- | | | | 287 (select decoder based on PT) 288 +-- | / | / 289 | +-----+ | / 290 | / | |/ 291 Payload | v v v 292 Formats | +---+ +---+ +---+ 293 | |Dec| |Dec| |Dec| Decoders 294 | +---+ +---+ +---+ 295 +-- 297 Figure 1: RTP Demultiplexing Process 299 3.2.1. RTP Session 301 An RTP session is the highest semantic layer in the RTP protocol, and 302 represents an association between a group of communicating endpoints. 303 RTP does not contain a session identifier, yet different RTP sessions 304 must be possible to identify both across a set of different endpoints 305 and from the perspective of a single endpoint. 307 For RTP session separation across endpoints, the set of participants 308 that form an RTP session is defined as those that share a single 309 synchronisation source space [RFC3550]. That is, if a group of 310 participants are each aware of the synchronisation source identifiers 311 belonging to the other participants, then those participants are in a 312 single RTP session. A participant can become aware of a 313 synchronisation source identifier by receiving an RTP packet 314 containing it in the SSRC field or CSRC list, by receiving an RTCP 315 packet mentioning it in an SSRC field, or through signalling (e.g., 316 the Session Description Protocol (SDP) [RFC4566] "a=ssrc:" attribute 317 [RFC5576]). Thus, the scope of an RTP session is determined by the 318 participants' network interconnection topology, in combination with 319 RTP and RTCP forwarding strategies deployed by the endpoints and any 320 middleboxes, and by the signalling. 322 For RTP session separation within a single endpoint RTP relies on the 323 underlying transport layer, and on the signalling to identify RTP 324 sessions in a manner that is meaningful to the application. A single 325 endpoint can have one or more transport flows for the same RTP 326 session, and a single RTP session can span multiple transport layer 327 flows even if all endpoints use a single transport layer flow per 328 endpoint for that RTP session. The signalling layer might give RTP 329 sessions an explicit identifier, or the identification might be 330 implicit based on the addresses and ports used. Accordingly, a 331 single RTP session can have multiple associated identifiers, explicit 332 and implicit, belonging to different contexts. For example, when 333 running RTP on top of UDP/IP, an endpoint can identify and delimit an 334 RTP session from other RTP sessions by their UDP source and 335 destination IP addresses and UDP port numbers. A single RTP session 336 can be using multiple IP/UDP flows for receiving and/or sending RTP 337 packets to other endpoints or middleboxes, even if the endpoint does 338 not have multiple IP addresses. Using multiple IP addresses only 339 makes it more likely to require multiple IP/UDP flows. Another 340 example is SDP media descriptions (the "m=" line and the following 341 associated lines) that signal the transport flow and RTP session 342 configuration for the endpoint's part of the RTP session. The SDP 343 grouping framework [RFC5888] allows labeling of the media 344 descriptions to be used so that RTP Session Groups can be created. 345 Through use of Negotiating Media Multiplexing Using the Session 346 Description Protocol (SDP) [I-D.ietf-mmusic-sdp-bundle-negotiation], 347 multiple media descriptions become part of a common RTP session where 348 each media description represents the RTP streams sent or received 349 for a media source. 351 The RTP protocol makes no normative statements about the relationship 352 between different RTP sessions, however the applications that use 353 more than one RTP session will have some higher layer understanding 354 of the relationship between the sessions they create. 356 3.2.2. Synchronisation Source (SSRC) 358 A synchronisation source (SSRC) identifies a source of an RTP stream, 359 or an RTP receiver when sending RTCP. Every endpoint has at least 360 one SSRC identifier, even if it does not send RTP packets. RTP 361 endpoints that are only RTP receivers still send RTCP and use their 362 SSRC identifiers in the RTCP packets they send. An endpoint can have 363 multiple SSRC identifiers if it sends multiple RTP streams. 364 Endpoints that are both RTP sender and RTP receiver use the same 365 SSRC(s) in both roles. 367 The SSRC is a 32-bit identifier. It is present in every RTP and RTCP 368 packet header, and in the payload of some RTCP packet types. It can 369 also be present in SDP signalling. Unless pre-signalled, e.g. using 370 the SDP "a=ssrc:" attribute [RFC5576], the SSRC is chosen at random. 371 It is not dependent on the network address of the endpoint, and is 372 intended to be unique within an RTP session. SSRC collisions can 373 occur, and are handled as specified in [RFC3550] and [RFC5576], 374 resulting in the SSRC of the colliding RTP streams or receivers 375 changing. An endpoint that changes its network transport address 376 during a session has to choose a new SSRC identifier to avoid being 377 interpreted as looped source, unless a mechanism providing a virtual 378 transport (such as ICE [RFC8445]) abstracts the changes. 380 SSRC identifiers that belong to the same synchronisation context 381 (i.e., that represent RTP streams that can be synchronised using 382 information in RTCP SR packets) use identical CNAME chunks in 383 corresponding RTCP SDES packets. SDP signalling can also be used to 384 provide explicit SSRC grouping [RFC5576]. 386 In some cases, the same SSRC identifier value is used to relate 387 streams in two different RTP sessions, such as in RTP retransmission 388 [RFC4588]. This is to be avoided since there is no guarantee that 389 SSRC values are unique across RTP sessions. For the RTP 390 retransmission [RFC4588] case it is recommended to use explicit 391 binding of the source RTP stream and the redundancy stream, e.g. 392 using the RepairedRtpStreamId RTCP SDES item [I-D.ietf-avtext-rid]. 393 The RepairedRtpStreamId is a rather recent mechanism, so one cannot 394 expect older applications to follow this recommendation. 396 Note that RTP sequence number and RTP timestamp are scoped by the 397 SSRC and thus specific per RTP stream. 399 Different types of entities use an SSRC to identify themselves, as 400 follows: 402 A real media source: Uses the SSRC to identify a "physical" media 403 source. 405 A conceptual media source: Uses the SSRC to identify the result of 406 applying some filtering function in a network node, for example a 407 filtering function in an RTP mixer that provides the most active 408 speaker based on some criteria, or a mix representing a set of 409 other sources. 411 An RTP receiver: Uses the SSRC to identify itself as the source of 412 its RTCP reports. 414 An endpoint that generates more than one media type, e.g. a 415 conference participant sending both audio and video, need not (and, 416 indeed, should not) use the same SSRC value across RTP sessions. 417 RTCP compound packets containing the CNAME SDES item is the 418 designated method to bind an SSRC to a CNAME, effectively cross- 419 correlating SSRCs within and between RTP Sessions as coming from the 420 same endpoint. The main property attributed to SSRCs associated with 421 the same CNAME is that they are from a particular synchronisation 422 context and can be synchronised at playback. 424 An RTP receiver receiving a previously unseen SSRC value will 425 interpret it as a new source. It might in fact be a previously 426 existing source that had to change SSRC number due to an SSRC 427 conflict. Use of the MID extension 428 [I-D.ietf-mmusic-sdp-bundle-negotiation] helps to identify which 429 media source the new SSRC represents and use of the RID extension 430 [I-D.ietf-mmusic-rid] helps to identify what encoding or redundancy 431 stream it represents, even though the SSRC changed. However, the 432 originator of the previous SSRC ought to have ended the conflicting 433 source by sending an RTCP BYE for it prior to starting to send with 434 the new SSRC, making the new SSRC a new source. 436 3.2.3. Contributing Source (CSRC) 438 The Contributing Source (CSRC) is not a separate identifier. Rather 439 an SSRC identifier is listed as a CSRC in the RTP header of a packet 440 generated by an RTP mixer or video MCU/switch, if the corresponding 441 SSRC was in the header of one of the packets that contributed to the 442 output. 444 It is not possible, in general, to extract media represented by an 445 individual CSRC since it is typically the result of a media merge 446 (e.g. mix) operation on the individual media streams corresponding to 447 the CSRC identifiers. The exception is the case when only a single 448 CSRC is indicated as this represent forwarding of an RTP stream, 449 possibly modified. The RTP header extension for Mixer-to-Client 450 Audio Level Indication [RFC6465] expands on the receiver's 451 information about a packet with a CSRC list. Due to these 452 restrictions, CSRC will not be considered a fully qualified 453 multiplexing point and will be disregarded in the rest of this 454 document. 456 3.2.4. RTP Payload Type 458 Each RTP stream utilises one or more RTP payload formats. An RTP 459 payload format describes how the output of a particular media codec 460 is framed and encoded into RTP packets. The payload format is 461 identified by the payload type (PT) field in the RTP packet header. 462 The combination of SSRC and PT therefore identifies a specific RTP 463 stream in a specific encoding format. The format definition can be 464 taken from [RFC3551] for statically allocated payload types, but 465 ought to be explicitly defined in signalling, such as SDP, both for 466 static and dynamic payload types. The term "format" here includes 467 those aspects described by out-of-band signalling means; in SDP, the 468 term "format" includes media type, RTP timestamp sampling rate, 469 codec, codec configuration, payload format configurations, and 470 various robustness mechanisms such as redundant encodings [RFC2198]. 472 The RTP payload type is scoped by the sending endpoint within an RTP 473 session. PT has the same meaning across all RTP streams in an RTP 474 session. All SSRCs sent from a single endpoint share the same 475 payload type definitions. The RTP payload type is designed such that 476 only a single payload type is valid at any time instant in the RTP 477 stream's timestamp time line, effectively time-multiplexing different 478 payload types if any change occurs. The payload type can change on a 479 per-packet basis for an SSRC, for example a speech codec making use 480 of generic comfort noise [RFC3389]. If there is a true need to send 481 multiple payload types for the same SSRC that are valid for the same 482 instant, then redundant encodings [RFC2198] can be used. Several 483 additional constraints than the ones mentioned above need to be met 484 to enable this use, one of which is that the combined payload sizes 485 of the different payload types ought not exceed the transport MTU. 487 Other aspects of RTP payload format use are described in How to Write 488 an RTP Payload Format [RFC8088]. 490 The payload type is not a multiplexing point at the RTP layer (see 491 Appendix A for a detailed discussion of why using the payload type as 492 an RTP multiplexing point does not work). The RTP payload type is, 493 however, used to determine how to consume and decode an RTP stream. 494 The RTP payload type number is sometimes used to associate an RTP 495 stream with the signalling, which in general requires that unique RTP 496 payload type numbers are used in each context. Use of MID, e.g. when 497 bundling "m=" sections [I-D.ietf-mmusic-sdp-bundle-negotiation], can 498 replace the payload type as signalling association and unique RTP 499 payload types are then no longer required for that purpose. 501 3.3. Issues Related to RTP Topologies 503 The impact of how RTP multiplexing is performed will in general vary 504 with how the RTP session participants are interconnected, described 505 by RTP Topology [RFC7667]. 507 Even the most basic use case, denoted Topo-Point-to-Point in 508 [RFC7667], raises a number of considerations that are discussed in 509 detail in following sections. They range over such aspects as: 511 o Does my communication peer support RTP as defined with multiple 512 SSRCs per RTP session? 514 o Do I need network differentiation in form of QoS (Section 4.2.1)? 516 o Can the application more easily process and handle the media 517 streams if they are in different RTP sessions? 519 o Do I need to use additional RTP streams for RTP retransmission or 520 FEC? 522 For some point to multi-point topologies (e.g. Topo-ASM and Topo-SSM 523 in [RFC7667]), multicast is used to interconnect the session 524 participants. Special considerations (documented in Section 4.2.3) 525 are then needed as multicast is a one-to-many distribution system. 527 Sometimes an RTP communication can end up in a situation when the 528 communicating peers are not compatible for various reasons: 530 o No common media codec for a media type thus requiring transcoding. 532 o Different support for multiple RTP streams and RTP sessions. 534 o Usage of different media transport protocols, i.e., RTP or other. 536 o Usage of different transport protocols, e.g., UDP, DCCP, or TCP. 538 o Different security solutions, e.g., IPsec, TLS, DTLS, or SRTP with 539 different keying mechanisms. 541 In many situations this is resolved by the inclusion of a translator 542 between the two peers, as described by Topo-PtP-Translator in 543 [RFC7667]. The translator's main purpose is to make the peers look 544 compatible to each other. There can also be other reasons than 545 compatibility to insert a translator in the form of a middlebox or 546 gateway, for example a need to monitor the RTP streams. Beware that 547 changing the stream transport characteristics in the translator can 548 require thorough understanding of aspects from congestion control and 549 media adaptation to application-layer semantics. 551 Within the uses enabled by the RTP standard, the point to point 552 topology can contain one or more RTP sessions with one or more media 553 sources per session, each having one or more RTP streams per media 554 source. 556 3.4. Issues Related to RTP and RTCP Protocol 558 Using multiple RTP streams is a well-supported feature of RTP. 559 However, for most implementers or people writing RTP/RTCP 560 applications or extensions attempting to apply multiple streams, it 561 can be unclear when it is most appropriate to add an additional RTP 562 stream in an existing RTP session and when it is better to use 563 multiple RTP sessions. This section discusses the various 564 considerations needed. 566 3.4.1. The RTP Specification 568 RFC 3550 contains some recommendations and a bullet list with 5 569 arguments for different aspects of RTP multiplexing. Please review 570 Section 5.2 of [RFC3550]. Five important aspects are quoted below. 572 1. If, say, two audio streams shared the same RTP session and the 573 same SSRC value, and one were to change encodings and thus acquire 574 a different RTP payload type, there would be no general way of 575 identifying which stream had changed encodings. 577 The first argument is to use different SSRC for each individual RTP 578 stream, which is fundamental to RTP operation. 580 2. An SSRC is defined to identify a single timing and sequence number 581 space. Interleaving multiple payload types would require 582 different timing spaces if the media clock rates differ and would 583 require different sequence number spaces to tell which payload 584 type suffered packet loss. 586 The second argument is advocating against demultiplexing RTP streams 587 within a session based only on their RTP payload type numbers, which 588 still stands as can been seen by the extensive list of issues found 589 in Appendix A. 591 3. The RTCP sender and receiver reports (see Section 6.4) can only 592 describe one timing and sequence number space per SSRC and do not 593 carry a payload type field. 595 The third argument is yet another argument against payload type 596 multiplexing. 598 4. An RTP mixer would not be able to combine interleaved streams of 599 incompatible media into one stream. 601 The fourth argument is against multiplexing RTP packets that require 602 different handling into the same session. In most cases the RTP 603 mixer must embed application logic to handle streams; the separation 604 of streams according to stream type is just another piece of 605 application logic, which might or might not be appropriate for a 606 particular application. One type of application that can mix 607 different media sources blindly is the audio-only telephone bridge, 608 although the ability to do that comes from the well-defined scenario 609 that is aided by use of a single media type, even though individual 610 streams may use incompatible codec types; most other types of 611 applications need application-specific logic to perform the mix 612 correctly. 614 5. Carrying multiple media in one RTP session precludes: the use of 615 different network paths or network resource allocations if 616 appropriate; reception of a subset of the media if desired, for 617 example just audio if video would exceed the available bandwidth; 618 and receiver implementations that use separate processes for the 619 different media, whereas using separate RTP sessions permits 620 either single- or multiple-process implementations. 622 The fifth argument discusses network aspects that are described in 623 Section 4.2. It also goes into aspects of implementation, like Split 624 Component Terminal (see Section 3.10 of [RFC7667]) endpoints where 625 different processes or inter-connected devices handle different 626 aspects of the whole multi-media session. 628 A summary of RFC 3550's view on multiplexing is to use unique SSRCs 629 for anything that is its own media/packet stream, and to use 630 different RTP sessions for media streams that don't share a media 631 type. This document supports the first point; it is very valid. The 632 latter needs further discussion, as imposing a single solution on all 633 usages of RTP is inappropriate. "Multiple Media Types in an RTP 634 Session specification" [I-D.ietf-avtcore-multi-media-rtp-session] 635 updates RFC 3550 to allow multiple media types in a RTP session. It 636 also provides a detailed analysis of the potential benefits and 637 issues in having multiple media types in the same RTP session. Thus, 638 that document provides a wider scope for an RTP session and considers 639 multiple media types in one RTP session as a possible choice for the 640 RTP application designer. 642 3.4.2. Multiple SSRCs in a Session 644 Using multiple SSRCs at one endpoint in an RTP session requires 645 resolving some unclear aspects of the RTP specification. These could 646 potentially lead to some interoperability issues as well as some 647 potential significant inefficiencies, as further discussed in "RTP 648 Considerations for Endpoints Sending Multiple Media Streams" 649 [RFC8108]. An RTP application designer should consider these issues 650 and the possible application impact from lack of appropriate RTP 651 handling or optimization in the peer endpoints. 653 Using multiple RTP sessions can potentially mitigate application 654 issues caused by multiple SSRCs in an RTP session. 656 3.4.3. Binding Related Sources 658 A common problem in a number of various RTP extensions has been how 659 to bind related RTP streams together. This issue is common to both 660 using additional SSRCs and multiple RTP sessions. 662 The solutions can be divided into a few groups: 664 o RTP/RTCP based 666 o Signalling based, e.g. SDP 668 o Grouping related RTP sessions 670 o Grouping SSRCs within an RTP session 672 Most solutions are explicit, but some implicit methods have also been 673 applied to the problem. 675 The SDP-based signalling solutions are: 677 SDP Media Description Grouping: The SDP Grouping Framework [RFC5888] 678 uses various semantics to group any number of media descriptions. 679 This has primarily been grouping RTP sessions, but in combination 680 with [I-D.ietf-mmusic-sdp-bundle-negotiation] it can also group 681 multiple media descriptions within a single RTP session. 683 SDP Media Multiplexing: Negotiating Media Multiplexing Using the 684 Session Description Protocol (SDP) 685 [I-D.ietf-mmusic-sdp-bundle-negotiation] 686 uses both SDP and RTCP information to associate RTP streams to SDP 687 media descriptions. This allows both to group RTP streams 688 belonging to an SDP media description, and to group multiple SDP 689 media descriptions into a single RTP session. 691 SDP SSRC grouping: Source-Specific Media Attributes in SDP [RFC5576] 692 includes a solution for grouping SSRCs the same way as the 693 Grouping framework groups Media Descriptions. 695 The above grouping constructs support many use cases. Those 696 solutions have shortcomings in cases where the session's dynamic 697 properties are such that it is difficult or a drain on resources to 698 keep the list of related SSRCs up to date. 700 An RTP/RTCP-based grouping solution is to use the RTCP SDES CNAME to 701 bind related RTP streams to an endpoint or to a synchronization 702 context. For applications with a single RTP stream per type (media, 703 source or redundancy stream), CNAME is sufficient for that purpose 704 independently of whether one or more RTP sessions are used. However, 705 some applications choose not to use CNAME because of perceived 706 complexity or a desire not to implement RTCP and instead use the same 707 SSRC value to bind related RTP streams across multiple RTP sessions. 708 RTP Retransmission [RFC4588] in multiple RTP session mode and Generic 709 FEC [RFC5109] both use the CNAME method to relate the RTP streams, 710 which may work but might have some downsides in RTP sessions with 711 many participating SSRCs. It is not recommended to use identical 712 SSRC values across RTP sessions to relate RTP streams; When an SSRC 713 collision occurs, this will force change of that SSRC in all RTP 714 sessions and thus resynchronize all of them instead of only the 715 single media stream having the collision. 717 Another method to implicitly bind SSRCs is used by RTP Retransmission 718 [RFC4588] when using the same RTP session as the source RTP stream 719 for retransmissions. The receiver missing a packet issues an RTP 720 retransmission request, and then awaits a new SSRC carrying the RTP 721 retransmission payload and where that SSRC is from the same CNAME. 722 This limits a requester to having only one outstanding retransmission 723 request on any new source SSRCs per endpoint. 725 RTP Payload Format Restrictions [I-D.ietf-mmusic-rid] provides an 726 RTP/RTCP based mechanism to unambiguously identify the RTP streams 727 within an RTP session and restrict the streams' payload format 728 parameters in a codec-agnostic way beyond what is provided with the 729 regular payload types. The mapping is done by specifying an "a=rid" 730 value in the SDP offer/answer signalling and having the corresponding 731 RtpStreamId value as an SDES item and an RTP header extension. The 732 RID solution also includes a solution for binding redundancy RTP 733 streams to their original source RTP streams, given that those use 734 RID identifiers. 736 Experience has found that an explicit binding between the RTP 737 streams, agnostic of SSRC values, behaves well. That way, solutions 738 using multiple RTP streams in a single RTP session and in multiple 739 RTP sessions will use the same type of binding. 741 3.4.4. Forward Error Correction 743 There exist a number of Forward Error Correction (FEC) based schemes 744 for how to mitigate packet loss in the original streams. Most of the 745 FEC schemes protect a single source flow. The protection is achieved 746 by transmitting a certain amount of redundant information that is 747 encoded such that it can repair one or more packet losses over the 748 set of packets the redundant information protects. This sequence of 749 redundant information needs to be transmitted as its own media 750 stream, or in some cases, instead of the original media stream. 751 Thus, many of these schemes create a need for binding related flows 752 as discussed above. Looking at the history of these schemes, there 753 are schemes using multiple SSRCs and schemes using multiple RTP 754 sessions, and some schemes that support both modes of operation. 756 Using multiple RTP sessions supports the case where some set of 757 receivers might not be able to utilise the FEC information. By 758 placing it in a separate RTP session and if separating RTP sessions 759 on transport level, FEC can easily be ignored already on the 760 transport level, without considering any RTP layer information. 762 In usages involving multicast, having the FEC information on its own 763 multicast group allows for similar flexibility. This is especially 764 useful when receivers see heterogeneous packet loss rates. A 765 receiver can decide, based on measurement of experienced packet loss 766 rates, whether to join a multicast group with the suitable FEC data 767 repair capabilities. 769 4. Considerations for RTP Multiplexing 771 4.1. Interworking Considerations 773 There are several different kinds of interworking, and this section 774 discusses two; interworking directly between different applications, 775 and interworking of applications through an RTP Translator. The 776 discussion includes the implications of potentially different RTP 777 multiplexing point choices and limitations that have to be considered 778 when working with some legacy applications. 780 4.1.1. Application Interworking 782 It is not uncommon that applications or services of similar but not 783 identical usage, especially the ones intended for interactive 784 communication, encounter a situation where one want to interconnect 785 two or more of these applications. 787 In these cases, one ends up in a situation where one might use a 788 gateway to interconnect applications. This gateway must then either 789 change the multiplexing structure or adhere to the respective 790 limitations in each application. 792 There are two fundamental approaches to building a gateway: using RTP 793 Translator interworking (RTP bridging), where the gateway acts as an 794 RTP Translator with the two interconnected applications being members 795 of the same RTP session; or using Gateway Interworking with RTP 796 termination, where there are independent RTP sessions between each 797 interconnected application and the gateway. 799 For interworking to be feasible, any security solution in use needs 800 to be compatible and capable of exchanging keys with either the peer 801 or the gateway under the used trust model. Secondly, the 802 applications need to use media streams in a way that makes sense in 803 both applications. 805 4.1.2. RTP Translator Interworking 807 From an RTP perspective, the RTP Translator approach could work if 808 all the applications are using the same codecs with the same payload 809 types, have made the same multiplexing choices, and have the same 810 capabilities in number of simultaneous RTP streams combined with the 811 same set of RTP/RTCP extensions being supported. Unfortunately, this 812 might not always be true. 814 When a gateway is implemented via an RTP Translator, an important 815 consideration is if the two applications being interconnected need to 816 use the same approach to multiplexing. If one side is using RTP 817 session multiplexing and the other is using SSRC multiplexing with 818 BUNDLE [I-D.ietf-mmusic-sdp-bundle-negotiation], it may be possible 819 for the RTP translator to map the RTP streams between both sides 820 using some method, e.g. based on the number and order of SDP "m=" 821 lines from each side. There are also challenges with SSRC collision 822 handling since, unless SSRC translation is applied on the RTP 823 translator, there may be a collision on the SSRC multiplexing side 824 that the RTP session multiplexing side will not be aware of. 825 Furthermore, if one of the applications is capable of working in 826 several modes (such as being able to use additional RTP streams in 827 one RTP session or multiple RTP sessions at will), and the other one 828 is not, successful interconnection depends on locking the more 829 flexible application into the operating mode where interconnection 830 can be successful, even if none of the participants are using the 831 less flexible application when the RTP sessions are being created. 833 4.1.3. Gateway Interworking 835 When one terminates RTP sessions at the gateway, there are certain 836 tasks that the gateway has to carry out: 838 o Generating appropriate RTCP reports for all RTP streams (possibly 839 based on incoming RTCP reports), originating from SSRCs controlled 840 by the gateway. 842 o Handling SSRC collision resolution in each application's RTP 843 sessions. 845 o Signalling, choosing, and policing appropriate bit-rates for each 846 session. 848 For applications that use any security mechanism, e.g., in the form 849 of SRTP, the gateway needs to be able to decrypt and verify source 850 integrity of the incoming packets, and re-encrypt, integrity protect, 851 and sign the packets as the peer in the other application's security 852 context. This is necessary even if all that's needed is a simple 853 remapping of SSRC numbers. If this is done, the gateway also needs 854 to be a member of the security contexts of both sides, and thus a 855 trusted entity. 857 The gateway might also need to apply transcoding (for incompatible 858 codec types), media-level adaptations that cannot be solved through 859 media negotiation (such as rescaling for incompatible video size 860 requirements), suppression of content that is known not to be handled 861 in the destination application, or the addition or removal of 862 redundancy coding or scalability layers to fit the needs of the 863 destination domain. 865 From the above, we can see that the gateway needs to have an intimate 866 knowledge of the application requirements; a gateway is by its nature 867 application specific, not a commodity product. 869 These gateways might therefore potentially block application 870 evolution by blocking RTP and RTCP extensions that the applications 871 have been extended with but that are unknown to the gateway. 873 If one uses security mechanism, like SRTP, the gateway and the 874 necessary trust in it by the peers is an additional risk to the 875 communication security. The gateway also incur additional 876 complexities in form of the decrypt-encrypt cycles needed for each 877 forwarded packet. SRTP, due to its keying structure, also requires 878 that each RTP session needs different master keys, as use of the same 879 key in two RTP sessions can for some ciphers result in a reuse of a 880 one-time pad that completely breaks the confidentiality of the 881 packets. 883 4.1.4. Multiple SSRC Legacy Considerations 885 Historically, the most common RTP use cases have been point-to-point 886 Voice over IP (VoIP) or streaming applications, commonly with no more 887 than one media source per endpoint and media type (typically audio or 888 video). Even in conferencing applications, especially voice-only, 889 the conference focus or bridge has provided a single stream to each 890 participant containing a mix of the other participants. It is also 891 common to have individual RTP sessions between each endpoint and the 892 RTP mixer, meaning that the mixer functions as an RTP-terminating 893 gateway. 895 Applications and systems that aren't updated to handle multiple 896 streams following these recommendations can have issues with 897 participating in RTP sessions containing multiple SSRCs within a 898 single session, such as: 900 1. Need to handle more than one stream simultaneously rather than 901 replacing an already existing stream with a new one. 903 2. Be capable of decoding multiple streams simultaneously. 905 3. Be capable of rendering multiple streams simultaneously. 907 This indicates that gateways attempting to interconnect to this class 908 of devices have to make sure that only one RTP stream of each media 909 type gets delivered to the endpoint if it's expecting only one, and 910 that the multiplexing format is what the device expects. It is 911 highly unlikely that RTP translator-based interworking can be made to 912 function successfully in such a context. 914 4.2. Network Considerations 916 The RTP implementer needs to consider that the RTP multiplexing 917 choice also impacts network level mechanisms. 919 4.2.1. Quality of Service 921 Quality of Service mechanisms are either flow based or packet marking 922 based. RSVP [RFC2205] is an example of a flow based mechanism, while 923 Diff-Serv [RFC2474] is an example of a packet marking based one. 925 For a flow based scheme, additional SSRC will receive the same QoS as 926 all other RTP streams being part of the same 5-tuple (protocol, 927 source address, destination address, source port, destination port), 928 which is the most common selector for flow based QoS. 930 For a packet marking based scheme, the method of multiplexing will 931 not affect the possibility to use QoS. Different Differentiated 932 Services Code Points (DSCP) can be assigned to different packets 933 within a transport flow (5-Tuple) as well as within an RTP stream, 934 assuming usage of UDP or other transport protocol that do not have 935 issues with packet reordering within the transport flow (5-tuple). 936 To avoid packet reording issues, packets belonging to the same RTP 937 flow should limits its use of DSCP to those whose corresponding Per 938 Hop Behavior (PHB) that do not enable reordering. If the transport 939 protocol used assumes in order delivery of packet, such as TCP and 940 SCTP, then a single DSCP should be used. For more discussion of this 941 see [RFC7657]. 943 The method for assigning marking to packets can impact what number of 944 RTP sessions to choose. If this marking is done using a network 945 ingress function, it can have issues discriminating the different RTP 946 streams. The network API on the endpoint also needs to be capable of 947 setting the marking on a per-packet basis to reach the full 948 functionality. 950 4.2.2. NAT and Firewall Traversal 952 In today's networks there exist a large number of middleboxes. The 953 ones that normally have most impact on RTP are Network Address 954 Translators (NAT) and Firewalls (FW). 956 Below we analyse and comment on the impact of requiring more 957 underlying transport flows in the presence of NATs and Firewalls: 959 End-Point Port Consumption: A given IP address only has 65536 960 available local ports per transport protocol for all consumers of 961 ports that exist on the machine. This is normally never an issue 962 for an end-user machine. It can become an issue for servers that 963 handle large number of simultaneous streams. However, if the 964 application uses ICE to authenticate STUN requests, a server can 965 serve multiple endpoints from the same local port, and use the 966 whole 5-tuple (source and destination address, source and 967 destination port, protocol) as identifier of flows after having 968 securely bound them to the remote endpoint address using the STUN 969 request. In theory, the minimum number of media server ports 970 needed are the maximum number of simultaneous RTP sessions a 971 single endpoint can use. In practice, implementation will 972 probably benefit from using more server ports to simplify 973 implementation or avoid performance bottlenecks. 975 NAT State: If an endpoint sits behind a NAT, each flow it generates 976 to an external address will result in a state that has to be kept 977 in the NAT. That state is a limited resource. In home or Small 978 Office/Home Office (SOHO) NATs, memory or processing are usually 979 the most limited resources. For large scale NATs serving many 980 internal endpoints, available external ports are likely the scarce 981 resource. Port limitations is primarily a problem for larger 982 centralised NATs where endpoint independent mapping requires each 983 flow to use one port for the external IP address. This affects 984 the maximum number of internal users per external IP address. 985 However, as a comparison, a real-time video conference session 986 with audio and video likely uses less than 10 UDP flows, compared 987 to certain web applications that can use 100+ TCP flows to various 988 servers from a single browser instance. 990 NAT Traversal Extra Delay: Performing the NAT/FW traversal takes a 991 certain amount of time for each flow. It also takes time in a 992 phase of communication between accepting to communicate and the 993 media path being established, which is fairly critical. The best 994 case scenario for additional NAT/FW traversal time after finding 995 the first valid candidate pair following the specified ICE 996 procedures is 1.5*RTT + Ta*(Additional_Flows-1), where Ta is the 997 pacing timer. That assumes a message in one direction, 998 immediately followed by a check back. The reason it isn't more, 999 is that ICE first finds one candidate pair that works prior to 1000 attempting to establish multiple flows. Thus, there is no extra 1001 time until one has found a working candidate pair. Based on that 1002 working pair, the extra time is needed to in parallel establish 1003 the, in most cases 2-3, additional flows. However, packet loss 1004 causes extra delays, at least 500 ms, which is the minimal 1005 retransmission timer for ICE. 1007 NAT Traversal Failure Rate: Due to the need to establish more than a 1008 single flow through the NAT, there is some risk that establishing 1009 the first flow succeeds but that one or more of the additional 1010 flows fail. The risk that this happens is hard to quantify, but 1011 ought to be fairly low as one flow from the same interfaces has 1012 just been successfully established. Thus only rare events such as 1013 NAT resource overload, or selecting particular port numbers that 1014 are filtered etc., ought to be reasons for failure. 1016 Deep Packet Inspection and Multiple Streams: Firewalls differ in how 1017 deeply they inspect packets. Due to all previous issues with 1018 firewall and Session Boarder Gateways (SBG) with RTP transport 1019 media e.g. in Voice over IP (VoIP) systems, there exists a 1020 significant risk that deeply inspecting firewalls will have 1021 similar legacy issues with multiple SSRCs as some RTP stack 1022 implementations. 1024 Using additional RTP streams in the same RTP session and transport 1025 flow does not introduce any additional NAT traversal complexities per 1026 RTP stream. This can be compared with normally one or two additional 1027 transport flows per RTP session when using multiple RTP sessions. 1028 Additional lower layer transport flows will be needed, unless an 1029 explicit de-multiplexing layer is added between RTP and the transport 1030 protocol. At time of writing no such mechanism was defined. 1032 4.2.3. Multicast 1034 Multicast groups provides a powerful tool for a number of real-time 1035 applications, especially the ones that desire broadcast-like 1036 behaviours with one endpoint transmitting to a large number of 1037 receivers, like in IPTV. There is also the RTP/RTCP extension to 1038 better support Source Specific Multicast (SSM) [RFC5760]. Many-to- 1039 many communication, which RTP [RFC3550] was originally built to 1040 support, has several limitations in common with multicast. 1042 One limitation is that, for any group, sender side adaptation with 1043 the intent to suit all receivers would have to adapt to the most 1044 limited receiver experiencing the worst conditions among the group 1045 participants, which imposes degradation for all participants. For 1046 broadcast-type applications with a large number of receivers, this is 1047 not acceptable. Instead, various receiver-based solutions are 1048 employed to ensure that the receivers achieve best possible 1049 performance. By using scalable encoding and placing each scalability 1050 layer in a different multicast group, the receiver can control the 1051 amount of traffic it receives. To have each scalability layer on a 1052 different multicast group, one RTP session per multicast group is 1053 used. 1055 In addition, the transport flow considerations in multicast are a bit 1056 different from unicast; NATs with port translation are not useful in 1057 the multicast environment, meaning that the entire port range of each 1058 multicast address is available for distinguishing between RTP 1059 sessions. 1061 Thus, when using broadcast applications it appears easiest and most 1062 straightforward to use multiple RTP sessions for sending different 1063 media flows used for adapting to network conditions. It is also 1064 common that streams improving transport robustness are sent in their 1065 own multicast group to allow for interworking with legacy or to 1066 support different levels of protection. 1068 Many-to-many applications have different needs and the most 1069 appropriate multiplexing choice will depend on how the actual 1070 application is realized. Multicast applications that are capable of 1071 using sender side congestion control can avoid the use of multiple 1072 multicast sessions and RTP sessions that result from use of receiver 1073 side congestion control. 1075 The properties of a broadcast application using RTP multicast: 1077 1. Uses a group of RTP sessions, not just one. Each endpoint will 1078 need to be a member of a number of RTP sessions in order to 1079 perform well. 1081 2. Within each RTP session, the number of RTP receivers is likely to 1082 be much larger than the number of RTP senders. 1084 3. The applications need signalling functions to identify the 1085 relationships between RTP sessions. 1087 4. The applications need signalling or RTP/RTCP functions to 1088 identify the relationships between SSRCs in different RTP 1089 sessions when needs beyond CNAME exist. 1091 Both broadcast and many-to-many multicast applications share a 1092 signalling requirement; all of the participants need the same RTP and 1093 payload type configuration. Otherwise, A could for example be using 1094 payload type 97 as the video codec H.264 while B thinks it is MPEG-2. 1095 SDP offer/answer [RFC3264] is not appropriate for ensuring this 1096 property in broadcast/multicast context. The signalling aspects of 1097 broadcast/multicast are not explored further in this memo. 1099 Security solutions for this type of group communication are also 1100 challenging. First, the key-management and the security protocol 1101 need to support group communication. Second, source authentication 1102 requires special solutions. For more discussion on this please 1103 review Options for Securing RTP Sessions [RFC7201]. 1105 4.3. Security and Key Management Considerations 1107 When dealing with point-to-point, 2-member RTP sessions only, there 1108 are few security issues that are relevant to the choice of having one 1109 RTP session or multiple RTP sessions. However, there are a few 1110 aspects of multiparty sessions that might warrant consideration. For 1111 general information of possible methods of securing RTP, please 1112 review RTP Security Options [RFC7201]. 1114 4.3.1. Security Context Scope 1116 When using SRTP [RFC3711], the security context scope is important 1117 and can be a necessary differentiation in some applications. As 1118 SRTP's crypto suites are (so far) built around symmetric keys, the 1119 receiver will need to have the same key as the sender. This results 1120 in that no one in a multi-party session can be certain that a 1121 received packet really was sent by the claimed sender and not by 1122 another party having access to the key. The single SRTP algorithm 1123 not having this propery is the TESLA source authentication [RFC4383]. 1124 However, TESLA adds delay to achieve source authentication. In most 1125 cases, symmetric ciphers provide sufficient security properties but 1126 create issues in a few cases. 1128 The first case is when someone leaves a multi-party session and one 1129 wants to ensure that the party that left can no longer access the RTP 1130 streams. This requires that everyone re-keys without disclosing the 1131 new keys to the excluded party. 1133 A second case is when using security as an enforcing mechanism for 1134 stream access differentiation between different receivers. Take for 1135 example a scalable layer or a high quality simulcast version that 1136 only users paying a premium are allowed to access. The mechanism 1137 preventing a receiver from getting the high quality stream can be 1138 based on the stream being encrypted with a key that user can't access 1139 without paying premium, using the key-management to limit access to 1140 the key. 1142 SRTP [RFC3711] as specified uses per SSRC unique keys, however the 1143 original assumption was a single session master key from which SSRC 1144 specific RTP and RTCP keys where derived. However, that assumption 1145 was proven incorrect, as the application usage and the developed key- 1146 mamangement mechanisms have chosen many different methods for 1147 ensuring SSRC unique keys. The key-management functions have 1148 different capabilities to establish different sets of keys, normally 1149 on a per-endpoint basis. For example, DTLS-SRTP [RFC5764] and 1150 Security Descriptions [RFC4568] establish different keys for outgoing 1151 and incoming traffic from an endpoint. This key usage has to be 1152 written into the cryptographic context, possibly associated with 1153 different SSRCs. Thus, limitations do exist depending on chosen key- 1154 management method and due to integration of particular 1155 implementations of the key-management and SRTP. 1157 4.3.2. Key Management for Multi-party Sessions 1159 The capabilities of the key-management combined with the RTP 1160 multiplexing choices affects the resulting security properties, 1161 control over the secured media, and who have access to it. 1163 Multi-party sessions contain at least one RTP stream from each active 1164 participant. Depending on the multi-party topology [RFC7667], each 1165 participant can both send and receive multiple RTP streams. 1166 Transport translator-based sessions (Topo-Trn-Translator) and 1167 multicast sessions (Topo-ASM), can neither use Security Description 1169 [RFC4568] nor DTLS-SRTP [RFC5764] without an extension as each 1170 endpoint provides its set of keys. In centralised conferences, the 1171 signalling counterpart is a conference server, and the transport 1172 translator is the media plane unicast counterpart (to which DTLS 1173 messages would be sent). Thus, an extension like Encrypted Key 1174 Transport [I-D.ietf-perc-srtp-ekt-diet] or a MIKEY [RFC3830] based 1175 solution that allows for keying all session participants with the 1176 same master key is needed. 1178 Privacy Enchanced RTP Conferencing (PERC) also enables a different 1179 trust model with semi-trusted media switching RTP middleboxes 1180 [I-D.ietf-perc-private-media-framework]. 1182 4.3.3. Complexity Implications 1184 The usage of security functions can surface complexity implications 1185 from the choice of multiplexing and topology. This becomes 1186 especially evident in RTP topologies having any type of middlebox 1187 that processes or modifies RTP/RTCP packets. While there is very 1188 small overhead for an RTP translator or mixer to rewrite an SSRC 1189 value in the RTP packet of an unencrypted session, the cost is higher 1190 when using cryptographic security functions. For example, if using 1191 SRTP [RFC3711], the actual security context and exact crypto key are 1192 determined by the SSRC field value. If one changes SSRC, the 1193 encryption and authentication must use another key. Thus, changing 1194 the SSRC value implies a decryption using the old SSRC and its 1195 security context, followed by an encryption using the new one. 1197 5. RTP Multiplexing Design Choices 1199 This section discusses how some RTP multiplexing design choices can 1200 be used in applications to achieve certain goals, and a summary of 1201 the implications of such choices. For each design there is 1202 discussion of benefits and downsides. 1204 5.1. Multiple Media Types in One Session 1206 This design uses a single RTP session for multiple different media 1207 types, like audio and video, and possibly also transport robustness 1208 mechanisms like FEC or retransmission. An endpoint can send zero, 1209 one or more media sources per media type, resulting in a number of 1210 RTP streams of various media types for both source and redundancy 1211 streams. 1213 The Advantages: 1215 1. Only a single RTP session is used, which implies: 1217 * Minimal need to keep NAT/FW state. 1219 * Minimal NAT/FW-traversal cost. 1221 * Fate-sharing for all media flows. 1223 * Minimal overhead for security association establishment. 1225 2. Dynamic allocation of RTP streams can be handled almost entirely 1226 at RTP level. How localized this can be kept to RTP level 1227 depends on the application's needs for explicit indication of the 1228 stream usage and how timely that can be signalled. 1230 The Disadvantages: 1232 a. It is less suitable for interworking with other applications that 1233 use individual RTP sessions per media type or multiple sessions 1234 for a single media type, due to the risk of SSRC collision and 1235 thus potential need for SSRC translation. 1237 b. Negotiation of individual bandwidths for the different media 1238 types is currently only possible in SDP when using RID 1239 [I-D.ietf-mmusic-rid]. 1241 c. It is not suitable for Split Component Terminal (see Section 3.10 1242 of [RFC7667]). 1244 d. Flow-based QoS cannot be used to provide separate treatment of 1245 RTP streams compared to others in the single RTP session. 1247 e. If there is significant asymmetry between the RTP streams' RTCP 1248 reporting needs, there are some challenges in configuration and 1249 usage to avoid wasting RTCP reporting on the RTP stream that does 1250 not need that frequent reporting. 1252 f. It is not suitable for applications where some receivers like to 1253 receive only a subset of the RTP streams, especially if multicast 1254 or transport translator is being used. 1256 g. There is some additional concern with legacy implementations that 1257 do not support the RTP specification fully when it comes to 1258 handling multiple SSRC per endpoint, as multiple simultaneous 1259 media types are sent as separate SSRC in the same RTP session. 1261 h. If the applications need finer control over which session 1262 participants are included in different sets of security 1263 associations, most key-management mechanisms will have 1264 difficulties establishing such a session. 1266 5.2. Multiple SSRCs of the Same Media Type 1268 In this design, each RTP session serves only a single media type. 1269 The RTP session can contain multiple RTP streams, either from a 1270 single endpoint or from multiple endpoints. This commonly creates a 1271 low number of RTP sessions, typically only one for audio and one for 1272 video, with a corresponding need for two listening ports when using 1273 RTP/RTCP multiplexing [RFC5761]. 1275 The Advantages 1277 1. It works well with Split Component Terminal (see Section 3.10 of 1278 [RFC7667]) where the split is per media type. 1280 2. It enables flow-based QoS with different prioritisation between 1281 media types. 1283 3. For applications with dynamic usage of RTP streams, i.e. 1284 frequently added and removed, having much of the state associated 1285 with the RTP session rather than per individual SSRC can avoid 1286 the need for in-session signalling of meta-information about each 1287 SSRC. In the simple cases this allows for unsignalled RTP 1288 streams where session level information and RTCP SDES item (e.g. 1289 CNAME) are suffient. In the more complex cases where more 1290 source-specific metadata needs to be signalled the SSRC can be 1291 associated with an intermediate identifier, e.g. the MID conveyed 1292 as an SDES item as defined in Section 15 of 1293 [I-D.ietf-mmusic-sdp-bundle-negotiation]. 1295 4. There is low overhead for security association establishment. 1297 The Disadvantages 1299 a. There are a slightly higher number of RTP sessions needed 1300 compared to Multiple Media Types in one Session Section 5.1. 1301 This implies: 1303 * More NAT/FW state is needed. 1305 * There is increased NAT/FW-traversal cost in both processing 1306 and delay. 1308 b. There is some potential for concern with legacy implementations 1309 that don't support the RTP specification fully when it comes to 1310 handling multiple SSRC per endpoint. 1312 c. It is not possible to control security association for sets of 1313 RTP streams within the same media type with today's key- 1314 management mechanisms, unless these are split into different RTP 1315 sessions (Section 5.3). 1317 For RTP applications where all RTP streams of the same media type 1318 share same usage, this structure provides efficiency gains in amount 1319 of network state used and provides more fate sharing with other media 1320 flows of the same type. At the same time, it is still maintaining 1321 almost all functionalities for the negotiation signaling of 1322 properties per individual media type, and also enables flow based QoS 1323 prioritisation between media types. It handles multi-party sessions 1324 well, independently of multicast or centralised transport 1325 distribution, as additional sources can dynamically enter and leave 1326 the session. 1328 5.3. Multiple Sessions for One Media Type 1330 This design goes one step further than above (Section 5.2) by using 1331 multiple RTP sessions also for a single media type. The main reason 1332 for going in this direction is that the RTP application needs 1333 separation of the RTP streams due to their usage, such as e.g. 1334 scalability over multicast, simulcast, need for extended QoS 1335 prioritisation, or the need for fine-grained signalling using RTP 1336 session-focused signalling tools. 1338 The Advantages: 1340 1. This is more suitable for multicast usage where receivers can 1341 individually select which RTP sessions they want to participate 1342 in, assuming each RTP session has its own multicast group. 1344 2. The application can indicate its usage of the RTP streams on RTP 1345 session level, when multiple different usages exist. 1347 3. There is less need for SSRC-specific explicit signalling for each 1348 media stream and thus reduced need for explicit and timely 1349 signalling when RTP streams are added or removed. 1351 4. It enables detailed QoS prioritisation for flow-based mechanisms. 1353 5. It works well with Split Component Terminal (see Section 3.10 of 1354 [RFC7667]). 1356 6. The scope for who is included in a security association can be 1357 structured around the different RTP sessions, thus enabling such 1358 functionality with existing key-management. 1360 The Disadvantages: 1362 a. There is an increased amount of session configuration state 1363 compared to Multiple SSRCs of the Same Media Type, due to the 1364 increased amount of RTP sessions. 1366 b. For RTP streams that are part of scalability, simulcast or 1367 transport robustness, a method to bind sources across multiple 1368 RTP sessions is needed. 1370 c. There is some potential for concern with legacy implementations 1371 that don't support the RTP specification fully when it comes to 1372 handling multiple SSRC per endpoint. 1374 d. There is higher overhead for security association establishment, 1375 due to the increased number of RTP sessions. 1377 e. If the applications need more fine-grained control than per RTP 1378 session over which participants that are included in different 1379 sets of security associations, most of today's key-management 1380 will have difficulties establishing such a session. 1382 For more complex RTP applications that have several different usages 1383 for RTP streams of the same media type, or uses scalability or 1384 simulcast, this solution can enable those functions at the cost of 1385 increased overhead associated with the additional sessions. This 1386 type of structure is suitable for more advanced applications as well 1387 as multicast-based applications requiring differentiation to 1388 different participants. 1390 5.4. Single SSRC per Endpoint 1392 In this design each endpoint in a point-to-point session has only a 1393 single SSRC, thus the RTP session contains only two SSRCs, one local 1394 and one remote. This session can be used both unidirectional, i.e. 1395 only a single RTP stream, or bi-directional, i.e. both endpoints have 1396 one RTP stream each. If the application needs additional media flows 1397 between the endpoints, it will have to establish additional RTP 1398 sessions. 1400 The Advantages: 1402 1. This design has great legacy interoperability potential as it 1403 will not tax any RTP stack implementations. 1405 2. The signalling has good possibilities to negotiate and describe 1406 the exact formats and bitrates for each RTP stream, especially 1407 using today's tools in SDP. 1409 3. It is possible to control security association per RTP stream 1410 with current key-management, since each RTP stream is directly 1411 related to an RTP session, and the most used keying mechanisms 1412 operates on a per-session basis. 1414 The Disadvantages: 1416 a. There is a linear growth of the amount of NAT/FW state with 1417 number of RTP streams. 1419 b. There is increased delay and resource consumption from NAT/FW- 1420 traversal. 1422 c. There are likely larger signalling message and signalling 1423 processing requirements due to the increased amount of session- 1424 related information. 1426 d. There is higher potential for a single RTP stream to fail during 1427 transport between the endpoints, due to the need for separate 1428 NAT/FW- traversal for every RTP stream since there is only one 1429 stream per session. 1431 e. The amount of explicit state for relating RTP streams grows, 1432 depending on how the application relates RTP streams. 1434 f. The port consumption might become a problem for centralised 1435 services, where the central node's port or 5-tuple filter 1436 consumption grows rapidly with the number of sessions. 1438 g. For applications where the RTP stream usage is highly dynamic, 1439 i.e. entering and leaving, the amount of signalling can become 1440 high. Issues can also arise from the need for timely 1441 establishment of additional RTP sessions. 1443 h. If, against the recommendation, the same SSRC value is reused in 1444 multiple RTP sessions rather than being randomly chosen, 1445 interworking with applications that use a different multiplexing 1446 structure will require SSRC translation. 1448 RTP applications with a strong need to interwork with legacy RTP 1449 applications can potentially benefit from this structure. However, a 1450 large number of media descriptions in SDP can also run into issues 1451 with existing implementations. For any application needing a larger 1452 number of media flows, the overhead can become very significant. 1453 This structure is also not suitable for non-mixed multi-party 1454 sessions, as any given RTP stream from each participant, although 1455 having same usage in the application, needs its own RTP session. In 1456 addition, the dynamic behaviour that can arise in multi-party 1457 applications can tax the signalling system and make timely media 1458 establishment more difficult. 1460 5.5. Summary 1462 Both the "Single SSRC per Endpoint" and the "Multiple Media Types in 1463 One Session" are cases that require full explicit signalling of the 1464 media stream relations. However, they operate on two different 1465 levels where the first primarily enables session level binding, and 1466 the second needs SSRC level binding. From another perspective, the 1467 two solutions are the two extreme points when it comes to number of 1468 RTP sessions needed. 1470 The two other designs, "Multiple SSRCs of the Same Media Type" and 1471 "Multiple Sessions for One Media Type", are two examples that 1472 primarily allows for some implicit mapping of the role or usage of 1473 the RTP streams based on which RTP session they appear in. It thus 1474 potentially allows for less signalling and in particular reduces the 1475 need for real-time signalling in sessions with dynamically changing 1476 number of RTP streams. They also represent points in-between the 1477 first two designs when it comes to amount of RTP sessions 1478 established, i.e. representing an attempt to balance the amount of 1479 RTP sessions with the functionality the communication session 1480 provides both on network level and on signalling level. 1482 6. Guidelines 1484 This section contains a number of multi-stream guidelines for 1485 implementers, system designers, or specification writers. 1487 Do not require use of the same SSRC value across RTP sessions: 1488 As discussed in Section 3.4.3 there exist drawbacks in using the 1489 same SSRC in multiple RTP sessions as a mechanism to bind related 1490 RTP streams together. It is instead recommended to use a 1491 mechanism to explicitly signal the relation, either in RTP/RTCP or 1492 in the signalling mechanism used to establish the RTP session(s). 1494 Use additional RTP streams for additional media sources: In the 1495 cases where an RTP endpoint needs to transmit additional RTP 1496 streams of the same media type in the application, with the same 1497 processing requirements at the network and RTP layers, it is 1498 suggested to send them in the same RTP session. For example a 1499 telepresence room where there are three cameras, and each camera 1500 captures 2 persons sitting at the table, sending each camera as 1501 its own RTP stream within a single RTP session is suggested. 1503 Use additional RTP sessions for streams with different requirements: 1505 When RTP streams have different processing requirements from the 1506 network or the RTP layer at the endpoints, it is suggested that 1507 the different types of streams are put in different RTP sessions. 1508 This includes the case where different participants want different 1509 subsets of the set of RTP streams. 1511 When using multiple RTP sessions, use grouping: When using multiple 1512 RTP session solutions, it is suggested to explicitly group the 1513 involved RTP sessions when needed using a signalling mechanism, 1514 for example The Session Description Protocol (SDP) Grouping 1515 Framework [RFC5888], using some appropriate grouping semantics. 1517 RTP/RTCP Extensions Support Multiple RTP Streams as Well as Multiple 1518 RTP Sessions: 1519 When defining an RTP or RTCP extension, the creator needs to 1520 consider if this extension is applicable to use with additional 1521 SSRCs and multiple RTP sessions. Any extension intended to be 1522 generic must support both. Extensions that are not as generally 1523 applicable will have to consider if interoperability is better 1524 served by defining a single solution or providing both options. 1526 Extensions for Transport Support: When defining new RTP/RTCP 1527 extensions intended for transport support, like the retransmission 1528 or FEC mechanisms, they must include support for both multiple RTP 1529 streams in the same RTP session and multiple RTP sessions, such 1530 that application developers can choose freely from the set of 1531 mechanisms without concerning themselves with which of the 1532 multiplexing choices a particular solution supports. 1534 7. IANA Considerations 1536 This document makes no request of IANA. 1538 Note to RFC Editor: this section can be removed on publication as an 1539 RFC. 1541 8. Security Considerations 1543 The security considerations of the RTP specification [RFC3550], any 1544 applicable RTP profile [RFC3551],[RFC4585],[RFC3711], and the 1545 extensions for sending multiple media types in a single RTP session 1546 [I-D.ietf-avtcore-multi-media-rtp-session], RID 1547 [I-D.ietf-mmusic-rid], BUNDLE 1548 [I-D.ietf-mmusic-sdp-bundle-negotiation], [RFC5760], [RFC5761], apply 1549 if selected and thus need to be considered in the evaluation. 1551 There is discussion of the security implications of choosing multiple 1552 SSRC vs multiple RTP sessions in Section 4.3. 1554 9. Contributors 1556 Hui Zheng (Marvin) contributed to WG draft versions -04 and -05 of 1557 the document. 1559 10. Acknowledgments 1561 The Authors like to acknowledge and thank Cullen Jennings, Dale R 1562 Worley, Huang Yihong (Rachel), Benjamin Kaduk, Mirja Kuehlewind, and 1563 Vijay Gurbani for review and comments. 1565 11. References 1567 11.1. Normative References 1569 [I-D.ietf-avtcore-multi-media-rtp-session] 1570 Westerlund, M., Perkins, C., and J. Lennox, "Sending 1571 Multiple Types of Media in a Single RTP Session", draft- 1572 ietf-avtcore-multi-media-rtp-session-13 (work in 1573 progress), December 2015. 1575 [I-D.ietf-mmusic-rid] 1576 Roach, A., "RTP Payload Format Restrictions", draft-ietf- 1577 mmusic-rid-15 (work in progress), May 2018. 1579 [I-D.ietf-mmusic-sdp-bundle-negotiation] 1580 Holmberg, C., Alvestrand, H., and C. Jennings, 1581 "Negotiating Media Multiplexing Using the Session 1582 Description Protocol (SDP)", draft-ietf-mmusic-sdp-bundle- 1583 negotiation-54 (work in progress), December 2018. 1585 [I-D.ietf-perc-srtp-ekt-diet] 1586 Jennings, C., Mattsson, J., McGrew, D., Wing, D., and F. 1587 Andreasen, "Encrypted Key Transport for DTLS and Secure 1588 RTP", draft-ietf-perc-srtp-ekt-diet-11 (work in progress), 1589 January 2020. 1591 [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. 1592 Jacobson, "RTP: A Transport Protocol for Real-Time 1593 Applications", STD 64, RFC 3550, DOI 10.17487/RFC3550, 1594 July 2003, . 1596 [RFC3551] Schulzrinne, H. and S. Casner, "RTP Profile for Audio and 1597 Video Conferences with Minimal Control", STD 65, RFC 3551, 1598 DOI 10.17487/RFC3551, July 2003, 1599 . 1601 [RFC3711] Baugher, M., McGrew, D., Naslund, M., Carrara, E., and K. 1602 Norrman, "The Secure Real-time Transport Protocol (SRTP)", 1603 RFC 3711, DOI 10.17487/RFC3711, March 2004, 1604 . 1606 [RFC4585] Ott, J., Wenger, S., Sato, N., Burmeister, C., and J. Rey, 1607 "Extended RTP Profile for Real-time Transport Control 1608 Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585, 1609 DOI 10.17487/RFC4585, July 2006, 1610 . 1612 [RFC5576] Lennox, J., Ott, J., and T. Schierl, "Source-Specific 1613 Media Attributes in the Session Description Protocol 1614 (SDP)", RFC 5576, DOI 10.17487/RFC5576, June 2009, 1615 . 1617 [RFC5760] Ott, J., Chesterfield, J., and E. Schooler, "RTP Control 1618 Protocol (RTCP) Extensions for Single-Source Multicast 1619 Sessions with Unicast Feedback", RFC 5760, 1620 DOI 10.17487/RFC5760, February 2010, 1621 . 1623 [RFC5761] Perkins, C. and M. Westerlund, "Multiplexing RTP Data and 1624 Control Packets on a Single Port", RFC 5761, 1625 DOI 10.17487/RFC5761, April 2010, 1626 . 1628 [RFC7656] Lennox, J., Gross, K., Nandakumar, S., Salgueiro, G., and 1629 B. Burman, Ed., "A Taxonomy of Semantics and Mechanisms 1630 for Real-Time Transport Protocol (RTP) Sources", RFC 7656, 1631 DOI 10.17487/RFC7656, November 2015, 1632 . 1634 [RFC7667] Westerlund, M. and S. Wenger, "RTP Topologies", RFC 7667, 1635 DOI 10.17487/RFC7667, November 2015, 1636 . 1638 11.2. Informative References 1640 [I-D.ietf-avtext-rid] 1641 Roach, A., Nandakumar, S., and P. Thatcher, "RTP Stream 1642 Identifier Source Description (SDES)", draft-ietf-avtext- 1643 rid-09 (work in progress), October 2016. 1645 [I-D.ietf-perc-private-media-framework] 1646 Jones, P., Benham, D., and C. Groves, "A Solution 1647 Framework for Private Media in Privacy Enhanced RTP 1648 Conferencing (PERC)", draft-ietf-perc-private-media- 1649 framework-12 (work in progress), June 2019. 1651 [JINGLE] Ludwig, S., Beda, J., Saint-Andre, P., McQueen, R., Egan, 1652 S., and J. Hildebrand, "XEP-0166: Jingle", XMPP.org 1653 https://xmpp.org/extensions/xep-0166.html, September 2018. 1655 [RFC2198] Perkins, C., Kouvelas, I., Hodson, O., Hardman, V., 1656 Handley, M., Bolot, J., Vega-Garcia, A., and S. Fosse- 1657 Parisis, "RTP Payload for Redundant Audio Data", RFC 2198, 1658 DOI 10.17487/RFC2198, September 1997, 1659 . 1661 [RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S. 1662 Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1 1663 Functional Specification", RFC 2205, DOI 10.17487/RFC2205, 1664 September 1997, . 1666 [RFC2474] Nichols, K., Blake, S., Baker, F., and D. Black, 1667 "Definition of the Differentiated Services Field (DS 1668 Field) in the IPv4 and IPv6 Headers", RFC 2474, 1669 DOI 10.17487/RFC2474, December 1998, 1670 . 1672 [RFC2974] Handley, M., Perkins, C., and E. Whelan, "Session 1673 Announcement Protocol", RFC 2974, DOI 10.17487/RFC2974, 1674 October 2000, . 1676 [RFC3261] Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston, 1677 A., Peterson, J., Sparks, R., Handley, M., and E. 1678 Schooler, "SIP: Session Initiation Protocol", RFC 3261, 1679 DOI 10.17487/RFC3261, June 2002, 1680 . 1682 [RFC3264] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model 1683 with Session Description Protocol (SDP)", RFC 3264, 1684 DOI 10.17487/RFC3264, June 2002, 1685 . 1687 [RFC3389] Zopf, R., "Real-time Transport Protocol (RTP) Payload for 1688 Comfort Noise (CN)", RFC 3389, DOI 10.17487/RFC3389, 1689 September 2002, . 1691 [RFC3830] Arkko, J., Carrara, E., Lindholm, F., Naslund, M., and K. 1692 Norrman, "MIKEY: Multimedia Internet KEYing", RFC 3830, 1693 DOI 10.17487/RFC3830, August 2004, 1694 . 1696 [RFC4103] Hellstrom, G. and P. Jones, "RTP Payload for Text 1697 Conversation", RFC 4103, DOI 10.17487/RFC4103, June 2005, 1698 . 1700 [RFC4383] Baugher, M. and E. Carrara, "The Use of Timed Efficient 1701 Stream Loss-Tolerant Authentication (TESLA) in the Secure 1702 Real-time Transport Protocol (SRTP)", RFC 4383, 1703 DOI 10.17487/RFC4383, February 2006, 1704 . 1706 [RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session 1707 Description Protocol", RFC 4566, DOI 10.17487/RFC4566, 1708 July 2006, . 1710 [RFC4568] Andreasen, F., Baugher, M., and D. Wing, "Session 1711 Description Protocol (SDP) Security Descriptions for Media 1712 Streams", RFC 4568, DOI 10.17487/RFC4568, July 2006, 1713 . 1715 [RFC4588] Rey, J., Leon, D., Miyazaki, A., Varsa, V., and R. 1716 Hakenberg, "RTP Retransmission Payload Format", RFC 4588, 1717 DOI 10.17487/RFC4588, July 2006, 1718 . 1720 [RFC5104] Wenger, S., Chandra, U., Westerlund, M., and B. Burman, 1721 "Codec Control Messages in the RTP Audio-Visual Profile 1722 with Feedback (AVPF)", RFC 5104, DOI 10.17487/RFC5104, 1723 February 2008, . 1725 [RFC5109] Li, A., Ed., "RTP Payload Format for Generic Forward Error 1726 Correction", RFC 5109, DOI 10.17487/RFC5109, December 1727 2007, . 1729 [RFC5389] Rosenberg, J., Mahy, R., Matthews, P., and D. Wing, 1730 "Session Traversal Utilities for NAT (STUN)", RFC 5389, 1731 DOI 10.17487/RFC5389, October 2008, 1732 . 1734 [RFC5764] McGrew, D. and E. Rescorla, "Datagram Transport Layer 1735 Security (DTLS) Extension to Establish Keys for the Secure 1736 Real-time Transport Protocol (SRTP)", RFC 5764, 1737 DOI 10.17487/RFC5764, May 2010, 1738 . 1740 [RFC5888] Camarillo, G. and H. Schulzrinne, "The Session Description 1741 Protocol (SDP) Grouping Framework", RFC 5888, 1742 DOI 10.17487/RFC5888, June 2010, 1743 . 1745 [RFC6465] Ivov, E., Ed., Marocco, E., Ed., and J. Lennox, "A Real- 1746 time Transport Protocol (RTP) Header Extension for Mixer- 1747 to-Client Audio Level Indication", RFC 6465, 1748 DOI 10.17487/RFC6465, December 2011, 1749 . 1751 [RFC7201] Westerlund, M. and C. Perkins, "Options for Securing RTP 1752 Sessions", RFC 7201, DOI 10.17487/RFC7201, April 2014, 1753 . 1755 [RFC7657] Black, D., Ed. and P. Jones, "Differentiated Services 1756 (Diffserv) and Real-Time Communication", RFC 7657, 1757 DOI 10.17487/RFC7657, November 2015, 1758 . 1760 [RFC7826] Schulzrinne, H., Rao, A., Lanphier, R., Westerlund, M., 1761 and M. Stiemerling, Ed., "Real-Time Streaming Protocol 1762 Version 2.0", RFC 7826, DOI 10.17487/RFC7826, December 1763 2016, . 1765 [RFC7983] Petit-Huguenin, M. and G. Salgueiro, "Multiplexing Scheme 1766 Updates for Secure Real-time Transport Protocol (SRTP) 1767 Extension for Datagram Transport Layer Security (DTLS)", 1768 RFC 7983, DOI 10.17487/RFC7983, September 2016, 1769 . 1771 [RFC8088] Westerlund, M., "How to Write an RTP Payload Format", 1772 RFC 8088, DOI 10.17487/RFC8088, May 2017, 1773 . 1775 [RFC8108] Lennox, J., Westerlund, M., Wu, Q., and C. Perkins, 1776 "Sending Multiple RTP Streams in a Single RTP Session", 1777 RFC 8108, DOI 10.17487/RFC8108, March 2017, 1778 . 1780 [RFC8445] Keranen, A., Holmberg, C., and J. Rosenberg, "Interactive 1781 Connectivity Establishment (ICE): A Protocol for Network 1782 Address Translator (NAT) Traversal", RFC 8445, 1783 DOI 10.17487/RFC8445, July 2018, 1784 . 1786 Appendix A. Dismissing Payload Type Multiplexing 1788 This section documents a number of reasons why using the payload type 1789 as a multiplexing point is unsuitable for most issues related to 1790 multiple RTP streams. Attempting to use Payload type multiplexing 1791 beyond its defined usage has well known negative effects on RTP 1792 discussed below. To use payload type as the single discriminator for 1793 multiple streams implies that all the different RTP streams are being 1794 sent with the same SSRC, thus using the same timestamp and sequence 1795 number space. This has many effects: 1797 1. Putting constraints on RTP timestamp rate for the multiplexed 1798 media. For example, RTP streams that use different RTP 1799 timestamp rates cannot be combined, as the timestamp values need 1800 to be consistent across all multiplexed media frames. Thus 1801 streams are forced to use the same RTP timestamp rate. When 1802 this is not possible, payload type multiplexing cannot be used. 1804 2. Many RTP payload formats can fragment a media object over 1805 multiple RTP packets, like parts of a video frame. These 1806 payload formats need to determine the order of the fragments to 1807 correctly decode them. Thus, it is important to ensure that all 1808 fragments related to a frame or a similar media object are 1809 transmitted in sequence and without interruptions within the 1810 object. This can relatively simple be solved on the sender side 1811 by ensuring that the fragments of each RTP stream are sent in 1812 sequence. 1814 3. Some media formats require uninterrupted sequence number space 1815 between media parts. These are media formats where any missing 1816 RTP sequence number will result in decoding failure or invoking 1817 a repair mechanism within a single media context. The text/ 1818 T140 payload format [RFC4103] is an example of such a format. 1819 These formats will need a sequence numbering abstraction 1820 function between RTP and the individual RTP stream before being 1821 used with payload type multiplexing. 1823 4. Sending multiple media streams in the same sequence number space 1824 makes it impossible to determine which media stream lost a 1825 packet. This as the payload type that is used for demultiplex 1826 the media stream is not received. Thus, causing the receiver 1827 difficulties in determining which stream to apply packet loss 1828 concealment or other stream-specific loss mitigation mechanisms. 1830 5. If RTP Retransmission [RFC4588] is used and there is a loss, it 1831 is possible to ask for the missing packet(s) by SSRC and 1832 sequence number, not by payload type. If only some of the 1833 payload type multiplexed streams are of interest, there is no 1834 way of telling which missing packet(s) belong to the interesting 1835 stream(s) and all lost packets need be requested, wasting 1836 bandwidth. 1838 6. The current RTCP feedback mechanisms are built around providing 1839 feedback on RTP streams based on stream ID (SSRC), packet 1840 (sequence numbers) and time interval (RTP timestamps). There is 1841 almost never a field to indicate which payload type is reported, 1842 so sending feedback for a specific RTP payload type is difficult 1843 without extending existing RTCP reporting. 1845 7. The current RTCP media control messages [RFC5104] specification 1846 is oriented around controlling particular media flows, i.e. 1847 requests are done addressing a particular SSRC. Such mechanisms 1848 would need to be redefined to support payload type multiplexing. 1850 8. The number of payload types are inherently limited. 1851 Accordingly, using payload type multiplexing limits the number 1852 of streams that can be multiplexed and does not scale. This 1853 limitation is exacerbated if one uses solutions like RTP and 1854 RTCP multiplexing [RFC5761] where a number of payload types are 1855 blocked due to the overlap between RTP and RTCP. 1857 9. At times, there is a need to group multiplexed streams and this 1858 is currently possible for RTP sessions and for SSRC, but there 1859 is no defined way to group payload types. 1861 10. It is currently not possible to signal bandwidth requirements 1862 per RTP stream when using payload type multiplexing. 1864 11. Most existing SDP media level attributes cannot be applied on a 1865 per payload type level and would require re-definition in that 1866 context. 1868 12. A legacy endpoint that does not understand the indication that 1869 different RTP payload types are different RTP streams might be 1870 slightly confused by the large amount of possibly overlapping or 1871 identically defined RTP payload types. 1873 Appendix B. Signalling Considerations 1875 Signalling is not an architectural consideration for RTP itself, so 1876 this discussion has been moved to an appendix. However, it is 1877 extremely important for anyone building complete applications, so it 1878 is deserving of discussion. 1880 We document salient issues here that need to be addressed by the WGs 1881 that use some form of signaling to establish RTP sessions. These 1882 issues cannot simply be addressed by tweaking, extending, or 1883 profiling RTP, but require a dedicated and indepth look at the 1884 signaling primitives that set up the RTP sessions. 1886 There exist various signalling solutions for establishing RTP 1887 sessions. Many are SDP [RFC4566] based, however SDP functionality is 1888 also dependent on the signalling protocols carrying the SDP. RTSP 1889 [RFC7826] and SAP [RFC2974] both use SDP in a declarative fashion, 1890 while SIP [RFC3261] uses SDP with the additional definition of Offer/ 1891 Answer [RFC3264]. The impact on signalling and especially SDP needs 1892 to be considered as it can greatly affect how to deploy a certain 1893 multiplexing point choice. 1895 B.1. Session Oriented Properties 1897 One aspect of the existing signalling is that it is focused on RTP 1898 sessions, or in the case of SDP, the media description concept. 1899 There are a number of things that are signalled on media description 1900 level but those are not necessarily strictly bound to an RTP session 1901 and could be of interest to signal specifically for a particular RTP 1902 stream (SSRC) within the session. The following properties have been 1903 identified as being potentially useful to signal not only on RTP 1904 session level: 1906 o Bitrate/Bandwidth exist today only at aggregate or as a common 1907 "any RTP stream" limit, unless either codec-specific bandwidth 1908 limiting or RTCP signalling using TMMBR [RFC5104] is used. 1910 o Which SSRC that will use which RTP payload type (this will be 1911 visible from the first media packet, but is sometimes useful to 1912 know before packet arrival). 1914 Some of these issues are clearly SDP's problem rather than RTP 1915 limitations. However, if the aim is to deploy an solution using 1916 additional SSRCs that contains several sets of RTP streams with 1917 different properties (encoding/packetization parameter, bit-rate, 1918 etc.), putting each set in a different RTP session would directly 1919 enable negotiation of the parameters for each set. If insisting on 1920 additional SSRC only, a number of signalling extensions are needed to 1921 clarify that there are multiple sets of RTP streams with different 1922 properties and that they need in fact be kept different, since a 1923 single set will not satisfy the application's requirements. 1925 For some parameters, such as RTP payload type, resolution and 1926 framerate, a SSRC-linked mechanism has been proposed in 1927 [I-D.ietf-mmusic-rid] 1929 B.2. SDP Prevents Multiple Media Types 1931 SDP chose to use the m= line both to delineate an RTP session and to 1932 specify the top level of the MIME media type; audio, video, text, 1933 image, application. This media type is used as the top-level media 1934 type for identifying the actual payload format and is bound to a 1935 particular payload type using the rtpmap attribute. This binding has 1936 to be loosened in order to use SDP to describe RTP sessions 1937 containing multiple MIME top level types. 1939 [I-D.ietf-mmusic-sdp-bundle-negotiation] describes how to let 1940 multiple SDP media descriptions use a single underlying transport in 1941 SDP, which allows to define one RTP session with media types having 1942 different MIME top level types. 1944 B.3. Signalling RTP Stream Usage 1946 RTP streams being transported in RTP have some particular usage in an 1947 RTP application. This usage of the RTP stream is in many 1948 applications so far implicitly signalled. For example, an 1949 application might choose to take all incoming audio RTP streams, mix 1950 them and play them out. However, in more advanced applications that 1951 use multiple RTP streams there will be more than a single usage or 1952 purpose among the set of RTP streams being sent or received. RTP 1953 applications will need to signal this usage somehow. The signalling 1954 used will have to identify the RTP streams affected by their RTP- 1955 level identifiers, which means that they have to be identified either 1956 by their session or by their SSRC + session. 1958 In some applications, the receiver cannot utilise the RTP stream at 1959 all before it has received the signalling message describing the RTP 1960 stream and its usage. In other applications, there exists a default 1961 handling that is appropriate. 1963 If all RTP streams in an RTP session are to be treated in the same 1964 way, identifying the session is enough. If SSRCs in a session are to 1965 be treated differently, signalling needs to identify both the session 1966 and the SSRC. 1968 If this signalling affects how any RTP central node, like an RTP 1969 mixer or translator that selects, mixes or processes streams, treats 1970 the streams, the node will also need to receive the same signalling 1971 to know how to treat RTP streams with different usage in the right 1972 fashion. 1974 Authors' Addresses 1976 Magnus Westerlund 1977 Ericsson 1978 Torshamnsgatan 23 1979 SE-164 80 Kista 1980 Sweden 1982 Phone: +46 10 714 82 87 1983 Email: magnus.westerlund@ericsson.com 1985 Bo Burman 1986 Ericsson 1987 Gronlandsgatan 31 1988 SE-164 60 Kista 1989 Sweden 1991 Email: bo.burman@ericsson.com 1993 Colin Perkins 1994 University of Glasgow 1995 School of Computing Science 1996 Glasgow G12 8QQ 1997 United Kingdom 1999 Email: csp@csperkins.org 2001 Harald Tveit Alvestrand 2002 Google 2003 Kungsbron 2 2004 Stockholm 11122 2005 Sweden 2007 Email: harald@alvestrand.no 2009 Roni Even 2011 Email: ron.even.tlv@gmail.com