idnits 2.17.1 draft-ietf-avtcore-multiplex-guidelines-09.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 22, 2019) is 1738 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'ALF' is defined on line 1607, but no explicit reference was found in the text == Outdated reference: A later version (-13) exists of draft-ietf-perc-srtp-ekt-diet-10 -- Obsolete informational reference (is this intentional?): RFC 4566 (Obsoleted by RFC 8866) -- Obsolete informational reference (is this intentional?): RFC 5389 (Obsoleted by RFC 8489) Summary: 0 errors (**), 0 flaws (~~), 3 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group M. Westerlund 3 Internet-Draft B. Burman 4 Intended status: Informational Ericsson 5 Expires: January 23, 2020 C. Perkins 6 University of Glasgow 7 H. Alvestrand 8 Google 9 R. Even 10 Huawei 11 July 22, 2019 13 Guidelines for using the Multiplexing Features of RTP to Support 14 Multiple Media Streams 15 draft-ietf-avtcore-multiplex-guidelines-09 17 Abstract 19 The Real-time Transport Protocol (RTP) is a flexible protocol that 20 can be used in a wide range of applications, networks, and system 21 topologies. That flexibility makes for wide applicability, but can 22 complicate the application design process. One particular design 23 question that has received much attention is how to support multiple 24 media streams in RTP. This memo discusses the available options and 25 design trade-offs, and provides guidelines on how to use the 26 multiplexing features of RTP to support multiple media streams. 28 Status of This Memo 30 This Internet-Draft is submitted in full conformance with the 31 provisions of BCP 78 and BCP 79. 33 Internet-Drafts are working documents of the Internet Engineering 34 Task Force (IETF). Note that other groups may also distribute 35 working documents as Internet-Drafts. The list of current Internet- 36 Drafts is at https://datatracker.ietf.org/drafts/current/. 38 Internet-Drafts are draft documents valid for a maximum of six months 39 and may be updated, replaced, or obsoleted by other documents at any 40 time. It is inappropriate to use Internet-Drafts as reference 41 material or to cite them other than as "work in progress." 43 This Internet-Draft will expire on January 23, 2020. 45 Copyright Notice 47 Copyright (c) 2019 IETF Trust and the persons identified as the 48 document authors. All rights reserved. 50 This document is subject to BCP 78 and the IETF Trust's Legal 51 Provisions Relating to IETF Documents 52 (https://trustee.ietf.org/license-info) in effect on the date of 53 publication of this document. Please review these documents 54 carefully, as they describe your rights and restrictions with respect 55 to this document. Code Components extracted from this document must 56 include Simplified BSD License text as described in Section 4.e of 57 the Trust Legal Provisions and are provided without warranty as 58 described in the Simplified BSD License. 60 Table of Contents 62 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 63 2. Definitions . . . . . . . . . . . . . . . . . . . . . . . . . 4 64 2.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 4 65 2.2. Subjects Out of Scope . . . . . . . . . . . . . . . . . . 5 66 3. RTP Multiplexing Overview . . . . . . . . . . . . . . . . . . 5 67 3.1. Reasons for Multiplexing and Grouping RTP Streams . . . . 5 68 3.2. RTP Multiplexing Points . . . . . . . . . . . . . . . . . 6 69 3.2.1. RTP Session . . . . . . . . . . . . . . . . . . . . . 7 70 3.2.2. Synchronisation Source (SSRC) . . . . . . . . . . . . 8 71 3.2.3. Contributing Source (CSRC) . . . . . . . . . . . . . 10 72 3.2.4. RTP Payload Type . . . . . . . . . . . . . . . . . . 10 73 3.3. Issues Related to RTP Topologies . . . . . . . . . . . . 11 74 3.4. Issues Related to RTP and RTCP Protocol . . . . . . . . . 12 75 3.4.1. The RTP Specification . . . . . . . . . . . . . . . . 13 76 3.4.2. Multiple SSRCs in a Session . . . . . . . . . . . . . 14 77 3.4.3. Binding Related Sources . . . . . . . . . . . . . . . 14 78 3.4.4. Forward Error Correction . . . . . . . . . . . . . . 16 79 4. Considerations for RTP Multiplexing . . . . . . . . . . . . . 17 80 4.1. Interworking Considerations . . . . . . . . . . . . . . . 17 81 4.1.1. Application Interworking . . . . . . . . . . . . . . 17 82 4.1.2. RTP Translator Interworking . . . . . . . . . . . . . 17 83 4.1.3. Gateway Interworking . . . . . . . . . . . . . . . . 18 84 4.1.4. Multiple SSRC Legacy Considerations . . . . . . . . . 19 85 4.2. Network Considerations . . . . . . . . . . . . . . . . . 20 86 4.2.1. Quality of Service . . . . . . . . . . . . . . . . . 20 87 4.2.2. NAT and Firewall Traversal . . . . . . . . . . . . . 20 88 4.2.3. Multicast . . . . . . . . . . . . . . . . . . . . . . 22 89 4.3. Security and Key Management Considerations . . . . . . . 23 90 4.3.1. Security Context Scope . . . . . . . . . . . . . . . 24 91 4.3.2. Key Management for Multi-party Sessions . . . . . . . 24 92 4.3.3. Complexity Implications . . . . . . . . . . . . . . . 25 94 5. RTP Multiplexing Design Choices . . . . . . . . . . . . . . . 25 95 5.1. Multiple Media Types in One Session . . . . . . . . . . . 25 96 5.2. Multiple SSRCs of the Same Media Type . . . . . . . . . . 27 97 5.3. Multiple Sessions for One Media Type . . . . . . . . . . 28 98 5.4. Single SSRC per Endpoint . . . . . . . . . . . . . . . . 29 99 5.5. Summary . . . . . . . . . . . . . . . . . . . . . . . . . 31 100 6. Guidelines . . . . . . . . . . . . . . . . . . . . . . . . . 31 101 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 32 102 8. Security Considerations . . . . . . . . . . . . . . . . . . . 32 103 9. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 33 104 10. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 33 105 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 33 106 11.1. Normative References . . . . . . . . . . . . . . . . . . 33 107 11.2. Informative References . . . . . . . . . . . . . . . . . 35 108 Appendix A. Dismissing Payload Type Multiplexing . . . . . . . . 38 109 Appendix B. Signalling Considerations . . . . . . . . . . . . . 40 110 B.1. Session Oriented Properties . . . . . . . . . . . . . . . 40 111 B.2. SDP Prevents Multiple Media Types . . . . . . . . . . . . 41 112 B.3. Signalling RTP Stream Usage . . . . . . . . . . . . . . . 41 113 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 42 115 1. Introduction 117 The Real-time Transport Protocol (RTP) [RFC3550] is a commonly used 118 protocol for real-time media transport. It is a protocol that 119 provides great flexibility and can support a large set of different 120 applications. RTP was from the beginning designed for multiple 121 participants in a communication session. It supports many topology 122 paradigms and usages, as defined in [RFC7667]. RTP has several 123 multiplexing points designed for different purposes. These enable 124 support of multiple RTP streams and switching between different 125 encoding or packetization of the media. By using multiple RTP 126 sessions, sets of RTP streams can be structured for efficient 127 processing or identification. Thus, an RTP application designer 128 needs to understand how to best use the RTP session, the RTP stream 129 identifier (SSRC), and the RTP payload type to meet the application's 130 needs. 132 There have been increased interest in more advanced usage of RTP. 133 For example, multiple RTP streams can be used when a single endpoint 134 has multiple media sources (like multiple cameras or microphones) 135 that need to be sent simultaneously. Consequently, questions are 136 raised regarding the most appropriate RTP usage. The limitations in 137 some implementations, RTP/RTCP extensions, and signalling have also 138 been exposed. The authors hope that clarification on the usefulness 139 of some functionalities in RTP will result in more complete 140 implementations in the future. 142 The purpose of this document is to provide clear information about 143 the possibilities of RTP when it comes to multiplexing. The RTP 144 application designer needs to understand the implications arising 145 from a particular usage of the RTP multiplexing points. The document 146 will provide some guidelines and recommend against some usages as 147 being unsuitable, in general or for particular purposes. 149 The document starts with some definitions and then goes into the 150 existing RTP functionalities around multiplexing. Both the desired 151 behaviour and the implications of a particular behaviour depend on 152 which topologies are used, which requires some consideration. This 153 is followed by a discussion of some choices in multiplexing behaviour 154 and their impacts. Some designs of RTP usage are discussed. 155 Finally, some guidelines and examples are provided. 157 2. Definitions 159 2.1. Terminology 161 The definitions in Section 3 of [RFC3550] are referenced normatively. 163 The taxonomy defined in [RFC7656] is referenced normatively. 165 The following terms and abbreviations are used in this document: 167 Multiparty: A communication situation including multiple endpoints. 168 In this document, it will be used to refer to situations where 169 more than two endpoints communicate. 171 Multiplexing: The operation of taking multiple entities as input, 172 aggregating them onto some common resource while keeping the 173 individual entities addressable such that they can later be fully 174 and unambiguously separated (de-multiplexed) again. 176 RTP Receiver: An Endpoint or Middlebox receiving RTP streams and 177 RTCP messages. It uses at least one SSRC to send RTCP messages. 178 An RTP Receiver may also be an RTP Sender. 180 RTP Sender: An Endpoint sending one or more RTP streams, but also 181 sending RTCP messages. 183 RTP Session Group: One or more RTP sessions that are used together 184 to perform some function. Examples are multiple RTP sessions used 185 to carry different layers of a layered encoding. In an RTP 186 Session Group, CNAMEs are assumed to be valid across all RTP 187 sessions, and designate synchronisation contexts that can cross 188 RTP sessions; i.e. SSRCs that map to a common CNAME can be assumed 189 to have RTCP SR timing information derived from a common clock 190 such that they can be synchronised for playout. 192 Signalling: The process of configuring endpoints to participate in 193 one or more RTP sessions. 195 Note: The above definitions of RTP Receiver and RTP Sender are 196 consistent with the usage in [RFC3550]. 198 2.2. Subjects Out of Scope 200 This document is focused on issues that affect RTP. Thus, issues 201 that involve signalling protocols, such as whether SIP [RFC3261], 202 Jingle [JINGLE] or some other protocol is in use for session 203 configuration, the particular syntaxes used to define RTP session 204 properties, or the constraints imposed by particular choices in the 205 signalling protocols, are mentioned only as examples in order to 206 describe the RTP issues more precisely. 208 This document assumes the applications will use RTCP. While there 209 are applications that don't send RTCP, they do not conform to the RTP 210 specification, and thus can be regarded as reusing the RTP packet 211 format but not implementing the RTP protocol. 213 3. RTP Multiplexing Overview 215 3.1. Reasons for Multiplexing and Grouping RTP Streams 217 There are several reasons why an endpoint might choose to send 218 multiple media streams. In the below discussion, please keep in mind 219 that the reasons for having multiple RTP streams vary and include but 220 are not limited to the following: 222 o Multiple media sources 224 o Multiple RTP streams might be needed to represent one media source 225 (for instance when using layered encodings) 227 o A retransmission stream might repeat some parts of the content of 228 another RTP stream 230 o A Forward Error Correction (FEC) stream might provide material 231 that can be used to repair another RTP stream 233 o Alternative encodings, for instance using different codecs for the 234 same audio stream 236 o Alternative formats, for instance multiple resolutions of the same 237 video stream 239 For each of these reasons, it is necessary to decide if each 240 additional RTP stream is sent within the same RTP session as the 241 other RTP streams, or if it is necessary to use additional RTP 242 sessions to group the RTP streams. The choice suitable for one 243 reason, might not be the choice suitable for another reason. The 244 clearest understanding is associated with multiplexing multiple media 245 sources of the same media type. However, all reasons warrant 246 discussion and clarification on how to deal with them. As the 247 discussion below will show, in reality we cannot choose a single one 248 of SSRC or RTP session multiplexing solutions for all purposes. To 249 utilise RTP well and as efficiently as possible, both are needed. 250 The real issue is finding the right guidance on when to create 251 additional RTP sessions and when additional RTP streams in the same 252 RTP session is the right choice. 254 3.2. RTP Multiplexing Points 256 This section describes the multiplexing points present in the RTP 257 protocol that can be used to distinguish RTP streams and groups of 258 RTP streams. Figure 1 outlines the process of demultiplexing 259 incoming RTP streams starting already at the socket representing 260 reception of one or transport flows, e.g. an UDP destination port. 261 It also demultiplexes RTP/RTCP from any other protocols, such as STUN 262 [RFC5389] and DTLS-SRTP [RFC5764] on the same transport as described 263 in [RFC7983]. 265 | 266 | packets 267 +-- v 268 | +------------+ 269 | | Socket | Transport Protocol Demultiplexing 270 | +------------+ 271 | || || 272 RTP | RTP/ || |+-----> DTLS (SRTP Keying, SCTP, etc) 273 Session | RTCP || +------> STUN (multiplexed using same port) 274 +-- || 275 +-- || 276 | (split by SSRC) 277 | || || || 278 | || || || 279 RTP | +--+ +--+ +--+ 280 Streams | |PB| |PB| |PB| Jitter buffer, process RTCP, etc. 281 | +--+ +--+ +--+ 282 +-- | | | 283 (select decoder based on PT) 284 +-- | / | 285 | +----+ | 286 | / | | 287 Payload | +---+ +---+ +---+ 288 Formats | |Dec| |Dec| |Dec| Decoders 289 | +---+ +---+ +---+ 290 +-- 292 Figure 1: RTP Demultiplexing Process 294 3.2.1. RTP Session 296 An RTP session is the highest semantic layer in the RTP protocol, and 297 represents an association between a group of communicating endpoints. 298 RTP does not contain a session identifier, yet different RTP sessions 299 must be possible to identify both across different endpoints and 300 within a single endpoint. 302 For RTP session separation across endpoints, the set of participants 303 that form an RTP session is defined as those that share a single 304 synchronisation source space [RFC3550]. That is, if a group of 305 participants are each aware of the synchronisation source identifiers 306 belonging to the other participants, then those participants are in a 307 single RTP session. A participant can become aware of a 308 synchronisation source identifier by receiving an RTP packet 309 containing it in the SSRC field or CSRC list, by receiving an RTCP 310 packet mentioning it in an SSRC field, or through signalling (e.g., 311 the Session Description Protocol (SDP) [RFC4566] "a=ssrc:" attribute 313 [RFC5576]). Thus, the scope of an RTP session is determined by the 314 participants' network interconnection topology, in combination with 315 RTP and RTCP forwarding strategies deployed by the endpoints and any 316 middleboxes, and by the signalling. 318 For RTP session separation within a single endpoint, RTP relies on 319 the underlying transport layer, and on the signalling to identify RTP 320 sessions in a manner that is meaningful to the application. A single 321 endpoint can have one or more transport flows for the same RTP 322 session, and a single RTP session can therefore span multiple 323 transport layer flows even if all endpoints use a single transport 324 layer flow per endpoint for that RTP session. The signalling layer 325 might give RTP sessions an explicit identifier, or the identification 326 might be implicit based on the addresses and ports used. 327 Accordingly, a single RTP session can have multiple associated 328 identifiers, explicit and implicit, belonging to different contexts. 329 For example, when running RTP on top of UDP/IP, an endpoint can 330 identify and delimit an RTP session from other RTP sessions by their 331 UDP source and destination IP addresses and UDP port numbers. 332 Independently if an endpoint has one or more IP addresses, a single 333 RTP session can be using multiple IP/UDP flows for receiving and/or 334 sending RTP packets to other endpoints or middleboxes. Another 335 example is SDP media descriptions (the "m=" line and the following 336 associated lines) that signals the transport flow and RTP session 337 configuration for the endpoint's part of the RTP session. The SDP 338 grouping framework [RFC5888] allows labeling of the media 339 descriptions to be used so that RTP Session Groups can be created. 340 Through use of Negotiating Media Multiplexing Using the Session 341 Description Protocol (SDP) [I-D.ietf-mmusic-sdp-bundle-negotiation], 342 multiple media descriptions become part of a common RTP session where 343 each media description represents the RTP streams sent or received 344 for a media source. 346 The RTP protocol makes no normative statements about the relationship 347 between different RTP sessions, however the applications that use 348 more than one RTP session will have some higher layer understanding 349 of the relationship between the sessions they create. 351 3.2.2. Synchronisation Source (SSRC) 353 A synchronisation source (SSRC) identifies a source of an RTP stream, 354 or an RTP receiver when sending RTCP. Every endpoint has at least 355 one SSRC identifier, even if it does not send RTP packets. RTP 356 endpoints that are only RTP receivers still send RTCP and use their 357 SSRC identifiers in the RTCP packets they send. An endpoint can have 358 multiple SSRC identifiers if it sends multiple RTP streams. 359 Endpoints that are both RTP sender and RTP receiver use the same SSRC 360 in both roles. 362 The SSRC is a 32-bit identifier. It is present in every RTP and RTCP 363 packet header, and in the payload of some RTCP packet types. It can 364 also be present in SDP signalling. Unless pre-signalled, e.g. using 365 the SDP "a=ssrc:" attribute [RFC5576], the SSRC is chosen at random. 366 It is not dependent on the network address of the endpoint, and is 367 intended to be unique within an RTP session. SSRC collisions can 368 occur, and are handled as specified in [RFC3550] and [RFC5576], 369 resulting in the SSRC of the colliding RTP streams or receivers 370 changing. An endpoint that changes its network transport address 371 during a session has to choose a new SSRC identifier to avoid being 372 interpreted as looped source, unless the transport layer mechanism, 373 e.g ICE [RFC8445], handles such changes. 375 SSRC identifiers that belong to the same synchronisation context 376 (i.e., that represent RTP streams that can be synchronised using 377 information in RTCP SR packets) use identical CNAME chunks in 378 corresponding RTCP SDES packets. SDP signalling can also be used to 379 provide explicit SSRC grouping [RFC5576]. 381 In some cases, the same SSRC identifier value is used to relate 382 streams in two different RTP sessions, such as in RTP retransmission 383 [RFC4588]. This is to be avoided since there is no guarantee that 384 SSRC values are unique across RTP sessions. For the RTP 385 retransmission [RFC4588] case it is recommended to use explicit 386 binding of the source RTP stream and the redundancy stream, e.g. 387 using the RepairedRtpStreamId RTCP SDES item [I-D.ietf-avtext-rid]. 389 Note that RTP sequence number and RTP timestamp are scoped by the 390 SSRC and thus specific per RTP stream. 392 Different types of entities use an SSRC to identify themselves, as 393 follows: 395 A real media source: Uses the SSRC to identify a "physical" media 396 source. 398 A conceptual media source: Uses the SSRC to identify the result of 399 applying some filtering function in a network node, for example a 400 filtering function in an RTP mixer that provides the most active 401 speaker based on some criteria, or a mix representing a set of 402 other sources. 404 An RTP receiver: Uses the SSRC to identify itself as the source of 405 its RTCP reports. 407 An endpoint that generates more than one media type, e.g. a 408 conference participant sending both audio and video, need not (and, 409 indeed, should not) use the same SSRC value across RTP sessions. 411 RTCP compound packets containing the CNAME SDES item is the 412 designated method to bind an SSRC to a CNAME, effectively cross- 413 correlating SSRCs within and between RTP Sessions as coming from the 414 same endpoint. The main property attributed to SSRCs associated with 415 the same CNAME is that they are from a particular synchronisation 416 context and can be synchronised at playback. 418 An RTP receiver receiving a previously unseen SSRC value will 419 interpret it as a new source. It might in fact be a previously 420 existing source that had to change SSRC number due to an SSRC 421 conflict. However, the originator of the previous SSRC ought to have 422 ended the conflicting source by sending an RTCP BYE for it prior to 423 starting to send with the new SSRC, making the new SSRC a new source. 425 3.2.3. Contributing Source (CSRC) 427 The Contributing Source (CSRC) is not a separate identifier. Rather 428 an SSRC identifier is listed as a CSRC in the RTP header of a packet 429 generated by an RTP mixer, if the corresponding SSRC was in the 430 header of one of the packets that contributed to the mix. 432 It is not possible, in general, to extract media represented by an 433 individual CSRC since it is typically the result of a media mixing 434 (merge) operation by an RTP mixer on the individual media streams 435 corresponding to the CSRC identifiers. The exception is the case 436 when only a single CSRC is indicated as this represent forwarding of 437 an RTP stream, possibly modified. The RTP header extension for 438 Mixer-to-Client Audio Level Indication [RFC6465] expands on the 439 receiver's information about a packet with a CSRC list. Due to these 440 restrictions, CSRC will not be considered a fully qualified 441 multiplexing point and will be disregarded in the rest of this 442 document. 444 3.2.4. RTP Payload Type 446 Each RTP stream utilises one or more RTP payload formats. An RTP 447 payload format describes how the output of a particular media codec 448 is framed and encoded into RTP packets. The payload format is 449 identified by the payload type (PT) field in the RTP packet header. 450 The combination of SSRC and PT therefore identifies a specific RTP 451 stream in a specific encoding format. The format definition can be 452 taken from [RFC3551] for statically allocated payload types, but 453 ought to be explicitly defined in signalling, such as SDP, both for 454 static and dynamic payload types. The term "format" here includes 455 those aspects described by out-of-band signalling means; in SDP, the 456 term "format" includes media type, RTP timestamp sampling rate, 457 codec, codec configuration, payload format configurations, and 458 various robustness mechanisms such as redundant encodings [RFC2198]. 460 The RTP payload type is scoped by the sending endpoint within an RTP 461 session. PT has the same meaning across all RTP streams in an RTP 462 session. All SSRCs sent from a single endpoint share the same 463 payload type definitions. The RTP payload type is designed such that 464 only a single payload type is valid at any time instant in the RTP 465 stream's timestamp time line, effectively time-multiplexing different 466 payload types if any change occurs. The payload type can change on a 467 per-packet basis for an SSRC, for example a speech codec making use 468 of generic comfort noise [RFC3389]. If there is a true need to send 469 multiple payload types for the same SSRC that are valid for the same 470 instant, then redundant encodings [RFC2198] can be used. Several 471 additional constraints than the ones mentioned above need to be met 472 to enable this use, one of which is that the combined payload sizes 473 of the different payload types ought not exceed the transport MTU. 474 If it is acceptable to send multiple formats of the same media source 475 as separate RTP streams (with separate SSRC), simulcast 476 [I-D.ietf-mmusic-sdp-simulcast] can be used. 478 Other aspects of RTP payload format use are described in How to Write 479 an RTP Payload Format [RFC8088]. 481 The payload type is not a multiplexing point at the RTP layer (see 482 Appendix A for a detailed discussion of why using the payload type as 483 an RTP multiplexing point does not work). The RTP payload type is, 484 however, used to determine how to consume and decode an RTP stream. 485 The RTP payload type number is sometimes used to associate an RTP 486 stream with the signalling; this is not recommended since a specific 487 payload type value can be used in multiple bundled "m=" sections 488 [I-D.ietf-mmusic-sdp-bundle-negotiation]. This association is only 489 possible if unique RTP payload type numbers are used in each context. 491 3.3. Issues Related to RTP Topologies 493 The impact of how RTP multiplexing is performed will in general vary 494 with how the RTP session participants are interconnected, described 495 by RTP Topology [RFC7667]. 497 Even the most basic use case, denoted Topo-Point-to-Point in 498 [RFC7667], raises a number of considerations that are discussed in 499 detail in following sections. They range over such aspects as: 501 o Does my communication peer support RTP as defined with multiple 502 SSRCs per RTP session? 504 o Do I need network differentiation in form of QoS? 506 o Can the application more easily process and handle the media 507 streams if they are in different RTP sessions? 509 o Do I need to use additional RTP streams for RTP retransmission or 510 FEC? 512 For some point to multi-point topologies (e.g. Topo-ASM and Topo-SSM 513 in [RFC7667]), multicast is used to interconnect the session 514 participants. Special considerations (documented in Section 4.2.3) 515 are then needed as multicast is a one-to-many distribution system. 517 Sometimes an RTP communication can end up in a situation when the 518 communicating peers are not compatible for various reasons: 520 o No common media codec for a media type thus requiring transcoding. 522 o Different support for multiple RTP streams and RTP sessions. 524 o Usage of different media transport protocols, i.e., RTP or other. 526 o Usage of different transport protocols, e.g., UDP, DCCP, or TCP. 528 o Different security solutions, e.g., IPsec, TLS, DTLS, or SRTP with 529 different keying mechanisms. 531 In many situations this is resolved by the inclusion of a translator 532 between the two peers, as described by Topo-PtP-Translator in 533 [RFC7667]. The translator's main purpose is to make the peers look 534 compatible to each other. There can also be other reasons than 535 compatibility to insert a translator in the form of a middlebox or 536 gateway, for example a need to monitor the RTP streams. Beware that 537 changing the stream transport characteristics in the translator, can 538 require thorough understanding of the application logic, specifically 539 any congestion control or media adaptation to ensure appropriate 540 media handling. 542 Within the uses enabled by the RTP standard the point to point 543 topology can contain one to many RTP sessions with one to many media 544 sources per session, each having one or more RTP streams per media 545 source. 547 3.4. Issues Related to RTP and RTCP Protocol 549 Using multiple RTP streams is a well-supported feature of RTP. 550 However, for most implementers or people writing RTP/RTCP 551 applications or extensions attempting to apply multiple streams, it 552 can be unclear when it is most appropriate to add an additional RTP 553 stream in an existing RTP session and when it is better to use 554 multiple RTP sessions. This section discusses the various 555 considerations needed. 557 3.4.1. The RTP Specification 559 RFC 3550 contains some recommendations and a bullet list with 5 560 arguments for different aspects of RTP multiplexing. Please review 561 Section 5.2 of [RFC3550]. Five important aspects are quoted below. 563 1. If, say, two audio streams shared the same RTP session and the 564 same SSRC value, and one were to change encodings and thus acquire 565 a different RTP payload type, there would be no general way of 566 identifying which stream had changed encodings. 568 The first argument is to use different SSRC for each individual RTP 569 stream, which is fundamental to RTP operation. 571 2. An SSRC is defined to identify a single timing and sequence number 572 space. Interleaving multiple payload types would require 573 different timing spaces if the media clock rates differ and would 574 require different sequence number spaces to tell which payload 575 type suffered packet loss. 577 The second argument is advocating against demultiplexing RTP streams 578 within a session based only on their RTP payload type numbers, which 579 still stands as can been seen by the extensive list of issues found 580 in Appendix A. 582 3. The RTCP sender and receiver reports (see Section 6.4) can only 583 describe one timing and sequence number space per SSRC and do not 584 carry a payload type field. 586 The third argument is yet another argument against payload type 587 multiplexing. 589 4. An RTP mixer would not be able to combine interleaved streams of 590 incompatible media into one stream. 592 The fourth argument is against multiplexing RTP packets that require 593 different handling into the same session. In most cases the RTP 594 mixer must embed application logic to handle streams; the separation 595 of streams according to stream type is just another piece of 596 application logic, which might or might not be appropriate for a 597 particular application. One type of application that can mix 598 different media sources blindly is the audio-only telephone bridge, 599 although the ability to do that comes from the well-defined scenario 600 that is aided by use of a single media type, even though individual 601 streams may use incompatible codec types; most other types of 602 applications need application-specific logic to perform the mix 603 correctly. 605 5. Carrying multiple media in one RTP session precludes: the use of 606 different network paths or network resource allocations if 607 appropriate; reception of a subset of the media if desired, for 608 example just audio if video would exceed the available bandwidth; 609 and receiver implementations that use separate processes for the 610 different media, whereas using separate RTP sessions permits 611 either single- or multiple-process implementations. 613 The fifth argument discusses network aspects that are described in 614 Section 4.2. It also goes into aspects of implementation, like Split 615 Component Terminal (see Section 3.10 of [RFC7667]) endpoints where 616 different processes or inter-connected devices handle different 617 aspects of the whole multi-media session. 619 A summary of RFC 3550's view on multiplexing is to use unique SSRCs 620 for anything that is its own media/packet stream, and to use 621 different RTP sessions for media streams that don't share a media 622 type. This document supports the first point; it is very valid. The 623 latter needs further discussion, as imposing a single solution on all 624 usages of RTP is inappropriate. Multiple Media Types in an RTP 625 Session specification [I-D.ietf-avtcore-multi-media-rtp-session] 626 provides a detailed analysis of the potential issues in having 627 multiple media types in the same RTP session. This document provides 628 a wider scope for an RTP session and considers multiple media types 629 in one RTP session as a possible choice for the RTP application 630 designer. 632 3.4.2. Multiple SSRCs in a Session 634 Using multiple SSRCs at one endpoint in an RTP session requires 635 resolving some unclear aspects of the RTP specification. These could 636 potentially lead to some interoperability issues as well as some 637 potential significant inefficiencies, as further discussed in "RTP 638 Considerations for Endpoints Sending Multiple Media Streams" 639 [RFC8108]. An RTP application designer should consider these issues 640 and the possible application impact from lack of appropriate RTP 641 handling or optimization in the peer endpoints. 643 Using multiple RTP sessions can potentially mitigate application 644 issues caused by multiple SSRCs in an RTP session. 646 3.4.3. Binding Related Sources 648 A common problem in a number of various RTP extensions has been how 649 to bind related RTP streams together. This issue is common to both 650 using additional SSRCs and multiple RTP sessions. 652 The solutions can be divided into a few groups: 654 o RTP/RTCP based 656 o Signalling based (SDP) 658 o Grouping related RTP sessions 660 o Grouping SSRCs within an RTP session 662 Most solutions are explicit, but some implicit methods have also been 663 applied to the problem. 665 The SDP-based signalling solutions are: 667 SDP Media Description Grouping: The SDP Grouping Framework [RFC5888] 668 uses various semantics to group any number of media descriptions. 669 This has previously been considered primarily as grouping RTP 670 sessions, but [I-D.ietf-mmusic-sdp-bundle-negotiation] groups 671 multiple media descriptions as a single RTP session. 673 SDP SSRC grouping: Source-Specific Media Attributes in SDP [RFC5576] 674 includes a solution for grouping SSRCs the same way as the 675 Grouping framework groups Media Descriptions. 677 The above grouping constructs support many use cases. Those 678 solutions have shortcomings in cases where the session's dynamic 679 properties are such that it is difficult or a drain on resources to 680 keep the list of related SSRCs up to date. 682 An RTP/RTCP-based grouping solution is to use the RTCP SDES CNAME to 683 bind related RTP streams to an endpoint or to a synchronization 684 context. For applications with a single RTP stream per type (media, 685 source or redundancy stream), CNAME is sufficient for that purpose 686 independent if one or more RTP sessions are used. However, some 687 applications choose not to use CNAME because of perceived complexity 688 or a desire not to implement RTCP and instead use the same SSRC value 689 to bind related RTP streams across multiple RTP sessions. RTP 690 Retransmission [RFC4588] in multiple RTP session mode and Generic FEC 691 [RFC5109] both use the CNAME method to relate the RTP streams, which 692 may work but might have some downsides in RTP sessions with many 693 participating SSRCs. It is not recommended to use identical SSRC 694 values across RTP sessions to relate RTP streams; When an SSRC 695 collision occurs, this will force change of that SSRC in all RTP 696 sessions and thus resynchronize all of them instead of only the 697 single media stream having the collision. 699 Another method to implicitly bind SSRCs is used by RTP Retransmission 700 [RFC4588] when using the same RTP session as the source RTP stream 701 for retransmissions. The receiver missing a packet issues an RTP 702 retransmission request, and then awaits a new SSRC carrying the RTP 703 retransmission payload and where that SSRC is from the same CNAME. 704 This limits a requester to having only one outstanding retransmission 705 request on any new source SSRCs per endpoint. 707 RTP Payload Format Restrictions [I-D.ietf-mmusic-rid] provides an 708 RTP/RTCP based mechanism to unambiguously identify the RTP streams 709 within an RTP session and restrict the streams' payload format 710 parameters in a codec-agnostic way beyond what is provided with the 711 regular payload types. The mapping is done by specifying an "a=rid" 712 value in the SDP offer/answer signalling and having the corresponding 713 RtpStreamId value as an SDES item and an RTP header extension. The 714 RID solution also includes a solution for binding redundancy RTP 715 streams to their original source RTP streams, given that those use 716 RID identifiers. 718 Section 8.3 of the RTP Specification [RFC3550] recommends using a 719 single SSRC space across all RTP sessions for layered coding. Based 720 on the experience so far however, we recommend to use a solution with 721 explicit binding between the RTP streams that is agnostic to the used 722 SSRC values. That way, solutions using multiple RTP streams in a 723 single RTP session and in multiple RTP sessions will use the same 724 type of binding. 726 3.4.4. Forward Error Correction 728 There exist a number of Forward Error Correction (FEC) based schemes 729 for how to reduce the packet loss of the original streams. Most of 730 the FEC schemes protects a single source flow. The protection is 731 achieved by transmitting a certain amount of redundant information 732 that is encoded such that it can repair one or more packet losses 733 over the set of packets the redundant information protects. This 734 sequence of redundant information needs to be transmitted as its own 735 media stream, or in some cases, instead of the original media stream. 736 Thus, many of these schemes create a need for binding related flows 737 as discussed above. Looking at the history of these schemes, there 738 are schemes using multiple SSRCs and schemes using multiple RTP 739 sessions, and some schemes that support both modes of operation. 741 Using multiple RTP sessions supports the case where some set of 742 receivers might not be able to utilise the FEC information. By 743 placing it in a separate RTP session and if separating RTP sessions 744 on transport level, FEC can easily be ignored already on transport 745 level, without considering any RTP layer information. 747 In usages involving multicast, having the FEC information on its own 748 multicast group allows for similar flexibility. This is especially 749 useful when receivers see heterogeneous packet loss rates. A 750 receiver can based on measurment of experienced packet loss decide to 751 join a multicast group with the suitable FEC data repair 752 capabilities. 754 4. Considerations for RTP Multiplexing 756 4.1. Interworking Considerations 758 There are several different kinds of interworking, and this section 759 discusses two; interworking directly between different applications, 760 and interworking of applications through an RTP Translator. The 761 discussion includes the implications of potentially different RTP 762 multiplexing point choices and limitations that have to be considered 763 when working with some legacy applications. 765 4.1.1. Application Interworking 767 It is not uncommon that applications or services of similar but not 768 identical usage, especially the ones intended for interactive 769 communication, encounter a situation where one want to interconnect 770 two or more of these applications. 772 In these cases, one ends up in a situation where one might use a 773 gateway to interconnect applications. This gateway must then either 774 change the multiplexing structure or adhere to the respective 775 limitations in each application. 777 There are two fundamental approaches to building a gateway: using RTP 778 Translator interworking (RTP bridging), where the gateway acts as an 779 RTP Translator with the two interconnected applications being members 780 of the same RTP session; or using Gateway Interworking with RTP 781 termination, where there are independent RTP sessions between each 782 interconnected application and the gateway. 784 4.1.2. RTP Translator Interworking 786 From an RTP perspective, the RTP Translator approach could work if 787 all the applications are using the same codecs with the same payload 788 types, have made the same multiplexing choices, and have the same 789 capabilities in number of simultaneous RTP streams combined with the 790 same set of RTP/RTCP extensions being supported. Unfortunately, this 791 might not always be true. 793 When a gateway is implemented via an RTP Translator, an important 794 consideration is if the two applications being interconnected need to 795 use the same approach to multiplexing. If one side is using RTP 796 session multiplexing and the other is using SSRC multiplexing with 797 BUNDLE [I-D.ietf-mmusic-sdp-bundle-negotiation], it is possible for 798 the RTP translator to map the RTP streams between both sides using 799 some method, e.g. if the number and order of SDP "m=" lines between 800 both sides are the same. There are also challenges with SSRC 801 collision handling since, unless SSRC translation is applied on the 802 RTP translator, there may be a collision on the SSRC multiplexing 803 side that the RTP session multiplexing side will not be aware of. 804 Furthermore, if one of the applications is capable of working in 805 several modes (such as being able to use additional RTP streams in 806 one RTP session or multiple RTP sessions at will), and the other one 807 is not, successful interconnection depends on locking the more 808 flexible application into the operating mode where interconnection 809 can be successful, even if none of the participants are using the 810 less flexible application when the RTP sessions are being created. 812 4.1.3. Gateway Interworking 814 When one terminates RTP sessions at the gateway, there are certain 815 tasks that the gateway has to carry out: 817 o Generating appropriate RTCP reports for all RTP streams (possibly 818 based on incoming RTCP reports), originating from SSRCs controlled 819 by the gateway. 821 o Handling SSRC collision resolution in each application's RTP 822 sessions. 824 o Signalling, choosing and policing appropriate bit-rates for each 825 session. 827 For applications that use any security mechanism, e.g., in the form 828 of SRTP, the gateway needs to be able to decrypt and verify source 829 integrity of the incoming packets, and re-encrypt, integrity protect, 830 and sign the packets as peer in the other application's security 831 context. This is necessary even if all that's needed is a simple 832 remapping of SSRC numbers. If this is done, the gateway also needs 833 to be a member of the security contexts of both sides, of course. 835 The gateway might also need to apply transcoding (for incompatible 836 codec types), media-level adaptations that cannot be solved through 837 media negotiation (such as rescaling for incompatible video size 838 requirements), suppression of content that is known not to be handled 839 in the destination application, or the addition or removal of 840 redundancy coding or scalability layers to fit the needs of the 841 destination domain. 843 From the above, we can see that the gateway needs to have an intimate 844 knowledge of the application requirements; a gateway is by its nature 845 application specific, not a commodity product. 847 These gateways might therefore potentially block application 848 evolution by blocking RTP and RTCP extensions that the applications 849 have been extended with but that are unknown to the gateway. 851 If one uses security functions, like SRTP, and as can be seen from 852 above, they incur both additional risk due to the requirement to have 853 the gateway in the security association between the endpoints (unless 854 the gateway is on the transport level), and additional complexities 855 in form of the decrypt-encrypt cycles needed for each forwarded 856 packet. SRTP, due to its keying structure, also requires that each 857 RTP session needs different master keys, as use of the same key in 858 two RTP sessions can for some ciphers result in two-time pads that 859 completely breaks the confidentiality of the packets. 861 4.1.4. Multiple SSRC Legacy Considerations 863 Historically, the most common RTP use cases have been point-to-point 864 Voice over IP (VoIP) or streaming applications, commonly with no more 865 than one media source per endpoint and media type (typically audio or 866 video). Even in conferencing applications, especially voice-only, 867 the conference focus or bridge has provided a single stream to each 868 participant containing a mix of the other participants. It is also 869 common to have individual RTP sessions between each endpoint and the 870 RTP mixer, meaning that the mixer functions as an RTP-terminating 871 gateway. 873 Endpoints that aren't updated to handle multiple streams following 874 these recommendations can have issues with participating in RTP 875 sessions containing multiple SSRCs within a single session, such as: 877 1. Need to handle more than one stream simultaneously rather than 878 replacing an already existing stream with a new one. 880 2. Be capable of decoding multiple streams simultaneously. 882 3. Be capable of rendering multiple streams simultaneously. 884 This indicates that gateways attempting to interconnect to this class 885 of devices have to make sure that only one RTP stream of each media 886 type gets delivered to the endpoint if it's expecting only one, and 887 that the multiplexing format is what the device expects. It is 888 highly unlikely that RTP translator-based interworking can be made to 889 function successfully in such a context. 891 4.2. Network Considerations 893 The RTP implementer needs to consider that the RTP multiplexing 894 choice also impacts network level mechanisms. 896 4.2.1. Quality of Service 898 Quality of Service mechanisms are either flow based or packet marking 899 based. RSVP [RFC2205] is an example of a flow based mechanism, while 900 Diff-Serv [RFC2474] is an example of a packet marking based one. 902 For a flow based scheme, additional SSRC will receive the same QoS as 903 all other RTP streams being part of the same 5-tuple (protocol, 904 source address, destination address, source port, destination port), 905 which is the most common selector for flow based QoS. 907 For a packet marking based scheme, the method of multiplexing will 908 not affect the possibility to use QoS. Different Differentiated 909 Services Code Points (DSCP) can be assigned to different packets 910 within a flow as well as within an RTP stream. However, care must be 911 taken when considering which forwarding behaviours that are applied 912 on path due to these DSCPs. In some cases the forwarding behaviour 913 can result in packet reordering. For more discussion of this see 914 [RFC7657]. 916 The method for assigning marking to packets can impact what number of 917 RTP sessions to choose. If this marking is done using a network 918 ingress function, it can have issues discriminating the different RTP 919 streams. The network API on the endpoint also needs to be capable of 920 setting the marking on a per-packet basis to reach the full 921 functionality. 923 4.2.2. NAT and Firewall Traversal 925 In today's networks there exist a large number of middleboxes. The 926 ones that normally have most impact on RTP are Network Address 927 Translators (NAT) and Firewalls (FW). 929 Below we analyse and comment on the impact of requiring more 930 underlying transport flows in the presence of NATs and Firewalls: 932 End-Point Port Consumption: A given IP address only has 65536 933 available local ports per transport protocol for all consumers of 934 ports that exist on the machine. This is normally never an issue 935 for an end-user machine. It can become an issue for servers that 936 handle large number of simultaneous streams. However, if the 937 application uses ICE to authenticate STUN requests, a server can 938 serve multiple endpoints from the same local port, and use the 939 whole 5-tuple (source and destination address, source and 940 destination port, protocol) as identifier of flows after having 941 securely bound them to the remote endpoint address using the STUN 942 request. In theory, the minimum number of media server ports 943 needed are the maximum number of simultaneous RTP sessions a 944 single endpoint can use. In practice, implementation will 945 probably benefit from using more server ports to simplify 946 implementation or avoid performance bottlenecks. 948 NAT State: If an endpoint sits behind a NAT, each flow it generates 949 to an external address will result in a state that has to be kept 950 in the NAT. That state is a limited resource. In home or Small 951 Office/Home Office (SOHO) NATs, memory or processing are usually 952 the most limited resources. For large scale NATs serving many 953 internal endpoints, available external ports are likely the scarce 954 resource. Port limitations is primarily a problem for larger 955 centralised NATs where endpoint independent mapping requires each 956 flow to use one port for the external IP address. This affects 957 the maximum number of internal users per external IP address. 958 However, as a comparison, a real-time video conference session 959 with audio and video likely uses less than 10 UDP flows, compared 960 to certain web applications that can use 100+ TCP flows to various 961 servers from a single browser instance. 963 NAT Traversal Extra Delay: Performing the NAT/FW traversal takes a 964 certain amount of time for each flow. It also takes time in a 965 phase of communication between accepting to communicate and the 966 media path being established, which is fairly critical. The best 967 case scenario for additional NAT/FW traversal time after finding 968 the first valid candidate pair following the specified ICE 969 procedures is 1.5*RTT + Ta*(Additional_Flows-1), where Ta is the 970 pacing timer. That assumes a message in one direction, 971 immediately followed by a check back. The reason it isn't more, 972 is that ICE first finds one candidate pair that works prior to 973 attempting to establish multiple flows. Thus, there is no extra 974 time until one has found a working candidate pair. Based on that 975 working pair, the extra time is needed to in parallel establish 976 the, in most cases 2-3, additional flows. However, packet loss 977 causes extra delays, at least 100 ms, which is the minimal 978 retransmission timer for ICE. 980 NAT Traversal Failure Rate: Due to the need to establish more than a 981 single flow through the NAT, there is some risk that establishing 982 the first flow succeeds but that one or more of the additional 983 flows fail. The risk that this happens is hard to quantify, but 984 ought to be fairly low as one flow from the same interfaces has 985 just been successfully established. Thus only rare events such as 986 NAT resource overload, or selecting particular port numbers that 987 are filtered etc., ought to be reasons for failure. 989 Deep Packet Inspection and Multiple Streams: Firewalls differ in how 990 deeply they inspect packets. There exist some risk that deeply 991 inspecting firewalls will have similar legacy issues with multiple 992 SSRCs as some RTP stack implementations. 994 Using additional RTP streams in the same RTP session and transport 995 flow does not introduce any additional NAT traversal complexities per 996 RTP stream. This can be compared with normally one or two additional 997 transport flows per RTP session when using multiple RTP sessions. 998 Additional lower layer transport flows will be needed, unless an 999 explicit de-multiplexing layer is added between RTP and the transport 1000 protocol. At time of writing no such mechanism was defined. 1002 4.2.3. Multicast 1004 Multicast groups provides a powerful tool for a number of real-time 1005 applications, especially the ones that desire broadcast-like 1006 behaviours with one endpoint transmitting to a large number of 1007 receivers, like in IPTV. There is also the RTP/RTCP extension to 1008 better support Source Specific Multicast (SSM) [RFC5760]. Many-to- 1009 many communication, which RTP [RFC3550] was originally built to 1010 support, has several limitations in common with multicast. 1012 One limitation is that, for any group, sender side adaptation with 1013 the intent to suit all receivers would have to adapt to the most 1014 limited receiver experiencing the worst conditions among the group 1015 participants, which imposes degradation for all participants. For 1016 broadcast-type applications with a large number of receivers, this is 1017 not acceptable. Instead, various receiver-based solutions are 1018 employed to ensure that the receivers achieve best possible 1019 performance. By using scalable encoding and placing each scalability 1020 layer in a different multicast group, the receiver can control the 1021 amount of traffic it receives. To have each scalability layer on a 1022 different multicast group, one RTP session per multicast group is 1023 used. 1025 In addition, the transport flow considerations in multicast are a bit 1026 different from unicast; NATs with port translation are not useful in 1027 the multicast environment, meaning that the entire port range of each 1028 multicast address is available for distinguishing between RTP 1029 sessions. 1031 Thus, when using broadcast applications it appears easiest and most 1032 straightforward to use multiple RTP sessions for sending different 1033 media flows used for adapting to network conditions. It is also 1034 common that streams improving transport robustness are sent in their 1035 own multicast group to allow for interworking with legacy or to 1036 support different levels of protection. 1038 Many-to-many applications have different needs and the most 1039 appropriate multiplexing choice will depend on how the actual 1040 application is realized. Multicast applications that are capable of 1041 using sender side congestion control can avoid the use of multiple 1042 multicast sessions and RTP sessions that result from use of receiver 1043 side congestion control. 1045 The properties of a broadcast application using RTP multicast: 1047 1. Uses a group of RTP sessions, not just one. Each endpoint will 1048 need to be a member of a number of RTP sessions in order to 1049 perform well. 1051 2. Within each RTP session, the number of RTP receivers is likely to 1052 be much larger than the number of RTP senders. 1054 3. The applications need signalling functions to identify the 1055 relationships between RTP sessions. 1057 4. The applications need signalling or RTP/RTCP functions to 1058 identify the relationships between SSRCs in different RTP 1059 sessions when needs beyond CNAME exist. 1061 Both broadcast and many-to-many multicast applications share a 1062 signalling requirement; all of the participants need the same RTP and 1063 payload type configuration. Otherwise, A could for example be using 1064 payload type 97 as the video codec H.264 while B thinks it is MPEG-2. 1065 SDP offer/answer [RFC3264] is not appropriate for ensuring this 1066 property in broadcast/multicast context. The signalling aspects of 1067 broadcast/multicast are not explored further in this memo. 1069 Security solutions for this type of group communication are also 1070 challenging. First, the key-management and the security protocol 1071 need to support group communication. Second, source authentication 1072 requires special solutions. For more discussion on this please 1073 review Options for Securing RTP Sessions [RFC7201]. 1075 4.3. Security and Key Management Considerations 1077 When dealing with point-to-point, 2-member RTP sessions only, there 1078 are few security issues that are relevant to the choice of having one 1079 RTP session or multiple RTP sessions. However, there are a few 1080 aspects of multiparty sessions that might warrant consideration. For 1081 general information of possible methods of securing RTP, please 1082 review RTP Security Options [RFC7201]. 1084 4.3.1. Security Context Scope 1086 When using SRTP [RFC3711], the security context scope is important 1087 and can be a necessary differentiation in some applications. As 1088 SRTP's crypto suites are (so far) built around symmetric keys, the 1089 receiver will need to have the same key as the sender. This results 1090 in that no one in a multi-party session can be certain that a 1091 received packet really was sent by the claimed sender and not by 1092 another party having access to the key. The single SRTP algorithm 1093 not having this propery is the TESLA source authentication [RFC4383]. 1094 However, TESLA adds delay to achieve source authentication. In most 1095 cases, symmetric ciphers provide sufficient security properties but 1096 create issues in a few cases. 1098 The first case is when someone leaves a multi-party session and one 1099 wants to ensure that the party that left can no longer access the RTP 1100 streams. This requires that everyone re-keys without disclosing the 1101 new keys to the excluded party. 1103 A second case is when using security as an enforcing mechanism for 1104 stream access differentiation between different receivers. Take for 1105 example a scalable layer or a high quality simulcast version that 1106 only premium users are allowed to access. The mechanism preventing a 1107 receiver from getting the high quality stream can be based on the 1108 stream being encrypted with a key that user can't access without 1109 paying premium, using the key-management to limit access to the key. 1111 SRTP [RFC3711] has no special functions for dealing with different 1112 sets of master keys for different SSRCs. The key-management 1113 functions have different capabilities to establish different sets of 1114 keys, normally on a per-endpoint basis. For example, DTLS-SRTP 1115 [RFC5764] and Security Descriptions [RFC4568] establish different 1116 keys for outgoing and incoming traffic from an endpoint. This key 1117 usage has to be written into the cryptographic context, possibly 1118 associated with different SSRCs. 1120 4.3.2. Key Management for Multi-party Sessions 1122 The capabilities of the key-management combined with the RTP 1123 multiplexing choices affects the resulting security properties, 1124 control over the secured media, and who have access to it. 1126 Multi-party sessions contain at least one RTP stream from each active 1127 participant. Depending on the multi-party topology [RFC7667], each 1128 participant can both send and receive multiple RTP streams. 1130 Transport translator-based sessions and multicast sessions, can 1131 neither use Security Description [RFC4568] nor DTLS-SRTP [RFC5764] 1132 without an extension as each endpoint provides its set of keys. In 1133 centralised conferences, the signalling counterpart is a conference 1134 server, and the transport translator is the media plane unicast 1135 counterpart (to which DTLS messages would be sent). Thus, an 1136 extension like Encrypted Key Transport [I-D.ietf-perc-srtp-ekt-diet] 1137 or a MIKEY [RFC3830] based solution that allows for keying all 1138 session participants with the same master key is needed. 1140 Privacy Enchanced RTP Conferencing (PERC) also enables a different 1141 trust model with semi-trusted media switching RTP middleboxes 1142 [I-D.ietf-perc-private-media-framework]. 1144 4.3.3. Complexity Implications 1146 The usage of security functions can surface complexity implications 1147 from the choice of multiplexing and topology. This becomes 1148 especially evident in RTP topologies having any type of middlebox 1149 that processes or modifies RTP/RTCP packets. While there is very 1150 small overhead for an RTP translator or mixer to rewrite an SSRC 1151 value in the RTP packet of an unencrypted session, the cost is higher 1152 when using cryptographic security functions. For example, if using 1153 SRTP [RFC3711], the actual security context and exact crypto key are 1154 determined by the SSRC field value. If one changes SSRC, the 1155 encryption and authentication must use another key. Thus, changing 1156 the SSRC value implies a decryption using the old SSRC and its 1157 security context, followed by an encryption using the new one. 1159 5. RTP Multiplexing Design Choices 1161 This section discusses how some RTP multiplexing design choices can 1162 be used in applications to achieve certain goals, and a summary of 1163 the implications of such choices. For each design there is 1164 discussion of benefits and downsides. 1166 5.1. Multiple Media Types in One Session 1168 This design uses a single RTP session for multiple different media 1169 types, like audio and video, and possibly also transport robustness 1170 mechanisms like FEC or retransmission. An endpoint can send zero, 1171 one or more media sources per media type, resulting in a number of 1172 RTP streams of various media types for both source and redundancy 1173 streams. 1175 The Advantages: 1177 1. Only a single RTP session is used, which implies: 1179 * Minimal need to keep NAT/FW state. 1181 * Minimal NAT/FW-traversal cost. 1183 * Fate-sharing for all media flows. 1185 * Minimal overhead for security association establishment. 1187 2. Dynamic allocation of RTP streams can be handled almost entirely 1188 at RTP level. How localized this can be kept to RTP level 1189 depends on the application's needs for explicit indication of the 1190 stream usage and how timely that can be signalled. 1192 The Disadvantages: 1194 a. It is less suitable for interworking with other applications that 1195 use individual RTP sessions per media type or multiple sessions 1196 for a single media type, due to the risk of SSRC collision and 1197 thus potential need for SSRC translation. 1199 b. Negotiation of individual bandwidths for the different media 1200 types is currently only possible in SDP when using RID 1201 [I-D.ietf-mmusic-rid]. 1203 c. It is not suitable for Split Component Terminal (see Section 3.10 1204 of [RFC7667]). 1206 d. Flow-based QoS cannot be used to provide separate treatment of 1207 RTP streams compared to others in the single RTP session. 1209 e. If there is significant asymmetry between the RTP streams' RTCP 1210 reporting needs, there are some challenges in configuration and 1211 usage to avoid wasting RTCP reporting on the RTP stream that does 1212 not need that frequent reporting. 1214 f. It is not suitable for applications where some receivers like to 1215 receive only a subset of the RTP streams, especially if multicast 1216 or transport translator is being used. 1218 g. There is some additional concern with legacy implementations that 1219 do not support the RTP specification fully when it comes to 1220 handling multiple SSRC per endpoint, as multiple simultaneous 1221 media types are sent as separate SSRC in the same RTP session. 1223 h. If the applications need finer control over which session 1224 participants that are included in different sets of security 1225 associations, most key-management will have difficulties 1226 establishing such a session. 1228 5.2. Multiple SSRCs of the Same Media Type 1230 In this design, each RTP session serves only a single media type. 1231 The RTP session can contain multiple RTP streams, either from a 1232 single endpoint or from multiple endpoints. This commonly creates a 1233 low number of RTP sessions, typically only one for audio and one for 1234 video, with a corresponding need for two listening ports when using 1235 RTP/RTCP multiplexing [RFC5761]. 1237 The Advantages 1239 1. It works well with Split Component Terminal (see Section 3.10 of 1240 [RFC7667]) where the split is per media type. 1242 2. It enables flow-based QoS with different prioritisation between 1243 media types. 1245 3. For applications with dynamic usage of RTP streams, i.e. 1246 frequently added and removed, having much of the state associated 1247 with the RTP session rather than per individual SSRC can avoid 1248 the need for in-session signalling of meta-information about each 1249 SSRC. 1251 4. There is low overhead for security association establishment. 1253 The Disadvantages 1255 a. There are a slightly higher number of RTP sessions needed 1256 compared to Multiple Media Types in one Session Section 5.1. 1257 This implies: 1259 * More NAT/FW state is needed. 1261 * There is increased NAT/FW-traversal cost in both processing 1262 and delay. 1264 b. There is some potential for concern with legacy implementations 1265 that don't support the RTP specification fully when it comes to 1266 handling multiple SSRC per endpoint. 1268 c. It is not possible to control security association for sets of 1269 RTP streams within the same media type with today's key- 1270 management mechanisms, unless these are split into different RTP 1271 sessions (Section 5.3). 1273 For RTP applications where all RTP streams of the same media type 1274 share same usage, this structure provides efficiency gains in amount 1275 of network state used and provides more fate sharing with other media 1276 flows of the same type. At the same time, it is still maintaining 1277 almost all functionalities for the negotiation signaling of 1278 properties per individual media type, and also enables flow based QoS 1279 prioritisation between media types. It handles multi-party sessions 1280 well, independently of multicast or centralised transport 1281 distribution, as additional sources can dynamically enter and leave 1282 the session. 1284 5.3. Multiple Sessions for One Media Type 1286 This design goes one step further than above (Section 5.2) by using 1287 multiple RTP sessions also for a single media type. The main reason 1288 for going in this direction is that the RTP application needs 1289 separation of the RTP streams due to their usage, such as e.g. 1290 scalability over multicast, simulcast, need for extended QoS 1291 prioritisation, or the need for fine-grained signalling using RTP 1292 session-focused signalling tools. 1294 The Advantages: 1296 1. This is more suitable for multicast usage where receivers can 1297 individually select which RTP sessions they want to participate 1298 in, assuming each RTP session has its own multicast group. 1300 2. The application can indicate its usage of the RTP streams on RTP 1301 session level, in case multiple different usages exist. 1303 3. There is less need for SSRC-specific explicit signalling for each 1304 media stream and thus reduced need for explicit and timely 1305 signalling when RTP streams are added or removed. 1307 4. It enables detailed QoS prioritisation for flow-based mechanisms. 1309 5. It works well with Split Component Terminal (see Section 3.10 of 1310 [RFC7667]). 1312 6. The scope for who is included in a security association can be 1313 structured around the different RTP sessions, thus enabling such 1314 functionality with existing key-management. 1316 The Disadvantages: 1318 a. There is an increased amount of session configuration state 1319 compared to Multiple SSRCs of the Same Media Type, due to the 1320 increased amount of RTP sessions. 1322 b. For RTP streams that are part of scalability, simulcast or 1323 transport robustness, a method to bind sources across multiple 1324 RTP sessions is needed. 1326 c. There is some potential for concern with legacy implementations 1327 that don't support the RTP specification fully when it comes to 1328 handling multiple SSRC per endpoint. 1330 d. There is higher overhead for security association establishment, 1331 due to the increased number of RTP sessions. 1333 e. If the applications need more fine-grained control than per RTP 1334 session over which participants that are included in different 1335 sets of security associations, most of today's key-management 1336 will have difficulties establishing such a session. 1338 For more complex RTP applications that have several different usages 1339 for RTP streams of the same media type, or uses scalability or 1340 simulcast, this solution can enable those functions at the cost of 1341 increased overhead associated with the additional sessions. This 1342 type of structure is suitable for more advanced applications as well 1343 as multicast-based applications requiring differentiation to 1344 different participants. 1346 5.4. Single SSRC per Endpoint 1348 In this design each endpoint in a point-to-point session has only a 1349 single SSRC, thus the RTP session contains only two SSRCs, one local 1350 and one remote. This session can be used both unidirectional, i.e. 1351 only a single RTP stream, or bi-directional, i.e. both endpoints have 1352 one RTP stream each. If the application needs additional media flows 1353 between the endpoints, it will have to establish additional RTP 1354 sessions. 1356 The Advantages: 1358 1. This design has great legacy interoperability potential as it 1359 will not tax any RTP stack implementations. 1361 2. The signalling has good possibilities to negotiate and describe 1362 the exact formats and bitrates for each RTP stream, especially 1363 using today's tools in SDP. 1365 3. It is possible to control security association per RTP stream 1366 with current key-management, since each RTP stream is directly 1367 related to an RTP session, and the most used keying mechanisms 1368 operates on a per-session basis. 1370 The Disadvantages: 1372 a. There is a linear growth of the amount of NAT/FW state with 1373 number of RTP streams. 1375 b. There is increased delay and resource consumption from NAT/FW- 1376 traversal. 1378 c. There are likely larger signalling message and signalling 1379 processing requirements due to the increased amount of session- 1380 related information. 1382 d. There is higher potential for a single RTP stream to fail during 1383 transport between the endpoints, due to the need for separate 1384 NAT/FW- traversal for every RTP stream since there is only one 1385 stream per session. 1387 e. The amount of explicit state for relating RTP streams grows, 1388 depending on how the application relates RTP streams. 1390 f. The port consumption might become a problem for centralised 1391 services, where the central node's port or 5-tuple filter 1392 consumption grows rapidly with the number of sessions. 1394 g. For applications where the RTP stream usage is highly dynamic, 1395 i.e. entering and leaving, the amount of signalling can become 1396 high. Issues can also arise from the need for timely 1397 establishment of additional RTP sessions. 1399 h. If, against the recommendation, the same SSRC value is reused in 1400 multiple RTP sessions rather than being randomly chosen, 1401 interworking with applications that use a different multiplexing 1402 structure will require SSRC translation. 1404 RTP applications with a strong need to interwork with legacy RTP 1405 applications can potentially benefit from this structure. However, a 1406 large number of media descriptions in SDP can also run into issues 1407 with existing implementations. For any application needing a larger 1408 number of media flows, the overhead can become very significant. 1409 This structure is also not suitable for non-mixed multi-party 1410 sessions, as any given RTP stream from each participant, although 1411 having same usage in the application, needs its own RTP session. In 1412 addition, the dynamic behaviour that can arise in multi-party 1413 applications can tax the signalling system and make timely media 1414 establishment more difficult. 1416 5.5. Summary 1418 Both the "Single SSRC per Endpoint" and the "Multiple Media Types in 1419 One Session" are cases that require full explicit signalling of the 1420 media stream relations. However, they operate on two different 1421 levels where the first primarily enables session level binding, and 1422 the second needs SSRC level binding. From another perspective, the 1423 two solutions are the two extreme points when it comes to number of 1424 RTP sessions needed. 1426 The two other designs, "Multiple SSRCs of the Same Media Type" and 1427 "Multiple Sessions for One Media Type", are two examples that 1428 primarily allows for some implicit mapping of the role or usage of 1429 the RTP streams based on which RTP session they appear in. It thus 1430 potentially allows for less signalling and in particular reduces the 1431 need for real-time signalling in sessions with dynamically changing 1432 number of RTP streams. They also represent points in-between the 1433 first two designs when it comes to amount of RTP sessions 1434 established, i.e. representing an attempt to balance the amount of 1435 RTP sessions with the functionality the communication session 1436 provides both on network level and on signalling level. 1438 6. Guidelines 1440 This section contains a number of multi-stream guidelines for 1441 implementers or specification writers. 1443 Do not require use of the same SSRC value across RTP sessions: 1444 As discussed in Section 3.4.3 there exist drawbacks in using the 1445 same SSRC in multiple RTP sessions as a mechanism to bind related 1446 RTP streams together. It is instead recommended to use a 1447 mechanism to explicitly signal the relation, either in RTP/RTCP or 1448 in the signalling mechanism used to establish the RTP session(s). 1450 Use additional RTP streams for additional media sources: In the 1451 cases where an RTP endpoint needs to transmit additional RTP 1452 streams of the same media type in the application, with the same 1453 processing requirements at the network and RTP layers, it is 1454 suggested to send them in the same RTP session. For example a 1455 telepresence room where there are three cameras, and each camera 1456 captures 2 persons sitting at the table, sending each camera as 1457 its own RTP stream within a single RTP session is suggested. 1459 Use additional RTP sessions for streams with different requirements: 1461 When RTP streams have different processing requirements from the 1462 network or the RTP layer at the endpoints, it is suggested that 1463 the different types of streams are put in different RTP sessions. 1465 This includes the case where different participants want different 1466 subsets of the set of RTP streams. 1468 When using multiple RTP sessions, use grouping: When using multiple 1469 RTP session solutions, it is suggested to explicitly group the 1470 involved RTP sessions when needed using a signalling mechanism, 1471 for example The Session Description Protocol (SDP) Grouping 1472 Framework [RFC5888], using some appropriate grouping semantics. 1474 RTP/RTCP Extensions Support Multiple RTP Streams as Well as Multiple 1475 RTP Sessions: 1476 When defining an RTP or RTCP extension, the creator needs to 1477 consider if this extension is applicable to use with additional 1478 SSRCs and multiple RTP sessions. Any extension intended to be 1479 generic must support both. Extensions that are not as generally 1480 applicable will have to consider if interoperability is better 1481 served by defining a single solution or providing both options. 1483 Transport Support Extensions: When defining new RTP/RTCP extensions 1484 intended for transport support, like the retransmission or FEC 1485 mechanisms, they must include support for both multiple RTP 1486 streams in the same RTP session and multiple RTP sessions, such 1487 that application developers can choose freely from the set of 1488 mechanisms without concerning themselves with which of the 1489 multiplexing choices a particular solution supports. 1491 7. IANA Considerations 1493 This document makes no request of IANA. 1495 Note to RFC Editor: this section can be removed on publication as an 1496 RFC. 1498 8. Security Considerations 1500 The security considerations of the RTP specification [RFC3550], any 1501 applicable RTP profile [RFC3551],[RFC4585],[RFC3711], and the 1502 extensions for sending multiple media types in a single RTP session 1503 [I-D.ietf-avtcore-multi-media-rtp-session], RID 1504 [I-D.ietf-mmusic-rid], BUNDLE 1505 [I-D.ietf-mmusic-sdp-bundle-negotiation], [RFC5760], [RFC5761], apply 1506 if selected and thus need to be considered in the evaluation. 1508 There is discussion of the security implications of choosing multiple 1509 SSRC vs multiple RTP sessions in Section 4.3. 1511 9. Contributors 1513 Hui Zheng (Marvin) contributed to WG draft versions -04 and -05 of 1514 the document. 1516 10. Acknowledgments 1518 The Authors like to acknowledge and thank Cullen Jennings, Dale R 1519 Worley, Huang Yihong (Rachel), and Vijay Gurbani for review and 1520 comments. 1522 11. References 1524 11.1. Normative References 1526 [I-D.ietf-avtcore-multi-media-rtp-session] 1527 Westerlund, M., Perkins, C., and J. Lennox, "Sending 1528 Multiple Types of Media in a Single RTP Session", draft- 1529 ietf-avtcore-multi-media-rtp-session-13 (work in 1530 progress), December 2015. 1532 [I-D.ietf-mmusic-rid] 1533 Roach, A., "RTP Payload Format Restrictions", draft-ietf- 1534 mmusic-rid-15 (work in progress), May 2018. 1536 [I-D.ietf-mmusic-sdp-bundle-negotiation] 1537 Holmberg, C., Alvestrand, H., and C. Jennings, 1538 "Negotiating Media Multiplexing Using the Session 1539 Description Protocol (SDP)", draft-ietf-mmusic-sdp-bundle- 1540 negotiation-54 (work in progress), December 2018. 1542 [I-D.ietf-mmusic-sdp-simulcast] 1543 Burman, B., Westerlund, M., Nandakumar, S., and M. Zanaty, 1544 "Using Simulcast in SDP and RTP Sessions", draft-ietf- 1545 mmusic-sdp-simulcast-14 (work in progress), March 2019. 1547 [I-D.ietf-perc-srtp-ekt-diet] 1548 Jennings, C., Mattsson, J., McGrew, D., Wing, D., and F. 1549 Andreasen, "Encrypted Key Transport for DTLS and Secure 1550 RTP", draft-ietf-perc-srtp-ekt-diet-10 (work in progress), 1551 July 2019. 1553 [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. 1554 Jacobson, "RTP: A Transport Protocol for Real-Time 1555 Applications", STD 64, RFC 3550, DOI 10.17487/RFC3550, 1556 July 2003, . 1558 [RFC3551] Schulzrinne, H. and S. Casner, "RTP Profile for Audio and 1559 Video Conferences with Minimal Control", STD 65, RFC 3551, 1560 DOI 10.17487/RFC3551, July 2003, 1561 . 1563 [RFC3711] Baugher, M., McGrew, D., Naslund, M., Carrara, E., and K. 1564 Norrman, "The Secure Real-time Transport Protocol (SRTP)", 1565 RFC 3711, DOI 10.17487/RFC3711, March 2004, 1566 . 1568 [RFC3830] Arkko, J., Carrara, E., Lindholm, F., Naslund, M., and K. 1569 Norrman, "MIKEY: Multimedia Internet KEYing", RFC 3830, 1570 DOI 10.17487/RFC3830, August 2004, 1571 . 1573 [RFC4585] Ott, J., Wenger, S., Sato, N., Burmeister, C., and J. Rey, 1574 "Extended RTP Profile for Real-time Transport Control 1575 Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585, 1576 DOI 10.17487/RFC4585, July 2006, 1577 . 1579 [RFC5576] Lennox, J., Ott, J., and T. Schierl, "Source-Specific 1580 Media Attributes in the Session Description Protocol 1581 (SDP)", RFC 5576, DOI 10.17487/RFC5576, June 2009, 1582 . 1584 [RFC5760] Ott, J., Chesterfield, J., and E. Schooler, "RTP Control 1585 Protocol (RTCP) Extensions for Single-Source Multicast 1586 Sessions with Unicast Feedback", RFC 5760, 1587 DOI 10.17487/RFC5760, February 2010, 1588 . 1590 [RFC5761] Perkins, C. and M. Westerlund, "Multiplexing RTP Data and 1591 Control Packets on a Single Port", RFC 5761, 1592 DOI 10.17487/RFC5761, April 2010, 1593 . 1595 [RFC7656] Lennox, J., Gross, K., Nandakumar, S., Salgueiro, G., and 1596 B. Burman, Ed., "A Taxonomy of Semantics and Mechanisms 1597 for Real-Time Transport Protocol (RTP) Sources", RFC 7656, 1598 DOI 10.17487/RFC7656, November 2015, 1599 . 1601 [RFC7667] Westerlund, M. and S. Wenger, "RTP Topologies", RFC 7667, 1602 DOI 10.17487/RFC7667, November 2015, 1603 . 1605 11.2. Informative References 1607 [ALF] Clark, D. and D. Tennenhouse, "Architectural 1608 Considerations for a New Generation of Protocols", SIGCOMM 1609 Symposium on Communications Architectures and 1610 Protocols (Philadelphia, Pennsylvania), pp. 200--208, IEEE 1611 Computer Communications Review, Vol. 20(4), September 1612 1990. 1614 [I-D.ietf-avtext-rid] 1615 Roach, A., Nandakumar, S., and P. Thatcher, "RTP Stream 1616 Identifier Source Description (SDES)", draft-ietf-avtext- 1617 rid-09 (work in progress), October 2016. 1619 [I-D.ietf-perc-private-media-framework] 1620 Jones, P., Benham, D., and C. Groves, "A Solution 1621 Framework for Private Media in Privacy Enhanced RTP 1622 Conferencing (PERC)", draft-ietf-perc-private-media- 1623 framework-12 (work in progress), June 2019. 1625 [JINGLE] Ludwig, S., Beda, J., Saint-Andre, P., McQueen, R., Egan, 1626 S., and J. Hildebrand, "XEP-0166: Jingle", XMPP.org 1627 https://xmpp.org/extensions/xep-0166.html, September 2018. 1629 [RFC2198] Perkins, C., Kouvelas, I., Hodson, O., Hardman, V., 1630 Handley, M., Bolot, J., Vega-Garcia, A., and S. Fosse- 1631 Parisis, "RTP Payload for Redundant Audio Data", RFC 2198, 1632 DOI 10.17487/RFC2198, September 1997, 1633 . 1635 [RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S. 1636 Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1 1637 Functional Specification", RFC 2205, DOI 10.17487/RFC2205, 1638 September 1997, . 1640 [RFC2474] Nichols, K., Blake, S., Baker, F., and D. Black, 1641 "Definition of the Differentiated Services Field (DS 1642 Field) in the IPv4 and IPv6 Headers", RFC 2474, 1643 DOI 10.17487/RFC2474, December 1998, 1644 . 1646 [RFC2974] Handley, M., Perkins, C., and E. Whelan, "Session 1647 Announcement Protocol", RFC 2974, DOI 10.17487/RFC2974, 1648 October 2000, . 1650 [RFC3261] Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston, 1651 A., Peterson, J., Sparks, R., Handley, M., and E. 1652 Schooler, "SIP: Session Initiation Protocol", RFC 3261, 1653 DOI 10.17487/RFC3261, June 2002, 1654 . 1656 [RFC3264] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model 1657 with Session Description Protocol (SDP)", RFC 3264, 1658 DOI 10.17487/RFC3264, June 2002, 1659 . 1661 [RFC3389] Zopf, R., "Real-time Transport Protocol (RTP) Payload for 1662 Comfort Noise (CN)", RFC 3389, DOI 10.17487/RFC3389, 1663 September 2002, . 1665 [RFC4103] Hellstrom, G. and P. Jones, "RTP Payload for Text 1666 Conversation", RFC 4103, DOI 10.17487/RFC4103, June 2005, 1667 . 1669 [RFC4383] Baugher, M. and E. Carrara, "The Use of Timed Efficient 1670 Stream Loss-Tolerant Authentication (TESLA) in the Secure 1671 Real-time Transport Protocol (SRTP)", RFC 4383, 1672 DOI 10.17487/RFC4383, February 2006, 1673 . 1675 [RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session 1676 Description Protocol", RFC 4566, DOI 10.17487/RFC4566, 1677 July 2006, . 1679 [RFC4568] Andreasen, F., Baugher, M., and D. Wing, "Session 1680 Description Protocol (SDP) Security Descriptions for Media 1681 Streams", RFC 4568, DOI 10.17487/RFC4568, July 2006, 1682 . 1684 [RFC4588] Rey, J., Leon, D., Miyazaki, A., Varsa, V., and R. 1685 Hakenberg, "RTP Retransmission Payload Format", RFC 4588, 1686 DOI 10.17487/RFC4588, July 2006, 1687 . 1689 [RFC5104] Wenger, S., Chandra, U., Westerlund, M., and B. Burman, 1690 "Codec Control Messages in the RTP Audio-Visual Profile 1691 with Feedback (AVPF)", RFC 5104, DOI 10.17487/RFC5104, 1692 February 2008, . 1694 [RFC5109] Li, A., Ed., "RTP Payload Format for Generic Forward Error 1695 Correction", RFC 5109, DOI 10.17487/RFC5109, December 1696 2007, . 1698 [RFC5389] Rosenberg, J., Mahy, R., Matthews, P., and D. Wing, 1699 "Session Traversal Utilities for NAT (STUN)", RFC 5389, 1700 DOI 10.17487/RFC5389, October 2008, 1701 . 1703 [RFC5764] McGrew, D. and E. Rescorla, "Datagram Transport Layer 1704 Security (DTLS) Extension to Establish Keys for the Secure 1705 Real-time Transport Protocol (SRTP)", RFC 5764, 1706 DOI 10.17487/RFC5764, May 2010, 1707 . 1709 [RFC5888] Camarillo, G. and H. Schulzrinne, "The Session Description 1710 Protocol (SDP) Grouping Framework", RFC 5888, 1711 DOI 10.17487/RFC5888, June 2010, 1712 . 1714 [RFC6465] Ivov, E., Ed., Marocco, E., Ed., and J. Lennox, "A Real- 1715 time Transport Protocol (RTP) Header Extension for Mixer- 1716 to-Client Audio Level Indication", RFC 6465, 1717 DOI 10.17487/RFC6465, December 2011, 1718 . 1720 [RFC7201] Westerlund, M. and C. Perkins, "Options for Securing RTP 1721 Sessions", RFC 7201, DOI 10.17487/RFC7201, April 2014, 1722 . 1724 [RFC7657] Black, D., Ed. and P. Jones, "Differentiated Services 1725 (Diffserv) and Real-Time Communication", RFC 7657, 1726 DOI 10.17487/RFC7657, November 2015, 1727 . 1729 [RFC7826] Schulzrinne, H., Rao, A., Lanphier, R., Westerlund, M., 1730 and M. Stiemerling, Ed., "Real-Time Streaming Protocol 1731 Version 2.0", RFC 7826, DOI 10.17487/RFC7826, December 1732 2016, . 1734 [RFC7983] Petit-Huguenin, M. and G. Salgueiro, "Multiplexing Scheme 1735 Updates for Secure Real-time Transport Protocol (SRTP) 1736 Extension for Datagram Transport Layer Security (DTLS)", 1737 RFC 7983, DOI 10.17487/RFC7983, September 2016, 1738 . 1740 [RFC8088] Westerlund, M., "How to Write an RTP Payload Format", 1741 RFC 8088, DOI 10.17487/RFC8088, May 2017, 1742 . 1744 [RFC8108] Lennox, J., Westerlund, M., Wu, Q., and C. Perkins, 1745 "Sending Multiple RTP Streams in a Single RTP Session", 1746 RFC 8108, DOI 10.17487/RFC8108, March 2017, 1747 . 1749 [RFC8445] Keranen, A., Holmberg, C., and J. Rosenberg, "Interactive 1750 Connectivity Establishment (ICE): A Protocol for Network 1751 Address Translator (NAT) Traversal", RFC 8445, 1752 DOI 10.17487/RFC8445, July 2018, 1753 . 1755 Appendix A. Dismissing Payload Type Multiplexing 1757 This section documents a number of reasons why using the payload type 1758 as a multiplexing point is unsuitable for most issues related to 1759 multiple RTP streams. Attempting to use Payload type multiplexing 1760 beyond its defined usage has well known negative effects on RTP 1761 discussed below. To use payload type as the single discriminator for 1762 multiple streams implies that all the different RTP streams are being 1763 sent with the same SSRC, thus using the same timestamp and sequence 1764 number space. This has many effects: 1766 1. Putting constraints on RTP timestamp rate for the multiplexed 1767 media. For example, RTP streams that use different RTP 1768 timestamp rates cannot be combined, as the timestamp values need 1769 to be consistent across all multiplexed media frames. Thus 1770 streams are forced to use the same RTP timestamp rate. When 1771 this is not possible, payload type multiplexing cannot be used. 1773 2. Many RTP payload formats can fragment a media object over 1774 multiple RTP packets, like parts of a video frame. These 1775 payload formats need to determine the order of the fragments to 1776 correctly decode them. Thus, it is important to ensure that all 1777 fragments related to a frame or a similar media object are 1778 transmitted in sequence and without interruptions within the 1779 object. This can relatively simple be solved on the sender side 1780 by ensuring that the fragments of each RTP stream are sent in 1781 sequence. 1783 3. Some media formats require uninterrupted sequence number space 1784 between media parts. These are media formats where any missing 1785 RTP sequence number will result in decoding failure or invoking 1786 a repair mechanism within a single media context. The text/ 1787 T140 payload format [RFC4103] is an example of such a format. 1788 These formats will need a sequence numbering abstraction 1789 function between RTP and the individual RTP stream before being 1790 used with payload type multiplexing. 1792 4. Sending multiple streams in the same sequence number space makes 1793 it impossible to determine which payload type, which stream a 1794 packet loss relates to, and thus to which stream to potentially 1795 apply packet loss concealment or other stream-specific loss 1796 mitigation mechanisms. 1798 5. If RTP Retransmission [RFC4588] is used and there is a loss, it 1799 is possible to ask for the missing packet(s) by SSRC and 1800 sequence number, not by payload type. If only some of the 1801 payload type multiplexed streams are of interest, there is no 1802 way of telling which missing packet(s) belong to the interesting 1803 stream(s) and all lost packets need be requested, wasting 1804 bandwidth. 1806 6. The current RTCP feedback mechanisms are built around providing 1807 feedback on RTP streams based on stream ID (SSRC), packet 1808 (sequence numbers) and time interval (RTP timestamps). There is 1809 almost never a field to indicate which payload type is reported, 1810 so sending feedback for a specific RTP payload type is difficult 1811 without extending existing RTCP reporting. 1813 7. The current RTCP media control messages [RFC5104] specification 1814 is oriented around controlling particular media flows, i.e. 1815 requests are done addressing a particular SSRC. Such mechanisms 1816 would need to be redefined to support payload type multiplexing. 1818 8. The number of payload types are inherently limited. 1819 Accordingly, using payload type multiplexing limits the number 1820 of streams that can be multiplexed and does not scale. This 1821 limitation is exacerbated if one uses solutions like RTP and 1822 RTCP multiplexing [RFC5761] where a number of payload types are 1823 blocked due to the overlap between RTP and RTCP. 1825 9. At times, there is a need to group multiplexed streams and this 1826 is currently possible for RTP sessions and for SSRC, but there 1827 is no defined way to group payload types. 1829 10. It is currently not possible to signal bandwidth requirements 1830 per RTP stream when using payload type multiplexing. 1832 11. Most existing SDP media level attributes cannot be applied on a 1833 per payload type level and would require re-definition in that 1834 context. 1836 12. A legacy endpoint that does not understand the indication that 1837 different RTP payload types are different RTP streams might be 1838 slightly confused by the large amount of possibly overlapping or 1839 identically defined RTP payload types. 1841 Appendix B. Signalling Considerations 1843 Signalling is not an architectural consideration for RTP itself, so 1844 this discussion has been moved to an appendix. However, it is 1845 extremely important for anyone building complete applications, so it 1846 is deserving of discussion. 1848 We document salient issues here that need to be addressed by the WGs 1849 that use some form of signaling to establish RTP sessions. These 1850 issues cannot simply be addressed by tweaking, extending, or 1851 profiling RTP, but require a dedicated and indepth look at the 1852 signaling primitives that set up the RTP sessions. 1854 There exist various signalling solutions for establishing RTP 1855 sessions. Many are SDP [RFC4566] based, however SDP functionality is 1856 also dependent on the signalling protocols carrying the SDP. RTSP 1857 [RFC7826] and SAP [RFC2974] both use SDP in a declarative fashion, 1858 while SIP [RFC3261] uses SDP with the additional definition of Offer/ 1859 Answer [RFC3264]. The impact on signalling and especially SDP needs 1860 to be considered as it can greatly affect how to deploy a certain 1861 multiplexing point choice. 1863 B.1. Session Oriented Properties 1865 One aspect of the existing signalling is that it is focused on RTP 1866 sessions, or at least in the case of SDP the media description. 1867 There are a number of things that are signalled on media description 1868 level but those are not necessarily strictly bound to an RTP session 1869 and could be of interest to signal specifically for a particular RTP 1870 stream (SSRC) within the session. The following properties have been 1871 identified as being potentially useful to signal not only on RTP 1872 session level: 1874 o Bitrate/Bandwidth exist today only at aggregate or as a common 1875 "any RTP stream" limit, unless either codec-specific bandwidth 1876 limiting or RTCP signalling using TMMBR is used. 1878 o Which SSRC that will use which RTP payload type (this will be 1879 visible from the first media packet, but is sometimes useful to 1880 know before packet arrival). 1882 Some of these issues are clearly SDP's problem rather than RTP 1883 limitations. However, if the aim is to deploy an solution using 1884 additional SSRCs that contains several sets of RTP streams with 1885 different properties (encoding/packetization parameter, bit-rate, 1886 etc.), putting each set in a different RTP session would directly 1887 enable negotiation of the parameters for each set. If insisting on 1888 additional SSRC only, a number of signalling extensions are needed to 1889 clarify that there are multiple sets of RTP streams with different 1890 properties and that they need in fact be kept different, since a 1891 single set will not satisfy the application's requirements. 1893 For some parameters, such as RTP payload type, resolution and 1894 framerate, a SSRC-linked mechanism has been proposed in 1895 [I-D.ietf-mmusic-rid] 1897 B.2. SDP Prevents Multiple Media Types 1899 SDP chose to use the m= line both to delineate an RTP session and to 1900 specify the top level of the MIME media type; audio, video, text, 1901 image, application. This media type is used as the top-level media 1902 type for identifying the actual payload format and is bound to a 1903 particular payload type using the rtpmap attribute. This binding has 1904 to be loosened in order to use SDP to describe RTP sessions 1905 containing multiple MIME top level types. 1907 [I-D.ietf-mmusic-sdp-bundle-negotiation] describes how to let 1908 multiple SDP media descriptions use a single underlying transport in 1909 SDP, which allows to define one RTP session with media types having 1910 different MIME top level types. 1912 B.3. Signalling RTP Stream Usage 1914 RTP streams being transported in RTP has some particular usage in an 1915 RTP application. This usage of the RTP stream is in many 1916 applications so far implicitly signalled. For example, an 1917 application might choose to take all incoming audio RTP streams, mix 1918 them and play them out. However, in more advanced applications that 1919 use multiple RTP streams there will be more than a single usage or 1920 purpose among the set of RTP streams being sent or received. RTP 1921 applications will need to signal this usage somehow. The signalling 1922 used will have to identify the RTP streams affected by their RTP- 1923 level identifiers, which means that they have to be identified either 1924 by their session or by their SSRC + session. 1926 In some applications, the receiver cannot utilise the RTP stream at 1927 all before it has received the signalling message describing the RTP 1928 stream and its usage. In other applications, there exists a default 1929 handling that is appropriate. 1931 If all RTP streams in an RTP session are to be treated in the same 1932 way, identifying the session is enough. If SSRCs in a session are to 1933 be treated differently, signalling needs to identify both the session 1934 and the SSRC. 1936 If this signalling affects how any RTP central node, like an RTP 1937 mixer or translator that selects, mixes or processes streams, treats 1938 the streams, the node will also need to receive the same signalling 1939 to know how to treat RTP streams with different usage in the right 1940 fashion. 1942 Authors' Addresses 1944 Magnus Westerlund 1945 Ericsson 1946 Torshamnsgatan 23 1947 SE-164 80 Kista 1948 Sweden 1950 Phone: +46 10 714 82 87 1951 Email: magnus.westerlund@ericsson.com 1953 Bo Burman 1954 Ericsson 1955 Gronlandsgatan 31 1956 SE-164 60 Kista 1957 Sweden 1959 Email: bo.burman@ericsson.com 1961 Colin Perkins 1962 University of Glasgow 1963 School of Computing Science 1964 Glasgow G12 8QQ 1965 United Kingdom 1967 Email: csp@csperkins.org 1969 Harald Tveit Alvestrand 1970 Google 1971 Kungsbron 2 1972 Stockholm 11122 1973 Sweden 1975 Email: harald@alvestrand.no 1976 Roni Even 1977 Huawei 1979 Email: roni.even@huawei.com