idnits 2.17.1 draft-ietf-avtcore-multiplex-guidelines-08.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack a both a reference to RFC 2119 and the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. RFC 2119 keyword, line 559: '... SHOULD be carried in a separate RTP...' RFC 2119 keyword, line 562: '...nd video streams SHOULD NOT be carried...' Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (December 14, 2018) is 1960 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-54) exists of draft-ietf-mmusic-sdp-bundle-negotiation-53 == Outdated reference: A later version (-14) exists of draft-ietf-mmusic-sdp-simulcast-13 == Outdated reference: A later version (-13) exists of draft-ietf-perc-srtp-ekt-diet-09 -- Obsolete informational reference (is this intentional?): RFC 4566 (Obsoleted by RFC 8866) Summary: 1 error (**), 0 flaws (~~), 4 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group M. Westerlund 3 Internet-Draft B. Burman 4 Intended status: Informational Ericsson 5 Expires: June 17, 2019 C. Perkins 6 University of Glasgow 7 H. Alvestrand 8 Google 9 R. Even 10 Huawei 11 December 14, 2018 13 Guidelines for using the Multiplexing Features of RTP to Support 14 Multiple Media Streams 15 draft-ietf-avtcore-multiplex-guidelines-08 17 Abstract 19 The Real-time Transport Protocol (RTP) is a flexible protocol that 20 can be used in a wide range of applications, networks, and system 21 topologies. That flexibility makes for wide applicability, but can 22 complicate the application design process. One particular design 23 question that has received much attention is how to support multiple 24 media streams in RTP. This memo discusses the available options and 25 design trade-offs, and provides guidelines on how to use the 26 multiplexing features of RTP to support multiple media streams. 28 Status of This Memo 30 This Internet-Draft is submitted in full conformance with the 31 provisions of BCP 78 and BCP 79. 33 Internet-Drafts are working documents of the Internet Engineering 34 Task Force (IETF). Note that other groups may also distribute 35 working documents as Internet-Drafts. The list of current Internet- 36 Drafts is at https://datatracker.ietf.org/drafts/current/. 38 Internet-Drafts are draft documents valid for a maximum of six months 39 and may be updated, replaced, or obsoleted by other documents at any 40 time. It is inappropriate to use Internet-Drafts as reference 41 material or to cite them other than as "work in progress." 43 This Internet-Draft will expire on June 17, 2019. 45 Copyright Notice 47 Copyright (c) 2018 IETF Trust and the persons identified as the 48 document authors. All rights reserved. 50 This document is subject to BCP 78 and the IETF Trust's Legal 51 Provisions Relating to IETF Documents 52 (https://trustee.ietf.org/license-info) in effect on the date of 53 publication of this document. Please review these documents 54 carefully, as they describe your rights and restrictions with respect 55 to this document. Code Components extracted from this document must 56 include Simplified BSD License text as described in Section 4.e of 57 the Trust Legal Provisions and are provided without warranty as 58 described in the Simplified BSD License. 60 Table of Contents 62 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 63 2. Definitions . . . . . . . . . . . . . . . . . . . . . . . . . 4 64 2.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 4 65 2.2. Subjects Out of Scope . . . . . . . . . . . . . . . . . . 5 66 3. RTP Multiplexing Overview . . . . . . . . . . . . . . . . . . 5 67 3.1. Reasons for Multiplexing and Grouping RTP Streams . . . . 5 68 3.2. RTP Multiplexing Points . . . . . . . . . . . . . . . . . 6 69 3.2.1. RTP Session . . . . . . . . . . . . . . . . . . . . . 7 70 3.2.2. Synchronisation Source (SSRC) . . . . . . . . . . . . 8 71 3.2.3. Contributing Source (CSRC) . . . . . . . . . . . . . 10 72 3.2.4. RTP Payload Type . . . . . . . . . . . . . . . . . . 10 73 3.3. Issues Related to RTP Topologies . . . . . . . . . . . . 11 74 3.4. Issues Related to RTP and RTCP Protocol . . . . . . . . . 12 75 3.4.1. The RTP Specification . . . . . . . . . . . . . . . . 13 76 3.4.2. Multiple SSRCs in a Session . . . . . . . . . . . . . 15 77 3.4.3. Binding Related Sources . . . . . . . . . . . . . . . 15 78 3.4.4. Forward Error Correction . . . . . . . . . . . . . . 17 79 4. Considerations for RTP Multiplexing . . . . . . . . . . . . . 17 80 4.1. Interworking Considerations . . . . . . . . . . . . . . . 17 81 4.1.1. Application Interworking . . . . . . . . . . . . . . 17 82 4.1.2. RTP Translator Interworking . . . . . . . . . . . . . 18 83 4.1.3. Gateway Interworking . . . . . . . . . . . . . . . . 18 84 4.1.4. Multiple SSRC Legacy Considerations . . . . . . . . . 19 85 4.2. Network Considerations . . . . . . . . . . . . . . . . . 20 86 4.2.1. Quality of Service . . . . . . . . . . . . . . . . . 20 87 4.2.2. NAT and Firewall Traversal . . . . . . . . . . . . . 21 88 4.2.3. Multicast . . . . . . . . . . . . . . . . . . . . . . 22 89 4.3. Security and Key Management Considerations . . . . . . . 24 90 4.3.1. Security Context Scope . . . . . . . . . . . . . . . 24 91 4.3.2. Key Management for Multi-party sessions . . . . . . . 25 92 4.3.3. Complexity Implications . . . . . . . . . . . . . . . 25 94 5. RTP Multiplexing Design Choices . . . . . . . . . . . . . . . 26 95 5.1. Multiple Media Types in one Session . . . . . . . . . . . 26 96 5.2. Multiple SSRCs of the Same Media Type . . . . . . . . . . 27 97 5.3. Multiple Sessions for one Media type . . . . . . . . . . 28 98 5.4. Single SSRC per Endpoint . . . . . . . . . . . . . . . . 29 99 5.5. Summary . . . . . . . . . . . . . . . . . . . . . . . . . 31 100 6. Guidelines . . . . . . . . . . . . . . . . . . . . . . . . . 31 101 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 32 102 8. Security Considerations . . . . . . . . . . . . . . . . . . . 33 103 9. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 33 104 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 33 105 10.1. Normative References . . . . . . . . . . . . . . . . . . 33 106 10.2. Informative References . . . . . . . . . . . . . . . . . 33 107 Appendix A. Dismissing Payload Type Multiplexing . . . . . . . . 37 108 Appendix B. Signalling Considerations . . . . . . . . . . . . . 39 109 B.1. Session Oriented Properties . . . . . . . . . . . . . . . 40 110 B.2. SDP Prevents Multiple Media Types . . . . . . . . . . . . 40 111 B.3. Signalling RTP stream Usage . . . . . . . . . . . . . . . 41 112 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 41 114 1. Introduction 116 The Real-time Transport Protocol (RTP) [RFC3550] is a commonly used 117 protocol for real-time media transport. It is a protocol that 118 provides great flexibility and can support a large set of different 119 applications. RTP was from the beginning designed for multiple 120 participants in a communication session. It supports many topology 121 paradigms and usages, as defined in [RFC7667]. RTP has several 122 multiplexing points designed for different purposes. These enable 123 support of multiple RTP streams and switching between different 124 encoding or packetization of the media. By using multiple RTP 125 sessions, sets of RTP streams can be structured for efficient 126 processing or identification. Thus, the question for any RTP 127 application designer is how to best use the RTP session, the RTP 128 stream identifier (SSRC), and the RTP payload type to meet the 129 application's needs. 131 There have been increased interest in more advanced usage of RTP. 132 For example, multiple RTP streams can be used when a single endpoint 133 has multiple media sources (like multiple cameras or microphones) 134 that need to be sent simultaneously. Consequently, questions are 135 raised regarding the most appropriate RTP usage. The limitations in 136 some implementations, RTP/RTCP extensions, and signalling has also 137 been exposed. The authors also hope that clarification on the 138 usefulness of some functionalities in RTP will result in more 139 complete implementations in the future. 141 The purpose of this document is to provide clear information about 142 the possibilities of RTP when it comes to multiplexing. The RTP 143 application designer needs to understand the implications that come 144 from a particular usage of the RTP multiplexing points. The document 145 will recommend against some usages as being unsuitable, in general or 146 for particular purposes. 148 The document starts with some definitions and then goes into the 149 existing RTP functionalities around multiplexing. Both the desired 150 behaviour and the implications of a particular behaviour depend on 151 which topologies are used, which requires some consideration. This 152 is followed by a discussion of some choices in multiplexing behaviour 153 and their impacts. Some designs of RTP usage are discussed. 154 Finally, some guidelines and examples are provided. 156 2. Definitions 158 2.1. Terminology 160 The definitions in Section 3 of [RFC3550] are referenced normatively. 162 The taxonomy defined in [RFC7656] is referenced normatively. 164 The following terms and abbreviations are used in this document: 166 Multiparty: A communication situation including multiple endpoints. 167 In this document, it will be used to refer to situations where 168 more than two endpoints communicate. 170 Multiplexing: The operation of taking multiple entities as input, 171 aggregating them onto some common resource while keeping the 172 individual entities addressable such that they can later be fully 173 and unambiguously separated (de-multiplexed) again. 175 RTP Receiver: An Endpoint or Middlebox receiving RTP streams and 176 RTCP messages. It uses at least one SSRC to send RTCP messages. 177 An RTP Receiver may also be an RTP sender. 179 RTP Sender: An Endpoint sending one or more RTP streams, but also 180 sending RTCP messages. 182 RTP Session Group: One or more RTP sessions that are used together 183 to perform some function. Examples are multiple RTP sessions used 184 to carry different layers of a layered encoding. In an RTP 185 Session Group, CNAMEs are assumed to be valid across all RTP 186 sessions, and designate synchronisation contexts that can cross 187 RTP sessions; i.e. SSRCs that map to a common CNAME can be assumed 188 to have RTCP SR timing information derived from a common clock 189 such that they can be synchronised for playout. 191 Signalling: The process of configuring endpoints to participate in 192 one or more RTP sessions. 194 Note: The above definitions of RTP Receiver and RTP sender are 195 intended to be consistent with the usage in [RFC3550]. 197 2.2. Subjects Out of Scope 199 This document is focused on issues that affect RTP. Thus, issues 200 that involve signalling protocols, such as whether SIP, Jingle or 201 some other protocol is in use for session configuration, the 202 particular syntaxes used to define RTP session properties, or the 203 constraints imposed by particular choices in the signalling 204 protocols, are mentioned only as examples in order to describe the 205 RTP issues more precisely. 207 This document assumes the applications will use RTCP. While there 208 are applications that don't send RTCP, they do not conform to the RTP 209 specification, and thus can be regarded as reusing the RTP packet 210 format but not implementing the RTP protocol. 212 3. RTP Multiplexing Overview 214 3.1. Reasons for Multiplexing and Grouping RTP Streams 216 There are several reasons why an endpoint might choose to send 217 multiple media streams. In the below discussion, please keep in mind 218 that the reasons for having multiple RTP streams vary and include but 219 are not limited to the following: 221 o Multiple media sources 223 o Multiple RTP streams might be needed to represent one media source 224 (for instance when using layered encodings) 226 o A retransmission stream might repeat some parts of the content of 227 another RTP stream 229 o An FEC stream might provide material that can be used to repair 230 another RTP stream 232 o Alternative encodings, for instance using different codecs for the 233 same audio stream 235 o Alternative formats, for instance multiple resolutions of the same 236 video stream 238 For each of these reasons, it is necessary to decide if each 239 additional RTP stream is sent within the same RTP session as the 240 other RTP streams, or if it is necessary to use additional RTP 241 sessions to group the RTP streams. The choice suitable for one 242 reason, might not be the choice suitable for another reason. The 243 clearest understanding is associated with multiplexing multiple media 244 sources of the same media type. However, all reasons warrant 245 discussion and clarification on how to deal with them. As the 246 discussion below will show, in reality we cannot choose a single one 247 of SSRC or RTP session multiplexing solutions. To utilise RTP well 248 and as efficiently as possible, both are needed. The real issue is 249 finding the right guidance on when to create additional RTP sessions 250 and when additional RTP streams in the same RTP session is the right 251 choice. 253 3.2. RTP Multiplexing Points 255 This section describes the multiplexing points present in the RTP 256 protocol that can be used to distinguish RTP streams and groups of 257 RTP streams. Figure 1 outlines the process of demultiplexing 258 incoming RTP streams: 260 | 261 | packets 262 +-- v 263 | +------------+ 264 | | Socket | Transport Protocol Demultiplexing 265 | +------------+ 266 | || || 267 RTP | RTP/ || |+-----> SCTP ( ...and any other protocols) 268 Session | RTCP || +------> STUN (multiplexed using same port) 269 +-- || 270 +-- || 271 | (split by SSRC) 272 | || || || 273 | || || || 274 RTP | +--+ +--+ +--+ 275 Streams | |PB| |PB| |PB| Jitter buffer, process RTCP, etc. 276 | +--+ +--+ +--+ 277 +-- | | | 278 (select decoder based on PT) 279 +-- | / | 280 | +----+ | 281 | / | | 282 Payload | +---+ +---+ +---+ 283 Formats | |Dec| |Dec| |Dec| Decoders 284 | +---+ +---+ +---+ 285 +-- 287 Figure 1: RTP Demultiplexing Process 289 3.2.1. RTP Session 291 An RTP Session is the highest semantic layer in the RTP protocol, and 292 represents an association between a group of communicating endpoints. 293 RTP does not contain a session identifier, yet RTP sessions must be 294 possible to separate both across different endpoints and within a 295 single endpoint. 297 For RTP session separation across endpoints, the set of participants 298 that form an RTP session is defined as those that share a single 299 synchronisation source space [RFC3550]. That is, if a group of 300 participants are each aware of the synchronisation source identifiers 301 belonging to the other participants, then those participants are in a 302 single RTP session. A participant can become aware of a 303 synchronisation source identifier by receiving an RTP packet 304 containing it in the SSRC field or CSRC list, by receiving an RTCP 305 packet mentioning it in an SSRC field, or through signalling (e.g., 306 the Session Description Protocol (SDP) [RFC4566] "a=ssrc:" attribute 308 [RFC5576]). Thus, the scope of an RTP session is determined by the 309 participants' network interconnection topology, in combination with 310 RTP and RTCP forwarding strategies deployed by the endpoints and any 311 middleboxes, and by the signalling. 313 For RTP session separation within a single endpoint, RTP relies on 314 the underlying transport layer, and on the signalling to identify RTP 315 sessions in a manner that is meaningful to the application. A single 316 endpoint can have one or more transport flows for the same RTP 317 session, and a single RTP session can span multiple transport layer 318 flows. The signalling layer might give RTP sessions an explicit 319 identifier, or the identification might be implicit based on the 320 addresses and ports used. Accordingly, a single RTP session can have 321 multiple associated identifiers, explicit and implicit, belonging to 322 different contexts. For example, when running RTP on top of UDP/IP, 323 an endpoint can identify and delimit an RTP session from other RTP 324 sessions by receiving the multiple UDP flows used as identified based 325 on their UDP source and destination IP addresses and UDP port 326 numbers. Another example is SDP media descriptions (the "m=" line 327 and the following associated lines) signals the transport flow and 328 RTP session configuration for the endpoints part of the RTP session. 329 SDP grouping framework [RFC5888] allows labeling of the media 330 descriptions, for example used so that RTP Session Groups can be 331 created. With Negotiating Media Multiplexing Using the Session 332 Description Protocol (SDP)[I-D.ietf-mmusic-sdp-bundle-negotiation], 333 multiple media descriptions where each represents the RTP streams 334 sent or received for a media source are part of a common RTP session. 336 The RTP protocol makes no normative statements about the relationship 337 between different RTP sessions, however the applications that use 338 more than one RTP session will have some higher layer understanding 339 of the relationship between the sessions they create. 341 3.2.2. Synchronisation Source (SSRC) 343 A synchronisation source (SSRC) identifies an source of an RTP stream 344 or an RTP receiver when sending RTCP. Every endpoint has at least 345 one SSRC identifier, even if it does not send RTP packets. RTP 346 endpoints that are only RTP receivers still send RTCP and use their 347 SSRC identifiers in the RTCP packets they send. An endpoint can have 348 multiple SSRC identifiers if it sends multiple RTP streams. 349 Endpoints that are both RTP sender and RTP receiver use the same SSRC 350 in both roles. 352 The SSRC is a 32-bit identifier. It is present in every RTP and RTCP 353 packet header, and in the payload of some RTCP packet types. It can 354 also be present in SDP signalling. Unless pre-signalled, e.g. using 355 the SDP "a=ssrc:" attribute [RFC5576], the SSRC is chosen at random. 357 It is not dependent on the network address of the endpoint, and is 358 intended to be unique within an RTP session. SSRC collisions can 359 occur, and are handled as specified in [RFC3550] and [RFC5576], 360 resulting in the SSRC of the colliding RTP streams or receivers 361 changing. An endpoint that changes its network transport address 362 during a session have to choose a new SSRC identifier to avoid being 363 interpreted as looped source, unless the transport layer mechanism, 364 e.g ICE [RFC8445], handles such changes. 366 SSRC identifiers that belong to the same synchronisation context 367 (i.e., that represent RTP streams that can be synchronised using 368 information in RTCP SR packets) use identical CNAME chunks in 369 corresponding RTCP SDES packets. SDP signalling can also be used to 370 provide explicit SSRC grouping [RFC5576]. 372 In some cases, the same SSRC identifier value is used to relate 373 streams in two different RTP sessions, such as in RTP retransmission 374 [RFC4588]. This is to be avoided since there is no guarantee that 375 SSRC values are unique across RTP sessions. For the RTP 376 retransmission [RFC4588] case it is recommended to use explicit 377 binding of the source RTP stream and the redundancy stream, e.g. 378 using the RepairedRtpStreamId RTCP SDES item [I-D.ietf-avtext-rid]. 380 Note that RTP sequence number and RTP timestamp are scoped by the 381 SSRC and thus specific per RTP stream. 383 Different types of entities use a SSRC to identify themselves, as 384 follows: 386 A real media source: Uses the SSRC to identify a "physical" media 387 source. 389 A conceptual media source: Uses the SSRC to identify the result of 390 applying some filtering function in a network node, for example a 391 filtering function in an RTP mixer that provides the most active 392 speaker based on some criteria, or a mix representing a set of 393 other sources. 395 An RTP receiver: Uses the SSRC to identify itself as the source of 396 its RTCP reports. 398 Note that an endpoint that generates more than one media type, e.g. 399 a conference participant sending both audio and video, need not (and 400 should not) use the same SSRC value across RTP sessions. RTCP 401 compound packets containing the CNAME SDES item is the designated 402 method to bind an SSRC to a CNAME, effectively cross-correlating 403 SSRCs within and between RTP Sessions as coming from the same 404 endpoint. The main property attributed to SSRCs associated with the 405 same CNAME is that they are from a particular synchronisation context 406 and can be synchronised at playback. 408 An RTP receiver receiving a previously unseen SSRC value will 409 interpret it as a new source. It might in fact be a previously 410 existing source that had to change SSRC number due to an SSRC 411 conflict. However, the originator of the previous SSRC ought to have 412 ended the conflicting source by sending an RTCP BYE for it prior to 413 starting to send with the new SSRC, so the new SSRC is anyway 414 effectively a new source. 416 3.2.3. Contributing Source (CSRC) 418 The Contributing Source (CSRC) is not a separate identifier. Rather 419 an SSRC identifier is listed as a CSRC in the RTP header of a packet 420 generated by an RTP mixer, if the corresponding SSRC was in the 421 header of one of the packets that contributed to the mix. 423 It is not possible, in general, to extract media represented by an 424 individual CSRC since it is typically the result of a media mixing 425 (merge) operation by an RTP mixer on the individual media streams 426 corresponding to the CSRC identifiers. The exception is the case 427 when only a single CSRC is indicated as this represent forwarding of 428 an RTP stream, possibly modified. The RTP header extension for 429 Mixer-to-Client Audio Level Indication [RFC6465] expands on the 430 receiver's information about a packet with a CSRC list. Due to these 431 restrictions, CSRC will not be considered a fully qualified 432 multiplexing point and will be disregarded in the rest of this 433 document. 435 3.2.4. RTP Payload Type 437 Each RTP stream utilises one or more RTP payload formats. An RTP 438 payload format describes how the output of a particular media codec 439 is framed and encoded into RTP packets. The payload format used is 440 identified by the payload type (PT) field in the RTP packet header. 441 The combination of SSRC and PT therefore identifies a specific RTP 442 stream encoding format. The format definition can be taken from 443 [RFC3551] for statically allocated payload types, but ought to be 444 explicitly defined in signalling, such as SDP, both for static and 445 dynamic payload types. The term "format" here includes whatever can 446 be described by out-of-band signalling means. In SDP, the term 447 "format" includes media type, RTP timestamp sampling rate, codec, 448 codec configuration, payload format configurations, and various 449 robustness mechanisms such as redundant encodings [RFC2198]. 451 The RTP payload type is scoped by the sending endpoint within an RTP 452 session. PT has the same meaning across all RTP streams in an RTP 453 session. All SSRCs sent from a single endpoint share the same 454 payload type definitions. The RTP payload type is designed such that 455 only a single payload type is valid at any time instant in the RTP 456 stream's timestamp time line, effectively time-multiplexing different 457 payload types if any change occurs. The payload type used can change 458 on a per-packet basis for an SSRC, for example a speech codec making 459 use of generic comfort noise [RFC3389]. If there is a true need to 460 send multiple payload types for the same SSRC that are valid for the 461 same instant, then redundant encodings [RFC2198] can be used. 462 Several additional constraints than the ones mentioned above need to 463 be met to enable this use, one of which is that the combined payload 464 sizes of the different payload types ought not exceed the transport 465 MTU. If it is acceptable to send multiple formats of the same media 466 source as separate RTP streams (with separate SSRC), simulcast 467 [I-D.ietf-mmusic-sdp-simulcast] can be used. 469 Other aspects of RTP payload format use are described in How to Write 470 an RTP Payload Format [RFC8088]. 472 The payload type is not a multiplexing point at the RTP layer (see 473 Appendix A for a detailed discussion of why using the payload type as 474 an RTP multiplexing point does not work). The RTP payload type is, 475 however, used to determine how to consume and decode an RTP stream. 476 The RTP payload type number is sometimes used to associate an RTP 477 stream with the signalling; this is not recommended since a specific 478 payload type value can be used in multiple bundled "m=" sections 479 [I-D.ietf-mmusic-sdp-bundle-negotiation]. This association is only 480 possible if unique RTP payload type numbers are used in each context. 482 3.3. Issues Related to RTP Topologies 484 The impact of how RTP multiplexing is performed will in general vary 485 with how the RTP session participants are interconnected, described 486 by RTP Topology [RFC7667]. 488 Even the most basic use case, denoted Topo-Point-to-Point in 489 [RFC7667], raises a number of considerations that are discussed in 490 detail in following sections. They range over such aspects as: 492 o Does my communication peer support RTP as defined with multiple 493 SSRCs per RTP session? 495 o Do I need network differentiation in form of QoS? 497 o Can the application more easily process and handle the media 498 streams if they are in different RTP sessions? 500 o Do I need to use additional RTP streams for RTP retransmission or 501 FEC? 503 For some point to multi-point topologies (e.g. Topo-ASM and Topo-SSM 504 in [RFC7667]), multicast is used to interconnect the session 505 participants. Special considerations (documented in Section 4.2.3) 506 are then needed as multicast is a one-to-many distribution system. 508 Sometimes an RTP communication can end up in a situation when the 509 communicating peers are not compatible for various reasons: 511 o No common media codec for a media type thus requiring transcoding. 513 o Different support for multiple RTP streams and RTP sessions. 515 o Usage of different media transport protocols, i.e., RTP or other. 517 o Usage of different transport protocols, e.g., UDP, DCCP, or TCP. 519 o Different security solutions, e.g., IPsec, TLS, DTLS, or SRTP with 520 different keying mechanisms. 522 In many situations this is resolved by the inclusion of a translator 523 between the two peers, as described by Topo-PtP-Translator in 524 [RFC7667]. The translator's main purpose is to make the peers look 525 compatible to each other. There can also be other reasons than 526 compatibility to insert a translator in the form of a middlebox or 527 gateway, for example a need to monitor the RTP streams. If the 528 stream transport characteristics are changed by the translator, 529 appropriate media handling can require thorough understanding of the 530 application logic, specifically any congestion control or media 531 adaptation. 533 The point to point topology can contain one to many RTP sessions with 534 one to many media sources per session, each having one or more RTP 535 streams per media source. 537 3.4. Issues Related to RTP and RTCP Protocol 539 Using multiple RTP streams is a well-supported feature of RTP. 540 However, for most implementers or people writing RTP/RTCP 541 applications or extensions attempting to apply multiple streams, it 542 can be unclear when it is most appropriate to add an additional RTP 543 stream in an existing RTP session and when it is better to use 544 multiple RTP sessions. This section discusses the various 545 considerations needed. 547 3.4.1. The RTP Specification 549 RFC 3550 contains some recommendations and a bullet list with 5 550 arguments for different aspects of RTP multiplexing. Let's review 551 Section 5.2 of [RFC3550], reproduced below: 553 "For efficient protocol processing, the number of multiplexing points 554 should be minimised, as described in the integrated layer processing 555 design principle [ALF]. In RTP, multiplexing is provided by the 556 destination transport address (network address and port number) which 557 is different for each RTP session. For example, in a teleconference 558 composed of audio and video media encoded separately, each medium 559 SHOULD be carried in a separate RTP session with its own destination 560 transport address. 562 Separate audio and video streams SHOULD NOT be carried in a single 563 RTP session and demultiplexed based on the payload type or SSRC 564 fields. Interleaving packets with different RTP media types but 565 using the same SSRC would introduce several problems: 567 1. If, say, two audio streams shared the same RTP session and the 568 same SSRC value, and one were to change encodings and thus 569 acquire a different RTP payload type, there would be no general 570 way of identifying which stream had changed encodings. 572 2. An SSRC is defined to identify a single timing and sequence 573 number space. Interleaving multiple payload types would require 574 different timing spaces if the media clock rates differ and would 575 require different sequence number spaces to tell which payload 576 type suffered packet loss. 578 3. The RTCP sender and receiver reports (see Section 6.4) can only 579 describe one timing and sequence number space per SSRC and do not 580 carry a payload type field. 582 4. An RTP mixer would not be able to combine interleaved streams of 583 incompatible media into one stream. 585 5. Carrying multiple media in one RTP session precludes: the use of 586 different network paths or network resource allocations if 587 appropriate; reception of a subset of the media if desired, for 588 example just audio if video would exceed the available bandwidth; 589 and receiver implementations that use separate processes for the 590 different media, whereas using separate RTP sessions permits 591 either single- or multiple-process implementations. 593 Using a different SSRC for each medium but sending them in the same 594 RTP session would avoid the first three problems but not the last 595 two. 597 On the other hand, multiplexing multiple related sources of the same 598 medium in one RTP session using different SSRC values is the norm for 599 multicast sessions. The problems listed above don't apply: an RTP 600 mixer can combine multiple audio sources, for example, and the same 601 treatment is applicable for all of them. It might also be 602 appropriate to multiplex streams of the same medium using different 603 SSRC values in other scenarios where the last two problems do not 604 apply." 606 Let's consider one argument at a time. The first argument is for 607 using different SSRC for each individual RTP stream, which is 608 fundamental to RTP operation. 610 The second argument is advocating against demultiplexing RTP streams 611 within a session based on their RTP payload type numbers, which still 612 stands as can been seen by the extensive list of issues found in 613 Appendix A. 615 The third argument is yet another argument against payload type 616 multiplexing. 618 The fourth argument is against multiplexing RTP packets that require 619 different handling into the same session. As we saw in the 620 discussion of RTP mixers, the RTP mixer must embed application logic 621 to handle streams anyway; the separation of streams according to 622 stream type is just another piece of application logic, which might 623 or might not be appropriate for a particular application. One type 624 of application that can mix different media sources "blindly" is the 625 audio-only "telephone" bridge; most other types of applications need 626 application-specific logic to perform the mix correctly. 628 The fifth argument discusses network aspects that we will discuss 629 more below in Section 4.2. It also goes into aspects of 630 implementation, like Split Component Terminal (see Section 3.10 of 631 [RFC7667]) endpoints where different processes or inter-connected 632 devices handle different aspects of the whole multi-media session. 634 A summary of RFC 3550's view on multiplexing is to use unique SSRCs 635 for anything that is its own media/packet stream, and to use 636 different RTP sessions for media streams that don't share a media 637 type. This document supports the first point; it is very valid. The 638 latter needs further discussion, as imposing a single solution on all 639 usages of RTP is inappropriate. Multiple Media Types in an RTP 640 Session specification [I-D.ietf-avtcore-multi-media-rtp-session] 641 provides a detailed analysis of the potential issues in having 642 multiple media types in the same RTP session. This document provides 643 a wider scope for an RTP session and considers multiple media types 644 in one RTP session as a possible choice for the RTP application 645 designer. 647 3.4.2. Multiple SSRCs in a Session 649 Using multiple SSRCs at one endpoint in an RTP session requires 650 resolving some unclear aspects of the RTP specification. These could 651 potentially lead to some interoperability issues as well as some 652 potential significant inefficiencies, as further discussed in "RTP 653 Considerations for Endpoints Sending Multiple Media Streams" 654 [RFC8108]. An RTP application designer should consider these issues 655 and the possible application impact from lack of appropriate RTP 656 handling or optimization in the peer endpoints. 658 Using multiple RTP sessions can potentially mitigate application 659 issues caused by multiple SSRCs in an RTP session. 661 3.4.3. Binding Related Sources 663 A common problem in a number of various RTP extensions has been how 664 to bind related RTP streams together. This issue is common to both 665 using additional SSRCs and multiple RTP sessions. 667 The solutions can be divided into a few groups: 669 o RTP/RTCP based 671 o Signalling based (SDP) 673 o Grouping related RTP sessions 675 o Grouping SSRCs within an RTP session 677 Most solutions are explicit, but some implicit methods have also been 678 applied to the problem. 680 The SDP-based signalling solutions are: 682 SDP Media Description Grouping: The SDP Grouping Framework [RFC5888] 683 uses various semantics to group any number of media descriptions. 684 These has previously been considered primarily as grouping RTP 685 sessions, [I-D.ietf-mmusic-sdp-bundle-negotiation] groups multiple 686 media descriptions as a single RTP session. 688 SDP SSRC grouping: Source-Specific Media Attributes in SDP [RFC5576] 689 includes a solution for grouping SSRCs the same way as the 690 Grouping framework groups Media Descriptions. 692 This supports a lot of use cases. All these solutions have 693 shortcomings in cases where the session's dynamic properties are such 694 that it is difficult or resource consuming to keep the list of 695 related SSRCs up to date. 697 An RTP/RTCP-based solution is to use the RTCP SDES CNAME to bind the 698 RTP streams to an endpoint or synchronization context. For 699 applications with a single RTP stream per type (Media, Source or 700 Redundancy) this is sufficient independent if one or more RTP 701 sessions are used. However, some applications choose not to use it 702 because of perceived complexity or a desire not to implement RTCP and 703 instead use the same SSRC value to bind related RTP streams across 704 multiple RTP sessions. RTP Retransmission [RFC4588] in multiple RTP 705 session mode and Generic FEC [RFC5109] both use this method. This 706 method may work but might have some downsides in RTP sessions with 707 many participating SSRCs. When an SSRC collision occurs, this will 708 force one to change SSRC in all RTP sessions and thus resynchronize 709 all of them instead of only the single media stream having the 710 collision. Therefore, it is not recommended to use identical SSRC 711 values to relate RTP streams. 713 Another solution to bind SSRCs is an implicit method used by RTP 714 Retransmission [RFC4588] when doing retransmissions in the same RTP 715 session as the source RTP stream. The receiver missing a packet 716 issues an RTP retransmission request, and then awaits a new SSRC 717 carrying the RTP retransmission payload and where that SSRC is from 718 the same CNAME. This limits a requester to having only one 719 outstanding request on any new source SSRCs per endpoint. 721 RTP Payload Format Restrictions [I-D.ietf-mmusic-rid] provides an 722 RTP/RTCP based mechanism to unambiguously identify the RTP streams 723 within an RTP session and restrict the streams' payload format 724 parameters in a codec-agnostic way beyond what is provided with the 725 regular Payload Types. The mapping is done by specifying an "a=rid" 726 value in the SDP offer/answer signalling and having the corresponding 727 "rtp-stream-id" value as an SDES item and an RTP header extension. 728 The RID solution also includes a solution for binding redundancy RTP 729 streams to their original source RTP streams, given that those use 730 RID identifiers. 732 It can be noted that Section 8.3 of the RTP Specification [RFC3550] 733 recommends using a single SSRC space across all RTP sessions for 734 layered coding. Based on the experience so far however, we recommend 735 to use a solution doing explicit binding between the RTP streams so 736 what the used SSRC values are do not matter. That way solutions 737 using multiple RTP streams in a single RTP session and multiple RTP 738 sessions uses the same solution. 740 3.4.4. Forward Error Correction 742 There exist a number of Forward Error Correction (FEC) based schemes 743 for how to reduce the packet loss of the original streams. Most of 744 the FEC schemes will protect a single source flow. The protection is 745 achieved by transmitting a certain amount of redundant information 746 that is encoded such that it can repair one or more packet losses 747 over the set of packets the redundant information protects. This 748 sequence of redundant information also needs to be transmitted as its 749 own media stream, or in some cases, instead of the original media 750 stream. Thus, many of these schemes create a need for binding 751 related flows as discussed above. Looking at the history of these 752 schemes, there are schemes using multiple SSRCs and schemes using 753 multiple RTP sessions, and some schemes that support both modes of 754 operation. 756 Using multiple RTP sessions supports the case where some set of 757 receivers might not be able to utilise the FEC information. By 758 placing it in a separate RTP session and if separating RTP sessions 759 on transport level, FEC can easily be ignored already on transport 760 level. 762 In usages involving multicast, having the FEC information on its own 763 multicast group allows for similar flexibility. This is especially 764 useful when receivers see very heterogeneous packet loss rates. 765 Those receivers that are not seeing packet loss don't need to join 766 the multicast group with the FEC data, and so avoid the overhead of 767 receiving unnecessary FEC packets, for example. 769 4. Considerations for RTP Multiplexing 771 4.1. Interworking Considerations 773 There are several different kinds of interworking, and this section 774 discusses two; interworking between different applications including 775 the implications of potentially different RTP multiplexing point 776 choices and limitations that have to be considered when working with 777 some legacy applications. 779 4.1.1. Application Interworking 781 It is not uncommon that applications or services of similar but not 782 identical usage, especially the ones intended for interactive 783 communication, encounter a situation where one want to interconnect 784 two or more of these applications. 786 In these cases, one ends up in a situation where one might use a 787 gateway to interconnect applications. This gateway must then either 788 change the multiplexing structure or adhere to the respective 789 limitations in each application. 791 There are two fundamental approaches to building a gateway: using an 792 RTP Translator interworking (RTP bridging), where the gateway acts as 793 an RTP Translator, with the two applications being members of the 794 same RTP session; or Gateway Interworking with RTP termination, where 795 there are independent RTP sessions running from each interconnected 796 application to the gateway. 798 4.1.2. RTP Translator Interworking 800 From an RTP perspective, the RTP Translator approach could work if 801 all the applications are using the same codecs with the same payload 802 types, have made the same multiplexing choices, and have the same 803 capabilities in number of simultaneous RTP streams combined with the 804 same set of RTP/RTCP extensions being supported. Unfortunately, this 805 might not always be true. 807 When a gateway is implemented via an RTP Translator, an important 808 consideration is if the two applications being interconnected need to 809 use the same approach to multiplexing. If one side is using RTP 810 session multiplexing and the other is using SSRC multiplexing with 811 bundle, it is possible for the RTP translator to map the RTP streams 812 between both sides if the order of SDP "m=" lines between both sides 813 are the same. There are also challenges with SSRC collision handling 814 since there may be a collision on the SSRC multiplexing side but the 815 RTP session multiplexing side will not be aware of any collision 816 unless SSRC translation is applied on the RTP translator. 817 Furthermore, if one of the applications is capable of working in 818 several modes (such as being able to use additional RTP streams in 819 one RTP session or multiple RTP sessions at will), and the other one 820 is not, successful interconnection depends on locking the more 821 flexible application into the operating mode where interconnection 822 can be successful, even if no participants are using the less 823 flexible application when the RTP sessions are being created. 825 4.1.3. Gateway Interworking 827 When one terminates RTP sessions at the gateway, there are certain 828 tasks that the gateway has to carry out: 830 o Generating appropriate RTCP reports for all RTP streams (possibly 831 based on incoming RTCP reports), originating from SSRCs controlled 832 by the gateway. 834 o Handling SSRC collision resolution in each application's RTP 835 sessions. 837 o Signalling, choosing and policing appropriate bit-rates for each 838 session. 840 For applications that uses any security mechanism, e.g., in the form 841 of SRTP, the gateway needs to be able to decrypt incoming packets and 842 re-encrypt them in the other application's security context. This is 843 necessary even if all that's needed is a simple remapping of SSRC 844 numbers. If this is done, the gateway also needs to be a member of 845 the security contexts of both sides, of course. 847 Other tasks a gateway might need to apply include transcoding (for 848 incompatible codec types), media-level adaptations that cannot be 849 solved through media negotiation (such as rescaling for incompatible 850 video size requirements), suppression of content that is known not to 851 be handled in the destination application, or the addition or removal 852 of redundancy coding or scalability layers to fit the needs of the 853 destination domain. 855 From the above, we can see that the gateway needs to have an intimate 856 knowledge of the application requirements; a gateway is by its nature 857 application specific, not a commodity product. 859 This fact reveals the potential for these gateways to block 860 application evolution by blocking RTP and RTCP extensions that the 861 applications have been extended with but that are unknown to the 862 gateway. 864 If one uses security functions, like SRTP, and as can be seen from 865 above, they incur both additional risk due to the requirement to have 866 the gateway in the security association between the endpoints (unless 867 the gateway is on the transport level), and additional complexities 868 in form of the decrypt-encrypt cycles needed for each forwarded 869 packet. SRTP, due to its keying structure, also requires that each 870 RTP session needs different master keys, as use of the same key in 871 two RTP sessions can for some ciphers result in two-time pads that 872 completely breaks the confidentiality of the packets. 874 4.1.4. Multiple SSRC Legacy Considerations 876 Historically, the most common RTP use cases have been point to point 877 Voice over IP (VoIP) or streaming applications, commonly with no more 878 than one media source per endpoint and media type (typically audio or 879 video). Even in conferencing applications, especially voice-only, 880 the conference focus or bridge has provided a single stream with a 881 mix of the other participants to each participant. It is also common 882 to have individual RTP sessions between each endpoint and the RTP 883 mixer, meaning that the mixer functions as an RTP-terminating 884 gateway. 886 When establishing RTP sessions that can contain endpoints that aren't 887 updated to handle multiple streams following these recommendations, a 888 particular application can have issues with multiple SSRCs within a 889 single session. These issues include: 891 1. Need to handle more than one stream simultaneously rather than 892 replacing an already existing stream with a new one. 894 2. Be capable of decoding multiple streams simultaneously. 896 3. Be capable of rendering multiple streams simultaneously. 898 This indicates that gateways attempting to interconnect to this class 899 of devices has to make sure that only one RTP stream of each type 900 gets delivered to the endpoint if it's expecting only one, and that 901 the multiplexing format is what the device expects. It is highly 902 unlikely that RTP translator-based interworking can be made to 903 function successfully in such a context. 905 4.2. Network Considerations 907 The RTP multiplexing choice has impact on network level mechanisms 908 that need to be considered by the implementer. 910 4.2.1. Quality of Service 912 When it comes to Quality of Service mechanisms, they are either flow 913 based or packet marking based. RSVP [RFC2205] is an example of a 914 flow based mechanism, while Diff-Serv [RFC2474] is an example of a 915 packet marking based one. For a packet marking based scheme, the 916 method of multiplexing will not affect the possibility to use QoS. 918 However, for a flow based scheme there is a clear difference between 919 the multiplexing methods. Additional SSRC will result in all RTP 920 streams being part of the same 5-tuple (protocol, source address, 921 destination address, source port, destination port) which is the most 922 common selector for flow based QoS. 924 It must also be noted that packet marking based QoS mechanisms can 925 have limitations. A general observation is that different 926 Differentiated Services Code Points (DSCP) can be assigned to 927 different packets within a flow as well as within an RTP stream. 928 However, care must be taken when considering which forwarding 929 behaviours that are applied on path due to these DSCPs. In some 930 cases the forwarding behaviour can result in packet reordering. For 931 more discussion of this see [RFC7657]. 933 The method for assigning marking to packets can impact what number of 934 RTP sessions to choose. If this marking is done using a network 935 ingress function, it can have issues discriminating the different RTP 936 streams. The network API on the endpoint also needs to be capable of 937 setting the marking on a per-packet basis to reach the full 938 functionality. 940 4.2.2. NAT and Firewall Traversal 942 In today's network there exist a large number of middleboxes. The 943 ones that normally have most impact on RTP are Network Address 944 Translators (NAT) and Firewalls (FW). 946 Below we analyse and comment on the impact of requiring more 947 underlying transport flows in the presence of NATs and Firewalls: 949 End-Point Port Consumption: A given IP address only has 65536 950 available local ports per transport protocol for all consumers of 951 ports that exist on the machine. This is normally never an issue 952 for an end-user machine. It can become an issue for servers that 953 handle large number of simultaneous streams. However, if the 954 application uses ICE to authenticate STUN requests, a server can 955 serve multiple endpoints from the same local port, and use the 956 whole 5-tuple (source and destination address, source and 957 destination port, protocol) as identifier of flows after having 958 securely bound them to the remote endpoint address using the STUN 959 request. In theory the minimum number of media server ports 960 needed are the maximum number of simultaneous RTP Sessions a 961 single endpoint can use. In practice, implementation will 962 probably benefit from using more server ports to simplify 963 implementation or avoid performance bottlenecks. 965 NAT State: If an endpoint sits behind a NAT, each flow it generates 966 to an external address will result in a state that has to be kept 967 in the NAT. That state is a limited resource. In home or Small 968 Office/Home Office (SOHO) NATs, memory or processing are usually 969 the most limited resources. For large scale NATs serving many 970 internal endpoints, available external ports are likely the scarce 971 resource. Port limitations is primarily a problem for larger 972 centralised NATs where endpoint independent mapping requires each 973 flow to use one port for the external IP address. This affects 974 the maximum number of internal users per external IP address. 975 However, it is worth pointing out that a real-time video 976 conference session with audio and video is likely using less than 977 10 UDP flows, compared to certain web applications that can use 978 100+ TCP flows to various servers from a single browser instance. 980 NAT Traversal Extra Delay: Performing the NAT/FW traversal takes a 981 certain amount of time for each flow. It also takes time in a 982 phase of communication between accepting to communicate and the 983 media path being established which is fairly critical. The best 984 case scenario for how much extra time it takes after finding the 985 first valid candidate pair following the specified ICE procedures 986 are: 1.5*RTT + Ta*(Additional_Flows-1), where Ta is the pacing 987 timer. That assumes a message in one direction, and then an 988 immediate triggered check back. The reason it isn't more, is that 989 ICE first finds one candidate pair that works prior to attempting 990 to establish multiple flows. Thus, there is no extra time until 991 one has found a working candidate pair. Based on that working 992 pair the needed extra time is to in parallel establish the, in 993 most cases 2-3, additional flows. However, packet loss causes 994 extra delays, at least 100 ms, which is the minimal retransmission 995 timer for ICE. 997 NAT Traversal Failure Rate: Due to the need to establish more than a 998 single flow through the NAT, there is some risk that establishing 999 the first flow succeeds but that one or more of the additional 1000 flows fail. The risk that this happens is hard to quantify, but 1001 ought to be fairly low as one flow from the same interfaces has 1002 just been successfully established. Thus only rare events such as 1003 NAT resource overload, or selecting particular port numbers that 1004 are filtered etc., ought to be reasons for failure. 1006 Deep Packet Inspection and Multiple Streams: Firewalls differ in how 1007 deeply they inspect packets. There exist some potential that 1008 deeply inspecting firewalls will have similar legacy issues with 1009 multiple SSRCs as some stack implementations. 1011 Using additional RTP streams in the same RTP session and transport 1012 flow does not introduce any additional NAT traversal complexities per 1013 RTP stream. This can be compared with normally one or two additional 1014 transport flows per RTP session when using multiple RTP sessions. 1015 Additional lower layer transport flows will be needed, unless an 1016 explicit de-multiplexing layer is added between RTP and the transport 1017 protocol. At time of writing no such mechanism was defined. 1019 4.2.3. Multicast 1021 Multicast groups provides a powerful tool for a number of real-time 1022 applications, especially the ones that desire broadcast-like 1023 behaviours with one endpoint transmitting to a large number of 1024 receivers, like in IPTV. There are also the RTP/RTCP extension to 1025 better support Source Specific Multicast (SSM) [RFC5760]. Another 1026 application is the Many to Many communication, which RTP [RFC3550] 1027 was originally built to support, but the multicast semantics do 1028 result in a certain number of limitations. 1030 One limitation is that for any group, sender side adaptation to the 1031 actual receiver properties causes degradation for all participants to 1032 what is supported by the receiver with the worst conditions among the 1033 group participants. For broadcast type of applications this is not 1034 acceptable. Instead, various receiver-based solutions are employed 1035 to ensure that the receivers achieve best possible performance. By 1036 using scalable encoding and placing each scalability layer in a 1037 different multicast group, the receiver can control the amount of 1038 traffic it receives. To have each scalability layer on a different 1039 multicast group, one RTP session per multicast group is used. 1041 In addition, the transport flow considerations in multicast are a bit 1042 different from unicast; NATs with port translation are not useful in 1043 the multicast environment, meaning that the entire port range of each 1044 multicast address is available for distinguishing between RTP 1045 sessions. 1047 Thus, when using broadcast applications it appears easiest and most 1048 straightforward to use multiple RTP sessions for sending different 1049 media flows used for adapting to network conditions. It is also 1050 common that streams that improve transport robustness are sent in 1051 their own multicast group to allow for interworking with legacy or to 1052 support different levels of protection. 1054 For many to many applications there are different needs. Here, the 1055 most appropriate choice will depend on how the actual application is 1056 realized. With sender side congestion control there might not exist 1057 any benefit with using multiple RTP sessions. 1059 The properties of a broadcast application using RTP multicast: 1061 1. Uses a group of RTP sessions, not one. Each endpoint will need 1062 to be a member of a number of RTP sessions in order to perform 1063 well. 1065 2. Within each RTP session, the number of RTP receivers is likely to 1066 be much larger than the number of RTP senders. 1068 3. The applications need signalling functions to identify the 1069 relationships between RTP sessions. 1071 4. The applications need signalling or RTP/RTCP functions to 1072 identify the relationships between SSRCs in different RTP 1073 sessions when needs beyond CNAME exist. 1075 Both broadcast and many to many multicast applications do share a 1076 signalling requirement; all of the participants will need to have the 1077 same RTP and payload type configuration. Otherwise, A could for 1078 example be using payload type 97 as the video codec H.264 while B 1079 thinks it is MPEG-2. It is to be noted that SDP offer/answer 1080 [RFC3264] is not appropriate for ensuring this property in broadcast/ 1081 multicast context. The signalling aspects of broadcast/multicast are 1082 not explored further in this memo. 1084 Security solutions for this type of group communications are also 1085 challenging. First, the key-management and the security protocol 1086 need to support group communication. Second, source authentication 1087 requires special solutions. For more discussion on this please 1088 review Options for Securing RTP Sessions [RFC7201]. 1090 4.3. Security and Key Management Considerations 1092 When dealing with point-to-point, 2-member RTP sessions only, there 1093 are few security issues that are relevant to the choice of having one 1094 RTP session or multiple RTP sessions. However, there are a few 1095 aspects of multiparty sessions that might warrant consideration. For 1096 general information of possible methods of securing RTP, please 1097 review RTP Security Options [RFC7201]. 1099 4.3.1. Security Context Scope 1101 When using SRTP [RFC3711] the security context scope is important and 1102 can be a necessary differentiation in some applications. As SRTP's 1103 crypto suites are (so far) built around symmetric keys, the receiver 1104 will need to have the same key as the sender. This results in that 1105 no one in a multi-party session can be certain that a received packet 1106 really was sent by the claimed sender and not by another party having 1107 access to the key. At least unless TESLA source authentication 1108 [RFC4383], which adds delay to achieve source authentication. In 1109 most cases symmetric ciphers provide sufficient security properties, 1110 but there are a few cases where this does create issues. 1112 The first case is when someone leaves a multi-party session and one 1113 wants to ensure that the party that left can no longer access the RTP 1114 streams. This requires that everyone re-keys without disclosing the 1115 keys to the excluded party. 1117 A second case is when using security as an enforcing mechanism for 1118 differentiation. Take for example a scalable layer or a high quality 1119 simulcast version that only premium users are allowed to access. The 1120 mechanism preventing a receiver from getting the high quality stream 1121 can be based on the stream being encrypted with a key that user can't 1122 access without paying premium, having the key-management limit access 1123 to the key. 1125 SRTP [RFC3711] has no special functions for dealing with different 1126 sets of master keys for different SSRCs. The key-management 1127 functions have different capabilities to establish different sets of 1128 keys, normally on a per endpoint basis. For example, DTLS-SRTP 1129 [RFC5764] and Security Descriptions [RFC4568] establish different 1130 keys for outgoing and incoming traffic from an endpoint. This key 1131 usage has to be written into the cryptographic context, possibly 1132 associated with different SSRCs. 1134 4.3.2. Key Management for Multi-party sessions 1136 Performing key-management for multi-party sessions can be a 1137 challenge. This section considers some of the issues. 1139 Multi-party sessions, such as transport translator based sessions and 1140 multicast sessions, can neither use Security Description [RFC4568] 1141 nor DTLS-SRTP [RFC5764] without an extension as each endpoint 1142 provides its set of keys. In centralised conferences, the signalling 1143 counterpart is a conference server and the media plane unicast 1144 counterpart (to which DTLS messages would be sent) is the transport 1145 translator. Thus, an extension like Encrypted Key Transport 1146 [I-D.ietf-perc-srtp-ekt-diet] or a MIKEY [RFC3830] based solution 1147 that allows for keying all session participants with the same master 1148 key is needed. 1150 4.3.3. Complexity Implications 1152 The usage of security functions can surface complexity implications 1153 from the choice of multiplexing and topology. This becomes 1154 especially evident in RTP topologies having any type of middlebox 1155 that processes or modifies RTP/RTCP packets. Where there is very 1156 small overhead for an RTP translator or mixer to rewrite an SSRC 1157 value in the RTP packet of an unencrypted session, the cost is higher 1158 when using cryptographic security functions. For example, if using 1159 SRTP [RFC3711], the actual security context and exact crypto key are 1160 determined by the SSRC field value. If one changes SSRC, the 1161 encryption and authentication must use another key. Thus, changing 1162 the SSRC value implies a decryption using the old SSRC and its 1163 security context, followed by an encryption using the new one. 1165 5. RTP Multiplexing Design Choices 1167 This section discusses how some RTP multiplexing design choices can 1168 be used in applications to achieve certain goals, and a summary of 1169 the implications of such choices. For each design there is 1170 discussion of benefits and downsides. 1172 5.1. Multiple Media Types in one Session 1174 This design uses a single RTP session for multiple different media 1175 types, like audio and video, and possibly also transport robustness 1176 mechanisms like FEC or Retransmission. An endpoint can have zero, 1177 one or more media sources per media type, resulting in a number of 1178 RTP streams of various media types and both source and redundancy 1179 type. 1181 The Pros: 1183 1. Single RTP session which implies: 1185 * Minimal NAT/FW state. 1187 * Minimal NAT/FW Traversal Cost. 1189 * Fate-sharing for all media flows. 1191 2. Can handle dynamic allocations of RTP streams well on an RTP 1192 level. Depends on the application's needs for explicit 1193 indication of the stream usage and how timely that can be 1194 signalled. 1196 3. Minimal overhead for security association establishment. 1198 The Cons: 1200 a. Less suitable for interworking with other applications that uses 1201 individual RTP sessions per media type or multiple sessions for a 1202 single media type, due to the potential need of SSRC translation. 1204 b. Negotiation of bandwidth for the different media types is 1205 currently only possible using RID [I-D.ietf-mmusic-rid] in SDP. 1207 c. Not suitable for Split Component Terminal (see Section 3.10 of 1208 [RFC7667]). 1210 d. Flow-based QoS cannot provide separate treatment of RTP streams 1211 compared to others in the single RTP session. 1213 e. If there is significant asymmetry between the RTP streams' RTCP 1214 reporting needs, there are some challenges in configuration and 1215 usage to avoid wasting RTCP reporting on the RTP stream that does 1216 not need that frequent reporting. 1218 f. Not suitable for applications where some receivers like to 1219 receive only a subset of the RTP streams, especially if multicast 1220 or transport translator is being used. 1222 g. Additional concern with legacy implementations that do not 1223 support the RTP specification fully when it comes to handling 1224 multiple SSRC per endpoint, as also multiple simultaneous media 1225 types need to be handled. 1227 h. If the applications need finer control over which session 1228 participants that are included in different sets of security 1229 associations, most key-management will have difficulties 1230 establishing such a session. 1232 5.2. Multiple SSRCs of the Same Media Type 1234 In this design, each RTP session serves only a single media type. 1235 The RTP session can contain multiple RTP streams, either from a 1236 single endpoint or from multiple endpoints. This commonly creates a 1237 low number of RTP sessions, typically only one for audio and one for 1238 video, with a corresponding need for two listening ports when using 1239 RTP/RTCP multiplexing. 1241 The Pros: 1243 1. Works well with Split Component Terminal (see Section 3.10 of 1244 [RFC7667]) where the split is per media type. 1246 2. Enables Flow-based QoS with different prioritisation between 1247 media types. 1249 3. For applications with dynamic usage of RTP streams, i.e. 1250 frequently added and removed, having much of the state associated 1251 with the RTP session rather than per individual SSRC can avoid 1252 the need for in-session signalling of meta-information about each 1253 SSRC. 1255 4. Low overhead for security association establishment. 1257 The Cons: 1259 a. Slightly higher number of RTP sessions needed compared to 1260 Multiple Media Types in one Session Section 5.1. This implies: 1262 * More NAT/FW state 1264 * Increased NAT/FW Traversal Cost in both processing and delay. 1266 b. Some potential for concern with legacy implementations that don't 1267 support the RTP specification fully when it comes to handling 1268 multiple SSRC per endpoint. 1270 c. Not possible to control security association for sets of RTP 1271 streams within the same media type with today's key- management 1272 mechanisms, unless these are split into different RTP sessions. 1274 For RTP applications where all RTP streams of the same media type 1275 share same usage, this structure provides efficiency gains in amount 1276 of network state used and provides more fate sharing with other media 1277 flows of the same type. At the same time, it is still maintaining 1278 almost all functionalities when it comes to negotiation in the 1279 signalling of the properties for the individual media type, and also 1280 enables flow based QoS prioritisation between media types. It 1281 handles multi-party session well, independently of multicast or 1282 centralised transport distribution, as additional sources can 1283 dynamically enter and leave the session. 1285 5.3. Multiple Sessions for one Media type 1287 This design goes one step further than above (Section 5.2) by using 1288 multiple RTP sessions also for a single media type. The main reason 1289 for going in this direction is that the RTP application needs 1290 separation of the RTP streams due to their usage. Some typical 1291 reasons for going to this design are scalability over multicast, 1292 simulcast, need for extended QoS prioritisation of RTP streams due to 1293 their usage in the application, or the need for fine- grained 1294 signalling using today's tools. 1296 The Pros: 1298 1. More suitable for multicast usage where receivers can 1299 individually select which RTP sessions they want to participate 1300 in, assuming each RTP session has its own multicast group. 1302 2. The application can indicate its usage of the RTP streams on RTP 1303 session level, in case multiple different usages exist. 1305 3. Less need for SSRC specific explicit signalling for each media 1306 stream and thus reduced need for explicit and timely signalling. 1308 4. Enables detailed QoS prioritisation for flow-based mechanisms. 1310 5. Works well with Split Component Terminal (see Section 3.10 of 1311 [RFC7667]). 1313 6. The scope for who is included in a security association can be 1314 structured around the different RTP sessions, thus enabling such 1315 functionality with existing key-management. 1317 The Cons: 1319 a. Increases the amount of RTP sessions compared to Multiple SSRCs 1320 of the Same Media Type. 1322 b. Increased amount of session configuration state. 1324 c. For RTP streams that are part of scalability, simulcast or 1325 transport robustness, a method to bind sources across multiple 1326 RTP sessions is needed. 1328 d. Some potential for concern with legacy implementations that does 1329 not support the RTP specification fully when it comes to handling 1330 multiple SSRC per endpoint. 1332 e. Higher overhead for security association establishment due to the 1333 increased number of RTP sessions. 1335 f. If the applications need finer control than on RTP session level 1336 over which participants that are included in different sets of 1337 security associations, most of today's key-management will have 1338 difficulties establishing such a session. 1340 For more complex RTP applications that have several different usages 1341 for RTP streams of the same media type, or uses scalability or 1342 simulcast, this solution can enable those functions at the cost of 1343 increased overhead associated with the additional sessions. This 1344 type of structure is suitable for more advanced applications as well 1345 as multicast-based applications requiring differentiation to 1346 different participants. 1348 5.4. Single SSRC per Endpoint 1350 In this design each endpoint in a point-to-point session has only a 1351 single SSRC, thus the RTP session contains only two SSRCs, one local 1352 and one remote. This session can be used both unidirectional, i.e. 1353 only a single RTP stream or bi-directional, i.e. both endpoints have 1354 one RTP stream each. If the application needs additional media flows 1355 between the endpoints, they will have to establish additional RTP 1356 sessions. 1358 The Pros: 1360 1. This design has great legacy interoperability potential as it 1361 will not tax any RTP stack implementations. 1363 2. The signalling has good possibilities to negotiate and describe 1364 the exact formats and bit-rates for each RTP stream, especially 1365 using today's tools in SDP. 1367 3. It is possible to control security association per RTP stream 1368 with current key-management, since each RTP stream is directly 1369 related to an RTP session, and the most used keying mechanisms 1370 operates on a per-session basis. 1372 The Cons: 1374 a. The number of RTP sessions grows directly in proportion with the 1375 number of RTP streams, which has the implications: 1377 * Linear growth of the amount of NAT/FW state with number of RTP 1378 streams. 1380 * Increased delay and resource consumption from NAT/FW 1381 traversal. 1383 * Likely larger signalling message and signalling processing 1384 requirement due to the amount of session related information. 1386 * Higher potential for a single RTP stream to fail during 1387 transport between the endpoints. 1389 b. When the number of RTP sessions grows, the amount of explicit 1390 state for relating RTP streams also grows, depending on how the 1391 application needs to relate RTP streams. 1393 c. The port consumption might become a problem for centralised 1394 services, where the central node's port or 5-tuple filter 1395 consumption grows rapidly with the number of sessions. 1397 d. For applications where the RTP stream usage is highly dynamic, 1398 i.e. entering and leaving, the amount of signalling can grow 1399 high. Issues can also arise from the timely establishment of 1400 additional RTP sessions. 1402 e. If, against the recommendation, the same SSRC value is reused in 1403 multiple RTP sessions rather than being randomly chosen, 1404 interworking with applications that use a different multiplexing 1405 structure will require SSRC translation. 1407 RTP applications that need to interwork with legacy RTP applications 1408 can potentially benefit from this structure. However, a large number 1409 of media descriptions in SDP can also run into issues with existing 1410 implementations. For any application needing a larger number of 1411 media flows, the overhead can become very significant. This 1412 structure is also not suitable for multi-party sessions, as any given 1413 RTP stream from each participant, although having same usage in the 1414 application, needs its own RTP session. In addition, the dynamic 1415 behaviour that can arise in multi-party applications can tax the 1416 signalling system and make timely media establishment more difficult. 1418 5.5. Summary 1420 There are some clear similarities between these designs. Both the 1421 "Single SSRC per Endpoint" and the "Multiple Media Types in one 1422 Session" are cases that require full explicit signalling of the media 1423 stream relations. However, they operate on two different levels 1424 where the first primarily enables session level binding, and the 1425 second needs SSRC level binding. From another perspective, the two 1426 solutions are the two extreme points when it comes to number of RTP 1427 sessions needed. 1429 The two other designs "Multiple SSRCs of the Same Media Type" and 1430 "Multiple Sessions for one Media Type" are two examples that 1431 primarily allows for some implicit mapping of the role or usage of 1432 the RTP streams based on which RTP session they appear in. It thus 1433 potentially allows for less signalling and in particular reduces the 1434 need for real-time signalling in dynamic sessions. They also 1435 represent points in between the first two designs when it comes to 1436 amount of RTP sessions established, i.e. representing an attempt to 1437 balance the amount of RTP sessions with the functionality the 1438 communication session provides both on network level and on 1439 signalling level. 1441 6. Guidelines 1443 This section contains a number of multi-stream guidelines for 1444 implementers or specification writers. 1446 Do not require use of the same SSRC value across RTP sessions: 1447 As discussed in Section 3.4.3 there exist drawbacks in using the 1448 same SSRC in multiple RTP sessions as a mechanism to bind related 1449 RTP streams together. It is instead recommended to use a 1450 mechanism to explicitly signal the relation, either in RTP/RTCP or 1451 in the signalling mechanism used to establish the RTP session(s). 1453 Use additional RTP streams for additional media sources: In the 1454 cases where an RTP endpoint needs to transmit additional RTP 1455 streams of the same media type in the application, with the same 1456 processing requirements at the network and RTP layers, it is 1457 suggested to send them in the same RTP session. For example a 1458 telepresence room where there are three cameras, and each camera 1459 captures 2 persons sitting at the table, sending each camera as 1460 its own RTP stream within a single RTP session is suggested. 1462 Use additional RTP sessions for streams with different requirements: 1464 When RTP streams have different processing requirements from the 1465 network or the RTP layer at the endpoints, it is suggested that 1466 the different types of streams are put in different RTP sessions. 1467 This includes the case where different participants want different 1468 subsets of the set of RTP streams. 1470 When using multiple RTP Sessions, use grouping: When using Multiple 1471 RTP session solutions, it is suggested to explicitly group the 1472 involved RTP sessions when needed using a signalling mechanism, 1473 for example The Session Description Protocol (SDP) Grouping 1474 Framework [RFC5888], using some appropriate grouping semantics. 1476 RTP/RTCP Extensions Support Multiple RTP Streams as well as Multiple 1477 RTP sessions: 1478 When defining an RTP or RTCP extension, the creator needs to 1479 consider if this extension is applicable to use with additional 1480 SSRCs and multiple RTP sessions. Any extension intended to be 1481 generic must support both. Extensions that are not as generally 1482 applicable will have to consider if interoperability is better 1483 served by defining a single solution or providing both options. 1485 Transport Support Extensions: When defining new RTP/RTCP extensions 1486 intended for transport support, like the retransmission or FEC 1487 mechanisms, they must include support for both multiple RTP 1488 streams in the same RTP sessions and multiple RTP sessions, such 1489 that application developers can choose freely from the set of 1490 mechanisms without concerning themselves with which of the 1491 multiplexing choices a particular solution supports. 1493 7. IANA Considerations 1495 This document makes no request of IANA. 1497 Note to RFC Editor: this section can be removed on publication as an 1498 RFC. 1500 8. Security Considerations 1502 The security considerations of the RTP specification [RFC3550] and 1503 any applicable RTP profile [RFC3551],[RFC4585],[RFC3711], the 1504 extensions for sending multiple media types in a single RTP session 1505 [I-D.ietf-avtcore-multi-media-rtp-session], RID 1506 [I-D.ietf-mmusic-rid], BUNDLE 1507 [I-D.ietf-mmusic-sdp-bundle-negotiation], [RFC5760], [RFC5761], apply 1508 if selected and thus needs to be considered in the evaluation. 1510 There is discussion of the security implications of choosing multiple 1511 SSRC vs multiple RTP sessions in Section 4.3. 1513 9. Contributors 1515 Hui Zheng (Marvin) from Huawei contributed to WG draft versions -04 1516 and -05 of the document. 1518 10. References 1520 10.1. Normative References 1522 [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. 1523 Jacobson, "RTP: A Transport Protocol for Real-Time 1524 Applications", STD 64, RFC 3550, DOI 10.17487/RFC3550, 1525 July 2003, . 1527 [RFC7656] Lennox, J., Gross, K., Nandakumar, S., Salgueiro, G., and 1528 B. Burman, Ed., "A Taxonomy of Semantics and Mechanisms 1529 for Real-Time Transport Protocol (RTP) Sources", RFC 7656, 1530 DOI 10.17487/RFC7656, November 2015, 1531 . 1533 10.2. Informative References 1535 [ALF] Clark, D. and D. Tennenhouse, "Architectural 1536 Considerations for a New Generation of Protocols", SIGCOMM 1537 Symposium on Communications Architectures and 1538 Protocols (Philadelphia, Pennsylvania), pp. 200--208, IEEE 1539 Computer Communications Review, Vol. 20(4), September 1540 1990. 1542 [I-D.ietf-avtcore-multi-media-rtp-session] 1543 Westerlund, M., Perkins, C., and J. Lennox, "Sending 1544 Multiple Types of Media in a Single RTP Session", draft- 1545 ietf-avtcore-multi-media-rtp-session-13 (work in 1546 progress), December 2015. 1548 [I-D.ietf-avtext-rid] 1549 Roach, A., Nandakumar, S., and P. Thatcher, "RTP Stream 1550 Identifier Source Description (SDES)", draft-ietf-avtext- 1551 rid-09 (work in progress), October 2016. 1553 [I-D.ietf-mmusic-rid] 1554 Roach, A., "RTP Payload Format Restrictions", draft-ietf- 1555 mmusic-rid-15 (work in progress), May 2018. 1557 [I-D.ietf-mmusic-sdp-bundle-negotiation] 1558 Holmberg, C., Alvestrand, H., and C. Jennings, 1559 "Negotiating Media Multiplexing Using the Session 1560 Description Protocol (SDP)", draft-ietf-mmusic-sdp-bundle- 1561 negotiation-53 (work in progress), September 2018. 1563 [I-D.ietf-mmusic-sdp-simulcast] 1564 Burman, B., Westerlund, M., Nandakumar, S., and M. Zanaty, 1565 "Using Simulcast in SDP and RTP Sessions", draft-ietf- 1566 mmusic-sdp-simulcast-13 (work in progress), June 2018. 1568 [I-D.ietf-perc-srtp-ekt-diet] 1569 Jennings, C., Mattsson, J., McGrew, D., Wing, D., and F. 1570 Andreasen, "Encrypted Key Transport for DTLS and Secure 1571 RTP", draft-ietf-perc-srtp-ekt-diet-09 (work in progress), 1572 October 2018. 1574 [RFC2198] Perkins, C., Kouvelas, I., Hodson, O., Hardman, V., 1575 Handley, M., Bolot, J., Vega-Garcia, A., and S. Fosse- 1576 Parisis, "RTP Payload for Redundant Audio Data", RFC 2198, 1577 DOI 10.17487/RFC2198, September 1997, 1578 . 1580 [RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S. 1581 Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1 1582 Functional Specification", RFC 2205, DOI 10.17487/RFC2205, 1583 September 1997, . 1585 [RFC2474] Nichols, K., Blake, S., Baker, F., and D. Black, 1586 "Definition of the Differentiated Services Field (DS 1587 Field) in the IPv4 and IPv6 Headers", RFC 2474, 1588 DOI 10.17487/RFC2474, December 1998, 1589 . 1591 [RFC2974] Handley, M., Perkins, C., and E. Whelan, "Session 1592 Announcement Protocol", RFC 2974, DOI 10.17487/RFC2974, 1593 October 2000, . 1595 [RFC3261] Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston, 1596 A., Peterson, J., Sparks, R., Handley, M., and E. 1597 Schooler, "SIP: Session Initiation Protocol", RFC 3261, 1598 DOI 10.17487/RFC3261, June 2002, 1599 . 1601 [RFC3264] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model 1602 with Session Description Protocol (SDP)", RFC 3264, 1603 DOI 10.17487/RFC3264, June 2002, 1604 . 1606 [RFC3389] Zopf, R., "Real-time Transport Protocol (RTP) Payload for 1607 Comfort Noise (CN)", RFC 3389, DOI 10.17487/RFC3389, 1608 September 2002, . 1610 [RFC3551] Schulzrinne, H. and S. Casner, "RTP Profile for Audio and 1611 Video Conferences with Minimal Control", STD 65, RFC 3551, 1612 DOI 10.17487/RFC3551, July 2003, 1613 . 1615 [RFC3711] Baugher, M., McGrew, D., Naslund, M., Carrara, E., and K. 1616 Norrman, "The Secure Real-time Transport Protocol (SRTP)", 1617 RFC 3711, DOI 10.17487/RFC3711, March 2004, 1618 . 1620 [RFC3830] Arkko, J., Carrara, E., Lindholm, F., Naslund, M., and K. 1621 Norrman, "MIKEY: Multimedia Internet KEYing", RFC 3830, 1622 DOI 10.17487/RFC3830, August 2004, 1623 . 1625 [RFC4103] Hellstrom, G. and P. Jones, "RTP Payload for Text 1626 Conversation", RFC 4103, DOI 10.17487/RFC4103, June 2005, 1627 . 1629 [RFC4383] Baugher, M. and E. Carrara, "The Use of Timed Efficient 1630 Stream Loss-Tolerant Authentication (TESLA) in the Secure 1631 Real-time Transport Protocol (SRTP)", RFC 4383, 1632 DOI 10.17487/RFC4383, February 2006, 1633 . 1635 [RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session 1636 Description Protocol", RFC 4566, DOI 10.17487/RFC4566, 1637 July 2006, . 1639 [RFC4568] Andreasen, F., Baugher, M., and D. Wing, "Session 1640 Description Protocol (SDP) Security Descriptions for Media 1641 Streams", RFC 4568, DOI 10.17487/RFC4568, July 2006, 1642 . 1644 [RFC4585] Ott, J., Wenger, S., Sato, N., Burmeister, C., and J. Rey, 1645 "Extended RTP Profile for Real-time Transport Control 1646 Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585, 1647 DOI 10.17487/RFC4585, July 2006, 1648 . 1650 [RFC4588] Rey, J., Leon, D., Miyazaki, A., Varsa, V., and R. 1651 Hakenberg, "RTP Retransmission Payload Format", RFC 4588, 1652 DOI 10.17487/RFC4588, July 2006, 1653 . 1655 [RFC5104] Wenger, S., Chandra, U., Westerlund, M., and B. Burman, 1656 "Codec Control Messages in the RTP Audio-Visual Profile 1657 with Feedback (AVPF)", RFC 5104, DOI 10.17487/RFC5104, 1658 February 2008, . 1660 [RFC5109] Li, A., Ed., "RTP Payload Format for Generic Forward Error 1661 Correction", RFC 5109, DOI 10.17487/RFC5109, December 1662 2007, . 1664 [RFC5576] Lennox, J., Ott, J., and T. Schierl, "Source-Specific 1665 Media Attributes in the Session Description Protocol 1666 (SDP)", RFC 5576, DOI 10.17487/RFC5576, June 2009, 1667 . 1669 [RFC5760] Ott, J., Chesterfield, J., and E. Schooler, "RTP Control 1670 Protocol (RTCP) Extensions for Single-Source Multicast 1671 Sessions with Unicast Feedback", RFC 5760, 1672 DOI 10.17487/RFC5760, February 2010, 1673 . 1675 [RFC5761] Perkins, C. and M. Westerlund, "Multiplexing RTP Data and 1676 Control Packets on a Single Port", RFC 5761, 1677 DOI 10.17487/RFC5761, April 2010, 1678 . 1680 [RFC5764] McGrew, D. and E. Rescorla, "Datagram Transport Layer 1681 Security (DTLS) Extension to Establish Keys for the Secure 1682 Real-time Transport Protocol (SRTP)", RFC 5764, 1683 DOI 10.17487/RFC5764, May 2010, 1684 . 1686 [RFC5888] Camarillo, G. and H. Schulzrinne, "The Session Description 1687 Protocol (SDP) Grouping Framework", RFC 5888, 1688 DOI 10.17487/RFC5888, June 2010, 1689 . 1691 [RFC6465] Ivov, E., Ed., Marocco, E., Ed., and J. Lennox, "A Real- 1692 time Transport Protocol (RTP) Header Extension for Mixer- 1693 to-Client Audio Level Indication", RFC 6465, 1694 DOI 10.17487/RFC6465, December 2011, 1695 . 1697 [RFC7201] Westerlund, M. and C. Perkins, "Options for Securing RTP 1698 Sessions", RFC 7201, DOI 10.17487/RFC7201, April 2014, 1699 . 1701 [RFC7657] Black, D., Ed. and P. Jones, "Differentiated Services 1702 (Diffserv) and Real-Time Communication", RFC 7657, 1703 DOI 10.17487/RFC7657, November 2015, 1704 . 1706 [RFC7667] Westerlund, M. and S. Wenger, "RTP Topologies", RFC 7667, 1707 DOI 10.17487/RFC7667, November 2015, 1708 . 1710 [RFC7826] Schulzrinne, H., Rao, A., Lanphier, R., Westerlund, M., 1711 and M. Stiemerling, Ed., "Real-Time Streaming Protocol 1712 Version 2.0", RFC 7826, DOI 10.17487/RFC7826, December 1713 2016, . 1715 [RFC8088] Westerlund, M., "How to Write an RTP Payload Format", 1716 RFC 8088, DOI 10.17487/RFC8088, May 2017, 1717 . 1719 [RFC8108] Lennox, J., Westerlund, M., Wu, Q., and C. Perkins, 1720 "Sending Multiple RTP Streams in a Single RTP Session", 1721 RFC 8108, DOI 10.17487/RFC8108, March 2017, 1722 . 1724 [RFC8445] Keranen, A., Holmberg, C., and J. Rosenberg, "Interactive 1725 Connectivity Establishment (ICE): A Protocol for Network 1726 Address Translator (NAT) Traversal", RFC 8445, 1727 DOI 10.17487/RFC8445, July 2018, 1728 . 1730 Appendix A. Dismissing Payload Type Multiplexing 1732 This section documents a number of reasons why using the payload type 1733 as a multiplexing point is unsuitable for most things related to 1734 multiple RTP streams. If one attempts to use Payload type 1735 multiplexing beyond its defined usage, that has well known negative 1736 effects on RTP. To use payload type as the single discriminator for 1737 multiple streams implies that all the different RTP streams are being 1738 sent with the same SSRC, thus using the same timestamp and sequence 1739 number space. This has many effects: 1741 1. Putting restraint on RTP timestamp rate for the multiplexed 1742 media. For example, RTP streams that use different RTP 1743 timestamp rates cannot be combined, as the timestamp values need 1744 to be consistent across all multiplexed media frames. Thus 1745 streams are forced to use the same RTP timestamp rate. When 1746 this is not possible, payload type multiplexing cannot be used. 1748 2. Many RTP payload formats can fragment a media object over 1749 multiple RTP packets, like parts of a video frame. These 1750 payload formats need to determine the order of the fragments to 1751 correctly decode them. Thus, it is important to ensure that all 1752 fragments related to a frame or a similar media object are 1753 transmitted in sequence and without interruptions within the 1754 object. This can relatively simple be solved on the sender side 1755 by ensuring that the fragments of each RTP stream are sent in 1756 sequence. 1758 3. Some media formats require uninterrupted sequence number space 1759 between media parts. These are media formats where any missing 1760 RTP sequence number will result in decoding failure or invoking 1761 a repair mechanism within a single media context. The text/ 1762 T140 payload format [RFC4103] is an example of such a format. 1763 These formats will need a sequence numbering abstraction 1764 function between RTP and the individual RTP stream before being 1765 used with payload type multiplexing. 1767 4. Sending multiple streams in the same sequence number space makes 1768 it impossible to determine which payload type, which stream a 1769 packet loss relates to, and thus to which stream to potentially 1770 apply packet loss concealment or other stream-specific loss 1771 mitigation mechanisms. 1773 5. If RTP Retransmission [RFC4588] is used and there is a loss, it 1774 is possible to ask for the missing packet(s) by SSRC and 1775 sequence number, not by payload type. If only some of the 1776 payload type multiplexed streams are of interest, there is no 1777 way of telling which missing packet(s) belong to the interesting 1778 stream(s) and all lost packets need be requested, wasting 1779 bandwidth. 1781 6. The current RTCP feedback mechanisms are built around providing 1782 feedback on RTP streams based on stream ID (SSRC), packet 1783 (sequence numbers) and time interval (RTP Timestamps). There is 1784 almost never a field to indicate which payload type is reported, 1785 so sending feedback for a specific RTP payload type is difficult 1786 without extending existing RTCP reporting. 1788 7. The current RTCP media control messages [RFC5104] specification 1789 is oriented around controlling particular media flows, i.e. 1790 requests are done addressing a particular SSRC. Such mechanisms 1791 would need to be redefined to support payload type multiplexing. 1793 8. The number of payload types are inherently limited. 1794 Accordingly, using payload type multiplexing limits the number 1795 of streams that can be multiplexed and does not scale. This 1796 limitation is exacerbated if one uses solutions like RTP and 1797 RTCP multiplexing [RFC5761] where a number of payload types are 1798 blocked due to the overlap between RTP and RTCP. 1800 9. At times, there is a need to group multiplexed streams and this 1801 is currently possible for RTP sessions and for SSRC, but there 1802 is no defined way to group payload types. 1804 10. It is currently not possible to signal bandwidth requirements 1805 per RTP stream when using payload type multiplexing. 1807 11. Most existing SDP media level attributes cannot be applied on a 1808 per payload type level and would require re-definition in that 1809 context. 1811 12. A legacy endpoint that does not understand the indication that 1812 different RTP payload types are different RTP streams might be 1813 slightly confused by the large amount of possibly overlapping or 1814 identically defined RTP payload types. 1816 Appendix B. Signalling Considerations 1818 Signalling is not an architectural consideration for RTP itself, so 1819 this discussion has been moved to an appendix. However, it is hugely 1820 important for anyone building complete applications, so it is 1821 deserving of discussion. 1823 The issues raised here need to be addressed in the WGs that deal with 1824 signalling; they cannot be addressed by tweaking, extending or 1825 profiling RTP. 1827 There exist various signalling solutions for establishing RTP 1828 sessions. Many are SDP [RFC4566] based, however SDP functionality is 1829 also dependent on the signalling protocols carrying the SDP. RTSP 1830 [RFC7826] and SAP [RFC2974] both use SDP in a declarative fashion, 1831 while SIP [RFC3261] uses SDP with the additional definition of Offer/ 1832 Answer [RFC3264]. The impact on signalling and especially SDP needs 1833 to be considered as it can greatly affect how to deploy a certain 1834 multiplexing point choice. 1836 B.1. Session Oriented Properties 1838 One aspect of the existing signalling is that it is focused around 1839 RTP sessions, or at least in the case of SDP the media description. 1840 There are a number of things that are signalled on media description 1841 level but those are not necessarily strictly bound to an RTP session 1842 and could be of interest to signal specifically for a particular RTP 1843 stream (SSRC) within the session. The following properties have been 1844 identified as being potentially useful to signal not only on RTP 1845 session level: 1847 o Bitrate/Bandwidth exist today only at aggregate or as a common 1848 "any RTP stream" limit, unless either codec-specific bandwidth 1849 limiting or RTCP signalling using TMMBR is used. 1851 o Which SSRC that will use which RTP payload types (this will be 1852 visible from the first media packet, but is sometimes useful to 1853 know before packet arrival). 1855 Some of these issues are clearly SDP's problem rather than RTP 1856 limitations. However, if the aim is to deploy an solution using 1857 additional SSRCs that contains several sets of RTP streams with 1858 different properties (encoding/packetization parameter, bit-rate, 1859 etc.), putting each set in a different RTP session would directly 1860 enable negotiation of the parameters for each set. If insisting on 1861 additional SSRC only, a number of signalling extensions are needed to 1862 clarify that there are multiple sets of RTP streams with different 1863 properties and that they need in fact be kept different, since a 1864 single set will not satisfy the application's requirements. 1866 For some parameters, such as RTP payload type, resolution and 1867 framerate, a SSRC-linked mechanism has been proposed in 1868 [I-D.ietf-mmusic-rid] 1870 B.2. SDP Prevents Multiple Media Types 1872 SDP chose to use the m= line both to delineate an RTP session and to 1873 specify the top level of the MIME media type; audio, video, text, 1874 image, application. This media type is used as the top-level media 1875 type for identifying the actual payload format and is bound to a 1876 particular payload type using the rtpmap attribute. This binding has 1877 to be loosened in order to use SDP to describe RTP sessions 1878 containing multiple MIME top level types. 1880 [I-D.ietf-mmusic-sdp-bundle-negotiation] describes how to let 1881 multiple SDP media descriptions use a single underlying transport in 1882 SDP, which allows to define one RTP session with media types having 1883 different MIME top level types. 1885 B.3. Signalling RTP stream Usage 1887 RTP streams being transported in RTP has some particular usage in an 1888 RTP application. This usage of the RTP stream is in many 1889 applications so far implicitly signalled. For example, an 1890 application might choose to take all incoming audio RTP streams, mix 1891 them and play them out. However, in more advanced applications that 1892 use multiple RTP streams there will be more than a single usage or 1893 purpose among the set of RTP streams being sent or received. RTP 1894 applications will need to signal this usage somehow. The signalling 1895 used will have to identify the RTP streams affected by their RTP- 1896 level identifiers, which means that they have to be identified either 1897 by their session or by their SSRC + session. 1899 In some applications, the receiver cannot utilise the RTP stream at 1900 all before it has received the signalling message describing the RTP 1901 stream and its usage. In other applications, there exists a default 1902 handling that is appropriate. 1904 If all RTP streams in an RTP session are to be treated in the same 1905 way, identifying the session is enough. If SSRCs in a session are to 1906 be treated differently, signalling needs to identify both the session 1907 and the SSRC. 1909 If this signalling affects how any RTP central node, like an RTP 1910 mixer or translator that selects, mixes or processes streams, treats 1911 the streams, the node will also need to receive the same signalling 1912 to know how to treat RTP streams with different usage in the right 1913 fashion. 1915 Authors' Addresses 1917 Magnus Westerlund 1918 Ericsson 1919 Torshamsgatan 23 1920 SE-164 80 Kista 1921 Sweden 1923 Phone: +46 10 714 82 87 1924 Email: magnus.westerlund@ericsson.com 1925 Bo Burman 1926 Ericsson 1927 Gronlandsgatan 31 1928 SE-164 60 Kista 1929 Sweden 1931 Phone: +46 10 714 13 11 1932 Email: bo.burman@ericsson.com 1934 Colin Perkins 1935 University of Glasgow 1936 School of Computing Science 1937 Glasgow G12 8QQ 1938 United Kingdom 1940 Email: csp@csperkins.org 1942 Harald Tveit Alvestrand 1943 Google 1944 Kungsbron 2 1945 Stockholm 11122 1946 Sweden 1948 Email: harald@alvestrand.no 1950 Roni Even 1951 Huawei 1953 Email: roni.even@huawei.com