idnits 2.17.1 draft-ietf-avtcore-rtp-topologies-update-06.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (March 2, 2015) is 3335 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-12) exists of draft-ietf-avtcore-rtp-multi-stream-optimisation-05 == Outdated reference: A later version (-54) exists of draft-ietf-mmusic-sdp-bundle-negotiation-16 -- Obsolete informational reference (is this intentional?): RFC 5117 (Obsoleted by RFC 7667) -- Obsolete informational reference (is this intentional?): RFC 5285 (Obsoleted by RFC 8285) -- Obsolete informational reference (is this intentional?): RFC 5766 (Obsoleted by RFC 8656) Summary: 0 errors (**), 0 flaws (~~), 3 warnings (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group M. Westerlund 3 Internet-Draft Ericsson 4 Obsoletes: 5117 (if approved) S. Wenger 5 Intended status: Informational Vidyo 6 Expires: September 3, 2015 March 2, 2015 8 RTP Topologies 9 draft-ietf-avtcore-rtp-topologies-update-06 11 Abstract 13 This document discusses point to point and multi-endpoint topologies 14 used in Real-time Transport Protocol (RTP)-based environments. In 15 particular, centralized topologies commonly employed in the video 16 conferencing industry are mapped to the RTP terminology. 18 This document is updated with additional topologies and replaces RFC 19 5117. 21 Status of This Memo 23 This Internet-Draft is submitted in full conformance with the 24 provisions of BCP 78 and BCP 79. 26 Internet-Drafts are working documents of the Internet Engineering 27 Task Force (IETF). Note that other groups may also distribute 28 working documents as Internet-Drafts. The list of current Internet- 29 Drafts is at http://datatracker.ietf.org/drafts/current/. 31 Internet-Drafts are draft documents valid for a maximum of six months 32 and may be updated, replaced, or obsoleted by other documents at any 33 time. It is inappropriate to use Internet-Drafts as reference 34 material or to cite them other than as "work in progress." 36 This Internet-Draft will expire on September 3, 2015. 38 Copyright Notice 40 Copyright (c) 2015 IETF Trust and the persons identified as the 41 document authors. All rights reserved. 43 This document is subject to BCP 78 and the IETF Trust's Legal 44 Provisions Relating to IETF Documents 45 (http://trustee.ietf.org/license-info) in effect on the date of 46 publication of this document. Please review these documents 47 carefully, as they describe your rights and restrictions with respect 48 to this document. Code Components extracted from this document must 49 include Simplified BSD License text as described in Section 4.e of 50 the Trust Legal Provisions and are provided without warranty as 51 described in the Simplified BSD License. 53 Table of Contents 55 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 56 2. Definitions . . . . . . . . . . . . . . . . . . . . . . . . . 3 57 2.1. Glossary . . . . . . . . . . . . . . . . . . . . . . . . 3 58 2.2. Definitions related to RTP grouping taxonomy . . . . . . 4 59 3. Topologies . . . . . . . . . . . . . . . . . . . . . . . . . 5 60 3.1. Point to Point . . . . . . . . . . . . . . . . . . . . . 5 61 3.2. Point to Point via Middlebox . . . . . . . . . . . . . . 6 62 3.2.1. Translators . . . . . . . . . . . . . . . . . . . . . 6 63 3.2.2. Back to Back RTP sessions . . . . . . . . . . . . . . 10 64 3.3. Point to Multipoint Using Multicast . . . . . . . . . . . 11 65 3.3.1. Any Source Multicast (ASM) . . . . . . . . . . . . . 11 66 3.3.2. Source Specific Multicast (SSM) . . . . . . . . . . . 12 67 3.3.3. SSM with Local Unicast Resources . . . . . . . . . . 14 68 3.4. Point to Multipoint Using Mesh . . . . . . . . . . . . . 16 69 3.5. Point to Multipoint Using the RFC 3550 Translator . . . . 19 70 3.5.1. Relay - Transport Translator . . . . . . . . . . . . 19 71 3.5.2. Media Translator . . . . . . . . . . . . . . . . . . 20 72 3.6. Point to Multipoint Using the RFC 3550 Mixer Model . . . 21 73 3.6.1. Media Mixing Mixer . . . . . . . . . . . . . . . . . 23 74 3.6.2. Media Switching . . . . . . . . . . . . . . . . . . . 26 75 3.7. Selective Forwarding Middlebox . . . . . . . . . . . . . 28 76 3.8. Point to Multipoint Using Video Switching MCUs . . . . . 31 77 3.9. Point to Multipoint Using RTCP-Terminating MCU . . . . . 33 78 3.10. Split Component Terminal . . . . . . . . . . . . . . . . 34 79 3.11. Non-Symmetric Mixer/Translators . . . . . . . . . . . . . 37 80 3.12. Combining Topologies . . . . . . . . . . . . . . . . . . 37 81 4. Topology Properties . . . . . . . . . . . . . . . . . . . . . 38 82 4.1. All to All Media Transmission . . . . . . . . . . . . . . 38 83 4.2. Transport or Media Interoperability . . . . . . . . . . . 39 84 4.3. Per Domain Bit-Rate Adaptation . . . . . . . . . . . . . 39 85 4.4. Aggregation of Media . . . . . . . . . . . . . . . . . . 40 86 4.5. View of All Session Participants . . . . . . . . . . . . 40 87 4.6. Loop Detection . . . . . . . . . . . . . . . . . . . . . 41 88 4.7. Consistency between header extensions and RTCP . . . . . 41 89 5. Comparison of Topologies . . . . . . . . . . . . . . . . . . 41 90 6. Security Considerations . . . . . . . . . . . . . . . . . . . 42 91 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 44 92 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 44 93 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 44 94 9.1. Normative References . . . . . . . . . . . . . . . . . . 44 95 9.2. Informative References . . . . . . . . . . . . . . . . . 45 96 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 46 98 1. Introduction 100 Real-time Transport Protocol (RTP) [RFC3550] topologies describe 101 methods for interconnecting RTP entities and their processing 102 behavior of RTP and RTCP. This document tries to address past and 103 existing confusion, especially with respect to terms not defined in 104 RTP but in common use in the conversational communication industry, 105 such as the Multipoint Control Unit or MCU. 107 When the Audio-Visual Profile with Feedback (AVPF) [RFC4585] was 108 developed the main emphasis lay in the efficient support of point to 109 point and small multipoint scenarios without centralized multipoint 110 control. In practice, however, most multipoint conferences operate 111 utilizing centralized units referred to as MCUs. MCUs may implement 112 Mixer or Translator functionality (in RTP [RFC3550] terminology), and 113 signalling support. They may also contain additional application 114 layer functionality. This document focuses on the media transport 115 aspects of the MCU that can be realized using RTP, as discussed 116 below. Further considered are the properties of Mixers and 117 Translators, and how some types of deployed MCUs deviate from these 118 properties. 120 This document also codifies new multipoint architectures that have 121 recently been introduced and which were not anticipated in RFC 5117, 122 thus this document replaces [RFC5117]. These architectures use 123 scalable video coding and simulcasting, and their associated 124 centralized units are referred to as Selective Forwarding Units 125 (SFU). This codification provides a common information basis for 126 future discussion and specification work. 128 The document's attempt to clarify and explain sections of the Real- 129 time Transport Protocol (RTP) spec [RFC3550] is informal. It is not 130 intended to update or change what is normatively specified within RFC 131 3550. 133 2. Definitions 135 2.1. Glossary 137 ASM: Any Source Multicast 139 AVPF: The Extended RTP Profile for RTCP-based Feedback 141 CSRC: Contributing Source 143 Link: The data transport to the next IP hop 144 Middlebox: A device that is on the Path that media travel between 145 two endpoints 147 MCU: Multipoint Control Unit 149 Path: The concatenation of multiple links, resulting in an end-to- 150 end data transfer. 152 PtM: Point to Multipoint 154 PtP: Point to Point 156 SFU: Selective Forwarding Unit 158 SSM: Source-Specific Multicast 160 SSRC: Synchronization Source 162 2.2. Definitions related to RTP grouping taxonomy 164 [Note to RFC editor: The following definitions have been taken from 165 draft-ietf-avtext-rtp-grouping-taxonomy-02 (taxonomy draft 166 henceforth). It is avtcore working group agreement to not delay the 167 publication of the topologies-update document through a dependency to 168 the taxonomy draft. If, however, the taxonomy draft and this draft 169 are in your work queue at the same time and there would be no 170 significant additional delay (through your schedule, normative 171 reference citations, or similar) in publishing both documents roughly 172 in parallel, it would be preferable to replace the definition 173 language with something like "as in [RFC YYYY]" where YYYY would be 174 the RFC number of the published taxonomy draft.] 176 The following definitions have been taken from draft-ietf-avtext-rtp- 177 grouping-taxonomy-02, and are used in capitalized form throughout the 178 document. 180 Communication Session: A Communication Session is an association 181 among group of participants communicating with each other via a 182 set of Multimedia Sessions. 184 Endpoint: A single addressable entity sending or receiving RTP 185 packets. It may be decomposed into several functional blocks, but 186 as long as it behaves as a single RTP stack entity it is 187 classified as a single "Endpoint". 189 Media Source: A Media Source is the logical source of a reference 190 clock synchronized, time progressing, digital media stream, called 191 a Source Stream. 193 Multimedia Session: A multimedia session is an association among a 194 group of participants engaged in the communication via one or more 195 RTP Sessions. 197 3. Topologies 199 This subsection defines several topologies that are relevant for 200 codec control but also RTP usage in other contexts. The section 201 starts with point to point cases, with or without middleboxes. Then 202 follows a number of different methods for establishing point to 203 multipoint communication. These are structured around the most 204 fundamental enabler, i.e., multicast, a mesh of connections, 205 translators, mixers and finally MCUs and SFUs. The section ends by 206 discussing de-composited terminals, asymmetric middlebox behaviors 207 and combining topologies. 209 The topologies may be referenced in other documents by a shortcut 210 name, indicated by the prefix "Topo-". 212 For each of the RTP-defined topologies, we discuss how RTP, RTCP, and 213 the carried media are handled. With respect to RTCP, we also discuss 214 the handling of RTCP feedback messages as defined in [RFC4585] and 215 [RFC5104]. 217 3.1. Point to Point 219 Shortcut name: Topo-Point-to-Point 221 The Point to Point (PtP) topology (Figure 1) consists of two 222 endpoints, communicating using unicast. Both RTP and RTCP traffic 223 are conveyed endpoint-to-endpoint, using unicast traffic only (even 224 if, in exotic cases, this unicast traffic happens to be conveyed over 225 an IP-multicast address). 227 +---+ +---+ 228 | A |<------->| B | 229 +---+ +---+ 231 Figure 1: Point to Point 233 The main property of this topology is that A sends to B, and only B, 234 while B sends to A, and only A. This avoids all complexities of 235 handling multiple endpoints and combining the requirements stemming 236 from them. Note that an endpoint can still use multiple RTP 237 Synchronization Sources (SSRCs) in an RTP session. The number of RTP 238 sessions in use between A and B can also be of any number, subject 239 only to system level limitations like the number range of ports. 241 RTCP feedback messages for the indicated SSRCs are communicated 242 directly between the endpoints. Therefore, this topology poses 243 minimal (if any) issues for any feedback messages. For RTP sessions 244 which use multiple SSRC per endpoint it can be relevant to implement 245 support for cross-reporting suppression as defined in "Sending 246 Multiple Media Streams in a Single RTP Session" 247 [I-D.ietf-avtcore-rtp-multi-stream-optimisation]. 249 3.2. Point to Point via Middlebox 251 This section discusses cases where two endpoints communicate but have 252 one or more middleboxes involved in the RTP session. 254 3.2.1. Translators 256 Shortcut name: Topo-PtP-Translator 258 Two main categories of Translators can be distinguished; Transport 259 Translators and Media translators. Both Translator types share 260 common attributes that separate them from Mixers. For each RTP 261 stream that the Translator receives, it generates an individual RTP 262 stream in the other domain. A translator keeps the SSRC for an RTP 263 stream across the translation, whereas a Mixer can select a single 264 RTP stream from multiple received RTP streams (in cases like audio/ 265 video switching), or send out an RTP stream composed of multiple 266 mixed media received in multiple RTP streams (in cases like audio 267 mixing or video tiling), but always under its own SSRC, possibly 268 using the CSRC field to indicate the source(s) of the content. 269 Mixers are more common in point to multipoint cases than in PtP. The 270 reason is that in PtP use cases the primary focus of a middlebox is 271 enabling interoperability, between otherwise non-interoperable 272 endpoints, such as transcoding to a codec the receiver supports, 273 which can be done by a media translator. 275 As specified in Section 7.1 of [RFC3550], the SSRC space is common 276 for all participants in the RTP session, independent of on which side 277 of the Translator the session resides. Therefore, it is the 278 responsibility of the endpoints (as the RTP session participants) to 279 run SSRC collision detection, and the SSRC is thus a field the 280 Translator cannot change. Any SDES information associated with a 281 SSRC or CSRC also needs to be forwarded between the domains for any 282 SSRC/CSRC used in the different domains. 284 A Translator commonly does not use an SSRC of its own, and is not 285 visible as an active participant in the RTP session. One reason to 286 have its own SSRC is when a Translator acts as a quality monitor that 287 sends RTCP reports and therefore is required to have an SSRC. 288 Another example is the case when a Translator is prepared to use RTCP 289 feedback messages. This may, for example, occur in a translator 290 configured to detect packet loss of important video packets and wants 291 to trigger repair by the media sending endpoint, by sending feedback 292 messages. While such feedback could use the SSRC of the target for 293 the translator (the receiving endpoint), this in turn would require 294 translation of the targets RTCP reports to make them consistent. It 295 may be simpler to expose an additional SSRC in the session. The only 296 concern is endpoints failing to support the full RTP specification 297 may have issues with multiple SSRCs reporting on the RTP streams sent 298 by that endpoint, as this use case may be viewed as excotic by 299 implementers. 301 In general, a Translator implementation should consider which RTCP 302 feedback messages or codec-control messages it needs to understand in 303 relation to the functionality of the Translator itself. This is 304 completely in line with the requirement to also translate RTCP 305 messages between the domains. 307 3.2.1.1. Transport Relay/Anchoring 309 There exist a number of different types of middleboxes that might be 310 inserted between two endpoints on the transport level, e.g., to 311 perform changes on the IP/UDP headers, and are, therefore, basic 312 transport translators. These middleboxes come in many variations 313 including NAT [RFC3022] traversal by pinning the media path to a 314 public address domain relay, network topologies where the RTP stream 315 is required to pass a particular point for audit by employing 316 relaying, or preserving privacy by hiding each peer's transport 317 addresses to the other party. Other protocols or functionalities 318 that provide this behavior are TURN [RFC5766] servers, Session Border 319 Gateways and Media Processing Nodes with media anchoring 320 functionalities. 322 +---+ +---+ +---+ 323 | A |<------>| T |<------->| B | 324 +---+ +---+ +---+ 326 Figure 2: Point to Point with Translator 328 A common element in these functions is that they are normally 329 transparent at the RTP level, i.e., they perform no changes on any 330 RTP or RTCP packet fields and only affect the lower layers. They may 331 affect, however, the path the RTP and RTCP packets are routed between 332 the endpoints in the RTP session, and thereby indirectly affect the 333 RTP session. For this reason, one could believe that transport 334 translator-type middleboxes do not need to be included in this 335 document. This topology, however, can raise additional requirements 336 in the RTP implementation and its interactions with the signalling 337 solution. Both in signalling and in certain RTCP fields, network 338 addresses other than those of the relay can occur since B has a 339 different network address than the relay (T). Implementations that 340 cannot support this will also not work correctly when endpoints are 341 subject to NAT. 343 The transport relay implementations also have to take into account 344 security considerations. In particular, source address filtering of 345 incoming packets is usually important in relays, to prevent attackers 346 to inject traffic into a session, which one peer may, in the absence 347 fo adequate security in the relay, think it comes from the other 348 peer. 350 3.2.1.2. Transport Translator 352 Transport Translators (Topo-Trn-Translator) do not modify the RTP 353 stream itself, but are concerned with transport parameters. 354 Transport parameters, in the sense of this section, comprise the 355 transport addresses (to bridge different domains such unicast to 356 multicast) and the media packetization to allow other transport 357 protocols to be interconnected to a session (in gateways). 359 Translators that bridge between different protocol worlds need to be 360 concerned about the mapping of the SSRC/CSRC (Contributing Source) 361 concept to the non-RTP protocol. When designing a Translator to a 362 non-RTP-based media transport, an important consideration is how to 363 handle different sources and their identities. This problem space is 364 not discussed henceforth. 366 Of the transport Translators, this memo is primarily interested in 367 those that use RTP on both sides, and this is assumed henceforth. 369 The most basic transport translators that operate below the RTP level 370 were already discussed in Section 3.2.1.1. 372 3.2.1.3. Media Translator 374 Media Translators (Topo-Media-Translator) modify the media inside the 375 RTP stream. This process is commonly known as transcoding. The 376 modification of the media can be as small as removing parts of the 377 stream, and it can go all the way to a full decoding and re-encoding 378 (down to the sample level or equivalent) utilizing a different media 379 codec. Media Translators are commonly used to connect endpoints 380 without a common interoperability point in the media encoding. 382 Stand-alone Media Translators are rare. Most commonly, a combination 383 of Transport and Media Translator is used to translate both the media 384 and the transport aspects of the RTP stream carrying the media 385 between two transport domains. 387 When media translation occurs, the Translator's task regarding 388 handling of RTCP traffic becomes substantially more complex. In this 389 case, the Translator needs to rewrite endpoint B's RTCP Receiver 390 Report before forwarding them to endpoint A. The rewriting is needed 391 as the RTP stream received by B is not the same RTP stream as the 392 other participants receive. For example, the number of packets 393 transmitted to B may be lower than what A sends, due to the different 394 media format and data rate. Therefore, if the Receiver Reports were 395 forwarded without changes, the extended highest sequence number would 396 indicate that B were substantially behind in reception, while most 397 likely it would not be. Therefore, the Translator must translate 398 that number to a corresponding sequence number for the stream the 399 Translator received. Similar requirements exists for most other 400 fields in the RTCP Receiver Reports. 402 A media Translator may in some cases act on behalf of the "real" 403 source (the endpoint originally sending the media to the Translator) 404 and respond to RTCP feedback messages. This may occur, for example, 405 when a receiving endpoint requests a bandwidth reduction, and the 406 media Translator has not detected any congestion or other reasons for 407 bandwidth reduction between the sending endpoint and itself. In that 408 case, it is sensible that the media Translator reacts to codec 409 control messages itself, for example, by transcoding to a lower media 410 rate. 412 A variant of translator behaviour worth pointing out is the one 413 depicted in Figure 3 of an endpoint A sending a RTP stream containing 414 media (only) to B. On the path there is a device T that on A's 415 behalf manipulates the RTP streams. One common example is that T 416 adds a second RTP stream containing Forward Error Correction (FEC) 417 information in order to protect A's (non FEC-protected) RTP stream. 418 In this case, T needs to semantically bind the new FEC RTP stream to 419 A's media-carrying RTP stream, for example by using the same CNAME as 420 A. 422 +------+ +------+ +------+ 423 | | | | | | 424 | A |------->| T |-------->| B | 425 | | | |---FEC-->| | 426 +------+ +------+ +------+ 428 Figure 3: Media Translator adding FEC 430 there may also be cases where information is added into the original 431 RTP stream, while leaving most or all of the original RTP packets 432 intact (with the exception of certain RTP header fields, such as the 433 sequence number). One example is the injection of meta-data into the 434 RTP stream, carried in their own RTP packets. 436 Similarly, a Media Translator can sometimes remove information from 437 the RTP stream, while otherwise leaving the remaining RTP packets 438 unchanged (again with the exception of certain RTP header fields). 440 Either type of functionality where T manipulates the RTP stream, or 441 adds an accompanying RTP stream, on behalf of A is also covered under 442 the media translator definition. 444 3.2.2. Back to Back RTP sessions 446 There exist middleboxes that interconnect two endpoints A and B 447 through themselves (MB), but not by being part of a common RTP 448 session. They establish instead two different RTP sessions, one 449 between A and the middlebox and another between the middlebox and B. 450 This topology is called Topo-Back-To-Back 452 |<--Session A-->| |<--Session B-->| 453 +------+ +------+ +------+ 454 | A |------->| MB |-------->| B | 455 +------+ +------+ +------+ 457 Figure 4: Back-to-back RTP sessions through Middlebox 459 The middlebox acts as an application-level gateway and bridges the 460 two RTP sessions. This bridging can be as basic as forwarding the 461 RTP payloads between the sessions, or more complex including media 462 transcoding. The difference of this topology relative to the single 463 RTP session context is the handling of the SSRCs and the other 464 session-related identifiers, such as CNAMEs. With two different RTP 465 sessions these can be freely changed and it becomes the middlebox's 466 respnsibility to maintain the correct relations. 468 The signalling or other above-RTP level functionalities referencing 469 RTP streams may be what is most impacted by using two RTP sessions 470 and changing identifiers. The structure with two RTP sessions also 471 puts a congestion control requirement on the middlebox, because it 472 becomes fully responsible for the media stream it sources into each 473 of the sessions. 475 Adherence to congestion control can be solved locally on each of the 476 two segments, or by bridging statistics from the receiving endpoint 477 through the middlebox to the sending endpoint. From an 478 implementation point, however, the latter requires dealing with a 479 number of inconsistencies. First, packet loss must be detected for 480 an RTP stream sent from A to the middlebox, and that loss must be 481 reported through a skipped sequence number in the RTP stream from the 482 middlebox to B. This coupling and the resulting inconsistencies are 483 conceptually easier to handle when considering the two RTP streams as 484 belonging to a single RTP session. 486 3.3. Point to Multipoint Using Multicast 488 Multicast is an IP layer functionality that is available in some 489 networks. Two main flavors can be distinguished: Any Source 490 Multicast (ASM) [RFC1112] where any multicast group participant can 491 send to the group address and expect the packet to reach all group 492 participants; and Source Specific Multicast (SSM) [RFC3569], where 493 only a particular IP host sends to the multicast group. Both these 494 models are discussed below in their respective sections. 496 3.3.1. Any Source Multicast (ASM) 498 Shortcut name: Topo-ASM (was Topo-Multicast) 500 +-----+ 501 +---+ / \ +---+ 502 | A |----/ \---| B | 503 +---+ / Multi- \ +---+ 504 + Cast + 505 +---+ \ Network / +---+ 506 | C |----\ /---| D | 507 +---+ \ / +---+ 508 +-----+ 510 Figure 5: Point to Multipoint Using Multicast 512 Point to Multipoint (PtM) is defined here as using a multicast 513 topology as a transmission model, in which traffic from any multicast 514 group participant reaches all the other multicast group participants, 515 except for cases such as: 517 o packet loss, or 519 o when a multicast group participant does not wish to receive the 520 traffic for a specific multicast group and, therefore, has not 521 subscribed to the IP multicast group in question. This scenario 522 can occur, for example, where a multimedia session is distributed 523 using two or more multicast groups and a multicast group 524 participant is subscribed only to a subset of these sessions. 526 In the above context, "traffic" encompasses both RTP and RTCP 527 traffic. The number of multicast group participants can vary between 528 one and many, as RTP and RTCP scale to very large multicast groups 529 (the theoretical limit of the number of participants in a single RTP 530 session is in the range of billions). The above can be realized 531 using Any Source Multicast (ASM). 533 For feedback usage, it is useful to define a "small multicast group" 534 as a group where the number of multicast group participants is so low 535 (and other factors such as the connectivity is so good) that it 536 allows the participants to use early or immediate feedback, as 537 defined in AVPF [RFC4585]. Even when the environment would allow for 538 the use of a small multicast group, some applications may still want 539 to use the more limited options for RTCP feedback available to large 540 multicast groups, for example when there is a likelihood that the 541 threshold of the small multicast group (in terms of multicast group 542 participants) may be exceeded during the lifetime of a session. 544 RTCP feedback messages in multicast reach, like media data, every 545 subscriber (subject to packet losses and multicast group 546 subscription). Therefore, the feedback suppression mechanism 547 discussed in [RFC4585] is typically required. Each individual 548 endpoint that is a multicast group participant needs to process every 549 feedback message it receives, not only to determine if it is affected 550 or if the feedback message applies only to some other endpoint, but 551 also to derive timing restrictions for the sending of its own 552 feedback messages, if any. 554 3.3.2. Source Specific Multicast (SSM) 556 In Any Source Multicast, any of the multicast group participants can 557 send to all the other multicast group participants, by sending a 558 packet to the multicast group. In contrast, Source Specific 559 Multicast [RFC3569][RFC4607] refers to scenarios where only a single 560 source (Distribution Source) can send to the multicast group, 561 creating a topology that looks like the one below: 563 +--------+ +-----+ 564 |Media | | | Source-specific 565 |Sender 1|<----->| D S | Multicast 566 +--------+ | I O | +--+----------------> R(1) 567 | S U | | | | 568 +--------+ | T R | | +-----------> R(2) | 569 |Media |<----->| R C |->+ | : | | 570 |Sender 2| | I E | | +------> R(n-1) | | 571 +--------+ | B | | | | | | 572 : | U | +--+--> R(n) | | | 573 : | T +-| | | | | 574 : | I | |<---------+ | | | 575 +--------+ | O |F|<---------------+ | | 576 |Media | | N |T|<--------------------+ | 577 |Sender M|<----->| | |<-------------------------+ 578 +--------+ +-----+ RTCP Unicast 580 FT = Feedback Target 581 Transport from the Feedback Target to the Distribution 582 Source is via unicast or multicast RTCP if they are not 583 co-located. 585 Figure 6: Point to Multipoint using Source Specific Multicast 587 In the SSM topology (Figure 6) a number of RTP sending endpoints (RTP 588 sources henceforth) (1 to M) are allowed to send media to the SSM 589 group. These sources send media to a dedicated distribution source, 590 which forwards the RTP streams to the multicast group on behalf of 591 the original RTP sources. The RTP streams reach the receiving 592 endpoints (Receivers henceforth) (R(1) to R(n)). The Receivers' RTCP 593 messages cannot be sent to the multicast group, as the SSM multicast 594 group by definition has only a single IP sender. To support RTCP, an 595 RTP extension for SSM [RFC5760] was defined. It uses unicast 596 transmission to send RTCP from each of the receivers to one or more 597 Feedback Targets (FT). The feedback targets relay the RTCP 598 unmodified, or provide a summary of the participants RTCP reports 599 towards the whole group by forwarding the RTCP traffic to the 600 distribution source. Figure 6 only shows a single feedback target 601 integrated in the distribution source, but for scalability the FT can 602 be distributed and each instance can have responsibility for sub- 603 groups of the receivers. For summary reports, however, there 604 typically must be a single feedback target aggregating all the 605 summaries to a common message to the whole receiver group. 607 The RTP extension for SSM specifies how feedback (both reception 608 information and specific feedback events) are handled. The more 609 general problems associated with the use of multicast, where everyone 610 receives what the distribution source sends needs to be accounted 611 for. 613 Aforementioned situation results in common behavior for RTP 614 multicast: 616 1. Multicast applications often use a group of RTP sessions, not 617 one. Each endpoint needs to be a member of most or all of these 618 RTP sessions in order to perform well. 620 2. Within each RTP session, the number of media sinks is likely to 621 be much larger than the number of RTP sources. 623 3. Multicast applications need signalling functions to identify the 624 relationships between RTP sessions. 626 4. Multicast applications need signalling functions to identify the 627 relationships between SSRCs in different RTP sessions. 629 All multicast configurations share a signalling requirement: all of 630 the endpoints need to have the same RTP and payload type 631 configuration. Otherwise, endpoint A could, for example, be using 632 payload type 97 to identify the video codec H.264, while endpoint B 633 would identify it as MPEG-2, with unpredicatble but almost certainly 634 not visually pleasing results. 636 Security solutions for this type of group communications are also 637 challenging. First, the key-management and the security protocol 638 must support group communication. Source authentication becomes more 639 difficult and requires specialized solutions. For more discussion on 640 this please review Options for Securing RTP Sessions [RFC7201]. 642 3.3.3. SSM with Local Unicast Resources 644 "Unicast-Based Rapid Acquisition of Multicast RTP Sessions" [RFC6285] 645 results in additional extensions to SSM Topology. 647 ----------- -------------- 648 | |------------------------------------>| | 649 | |.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.->| | 650 | | | | 651 | Multicast | ---------------- | | 652 | Source | | Retransmission | | | 653 | |-------->| Server (RS) | | | 654 | |.-.-.-.->| | | | 655 | | | ------------ | | | 656 ----------- | | Feedback | |<.=.=.=.=.| | 657 | | Target (FT)| |<~~~~~~~~~| RTP Receiver | 658 PRIMARY MULTICAST | ------------ | | (RTP_Rx) | 659 RTP SESSION with | | | | 660 UNICAST FEEDBACK | | | | 661 | | | | 662 - - - - - - - - - - - |- - - - - - - - |- - - - - |- - - - - - - |- - 663 | | | | 664 UNICAST BURST | ------------ | | | 665 (or RETRANSMISSION) | | Burst/ | |<~~~~~~~~>| | 666 RTP SESSION | | Retrans. | |.........>| | 667 | |Source (BRS)| |<.=.=.=.=>| | 668 | ------------ | | | 669 | | | | 670 ---------------- -------------- 672 -------> Multicast RTP Stream 673 .-.-.-.> Multicast RTCP Stream 674 .=.=.=.> Unicast RTCP Reports 675 ~~~~~~~> Unicast RTCP Feedback Messages 676 .......> Unicast RTP Stream 678 Figure 7: SSM with Local Unicast Resources (RAMS) 680 The Rapid acquisition extension allows an endpoint joining an SSM 681 multicast session to request media starting with the last sync-point 682 (from where media can be decoded without requiring context 683 established by the decoding of prior packets) to be sent at high 684 speed until such time where, after decoding of these burst-delivered 685 media packets, the correct media timing is established, i.e. media 686 packets are received within adequate buffer intervals for this 687 application. This is accomplished by first establishing a unicast 688 PtP RTP session between the Burst/Retransmission Source (BRS, 689 Figure 7) and the RTP Receiver. The unicast session is used to 690 transmit cached packets from the multicast group at higher then 691 normal speed in order to synchronize the receiver to the ongoing 692 multicast RTP stream. Once the RTP receiver and its decoder have 693 caught up with the multicast session's current delivery, the receiver 694 switches over to receiving directly from the multicast group. In 695 many deployed application, the (still existing) PtP RTP session is 696 used as a repair channel, i.e., for RTP Retransmission traffic of 697 those packets that were not received from the multicast group. 699 3.4. Point to Multipoint Using Mesh 701 Shortcut name: Topo-Mesh 703 +---+ +---+ 704 | A |<---->| B | 705 +---+ +---+ 706 ^ ^ 707 \ / 708 \ / 709 v v 710 +---+ 711 | C | 712 +---+ 714 Figure 8: Point to Multi-Point using Mesh 716 Based on the RTP session definition, it is clearly possible to have a 717 joint RTP session involving three or more endpoints over multiple 718 unicast transport flows, like the joint three endpoint session 719 depicted above. In this case, A needs to send its RTP streams and 720 RTCP packets to both B and C over their respective transport flows. 721 As long as all endpoints do the same, everyone will have a joint view 722 of the RTP session. 724 This topology does not create any additional requirements beyond the 725 need to have multiple transport flows associated with a single RTP 726 session. Note that an endpoint may use a single local port to 727 receive all these transport flows (in which case the sending port, IP 728 address, or SSRC can be used to demultiplex), or it might have 729 separate local reception ports for each of the endpoints. 731 +-A--------------------+ 732 |+---+ | 733 ||CAM| | +-B-----------+ 734 |+---+ +-UDP1------| |-UDP1------+ | 735 | | | +-RTP1----| |-RTP1----+ | | 736 | V | | +-Video-| |-Video-+ | | | 737 |+----+ | | | |<----------------|BV1 | | | | 738 ||ENC |----+-+-+--->AV1|---------------->| | | | | 739 |+----+ | | +-------| |-------+ | | | 740 | | | +---------| |---------+ | | 741 | | +-----------| |-----------+ | 742 | | | +-------------+ 743 | | | 744 | | | +-C-----------+ 745 | | +-UDP2------| |-UDP2------+ | 746 | | | +-RTP1----| |-RTP1----+ | | 747 | | | | +-Video-| |-Video-+ | | | 748 | +-------+-+-+--->AV1|---------------->| | | | | 749 | | | | |<----------------|CV1 | | | | 750 | | | +-------| |-------+ | | | 751 | | +---------| |---------+ | | 752 | +-----------| |-----------+ | 753 +----------------------+ +-------------+ 755 Figure 9: An Multi-unicast Mesh with a joint RTP session 757 A joint RTP session from endpoint A's perspective for the Mesh 758 depicted in Figure 8 with a joint RTP session have multiple transport 759 flows, here enumerated as UDP1 and UDP2. However, there is only one 760 RTP session (RTP1). The Media Source (CAM) is encoded and 761 transmitted over the SSRC (AV1) across both transport layers. 762 However, as this is a joint RTP session, the two streams must be the 763 same. Thus, an congestion control adaptation needed for the paths A 764 to B and A to C needs to use the most restricting path's properties. 766 An alternative structure for establishing the above topology is to 767 use independent RTP sessions between each pair of peers, i.e., three 768 different RTP sessions. In some scenarios, the same RTP stream may 769 be sent from the transmitting endpoint, however it also supports 770 local adaptation taking place in one or more of the RTP streams, 771 rendering them non-identical. 773 +-A----------------------+ +-B-----------+ 774 |+---+ | | | 775 ||MIC| +-UDP1------| |-UDP1------+ | 776 |+---+ | +-RTP1----| |-RTP1----+ | | 777 | | +----+ | | +-Audio-| |-Audio-+ | | | 778 | +->|ENC1|--+-+-+--->AA1|------------->| | | | | 779 | | +----+ | | | |<-------------|BA1 | | | | 780 | | | | +-------| |-------+ | | | 781 | | | +---------| |---------+ | | 782 | | +-----------| |-----------+ | 783 | | ------------| |-------------| 784 | | | |-------------+ 785 | | | 786 | | | +-C-----------+ 787 | | | | | 788 | | +-UDP2------| |-UDP2------+ | 789 | | | +-RTP2----| |-RTP2----+ | | 790 | | +----+ | | +-Audio-| |-Audio-+ | | | 791 | +->|ENC2|--+-+-+--->AA2|------------->| | | | | 792 | +----+ | | | |<-------------|CA1 | | | | 793 | | | +-------| |-------+ | | | 794 | | +---------| |---------+ | | 795 | +-----------| |-----------+ | 796 +------------------------+ +-------------+ 798 Figure 10: An Multi-unicast Mesh with independent RTP session 800 Lets review the topology when independent RTP sessions are used, from 801 A's perspective in Figure 10 by considering both how the media is 802 handled and the RTP sessions that are set-up in Figure 10. A's 803 microphone is captured and the audio is fed into two different 804 encoder instances, each being associated with two independent RTP 805 sessions (RTP1 and RTP2). The SSRCs (AA1 and AA2) in each RTP 806 session are completely independent and the media bit-rate produced by 807 the encoders can also be tuned differently to address any congestion 808 control requirements differing for the paths A to B compared to A to 809 C. 811 From a topologies viewpoint, an important difference exists in the 812 behavior around RTCP. First, when a single RTP session spans all 813 three endpoints A, B, and C, and their connecting RTP streams, a 814 common RTCP bandwidth is calculated and used for this single joint 815 session. In contrast, when there are multiple independent RTP 816 sessions, each RTP session has its local RTCP bandwidth allocation. 818 Further, when multiple sessions are used, endpoints not directly 819 involved in a session do not have any awareness of the conditions in 820 those sessions. For example, in the case of the three endpoint 821 configuration in Figure 8, endpoint A has no awareness of the 822 conditions occurring in the session between endpoints B and C 823 (whereas, if a single RTP session were used, it would have such 824 awareness). 826 Loop detection is also affected. With independent RTP sessions, the 827 SSRC/CSRC cannot be used to determine when an endpoint receives its 828 own media stream, or a mixed media stream including its own media 829 stream (a condition known as a loop). The identification of loops 830 and, in most cases, their avoidance, has to be achieved by other 831 means, for example through signaling or the use of an RTP external 832 name space binding SSRC/CSRC among any communicating RTP sessions in 833 the mesh. 835 3.5. Point to Multipoint Using the RFC 3550 Translator 837 This section discusses some additional usages related to point to 838 multipoint of Translators compared to the point to point only cases 839 in Section 3.2.1. 841 3.5.1. Relay - Transport Translator 843 Shortcut name: Topo-PtM-Trn-Translator 845 This section discusses Transport Translator only usages to enable 846 multipoint sessions. 848 +-----+ 849 +---+ / \ +------------+ +---+ 850 | A |<---/ \ | |<---->| B | 851 +---+ / Multi- \ | | +---+ 852 + cast +->| Translator | 853 +---+ \ Network / | | +---+ 854 | C |<---\ / | |<---->| D | 855 +---+ \ / +------------+ +---+ 856 +-----+ 858 Figure 11: Point to Multipoint Using Multicast 860 Figure 11 depicts an example of a Transport Translator performing at 861 least IP address translation. It allows the (non-multicast-capable) 862 endpoints B and D to take part in an any source multicast session 863 involving endpoints A and C, by having the Translator forward their 864 unicast traffic to the multicast addresses in use, and vice versa. 865 It must also forward B's traffic to D, and vice versa, to provide 866 each of B and D with a complete view of the session. 868 +---+ +------------+ +---+ 869 | A |<---->| |<---->| B | 870 +---+ | | +---+ 871 | Translator | 872 +---+ | | +---+ 873 | C |<---->| |<---->| D | 874 +---+ +------------+ +---+ 876 Figure 12: RTP Translator (Relay) with Only Unicast Paths 878 Another Translator scenario is depicted in Figure 12. The Translator 879 in this case connects multiple endpoints through unicast. This can 880 be implemented using a very simple transport Translator which, in 881 this document, is called a relay. The relay forwards all traffic it 882 receives, both RTP and RTCP, to all other endpoints. In doing so, a 883 multicast network is emulated without relying on a multicast-capable 884 network infrastructure. 886 For RTCP feedback this results in a similar set of considerations to 887 those described in the ASM RTP topology. It also puts some 888 additional signalling requirements onto the session establishment; 889 for example, a common configuration of RTP payload types is required. 891 Transport translators and relays should always consider implementing 892 source address filtering, to prevent attackers to inject traffic 893 using the listening ports on the translator. The translator can, 894 however, go one step further, and especially if explicit SSRC 895 signalling is used, prevent endpoints to send SSRCs other than its 896 own (that are, for example, used by other participants in the 897 session). This can improve the security properties of the session, 898 despite the use of group keys that on cryptographic level allows 899 anyone to impersonate another in the same RTP session. 901 A Translator that doesn't change the RTP/RTCP packets content can be 902 operated without the requiring it to have access to the security 903 contexts used to protect the RTP/RTCP traffic between the 904 participants. 906 3.5.2. Media Translator 908 In the context of multipoint communications a Media Translator is not 909 providing new mechanisms to establish a multipoint session. It is 910 more of an enabler, or facilitator, that ensures a given endpoint or 911 a defined sub-set of endpoints can participate in the session. 913 If endpoint B in Figure 11 were behind a limited network path, the 914 Translator may perform media transcoding to allow the traffic 915 received from the other endpoints to reach B without overloading the 916 path. This transcoding can help the other endpoints in the multicast 917 part of the session, by not requiring the quality transmitted by A to 918 be lowered to the bitrates that B is actually capable of receiving 919 (and vice versa). 921 3.6. Point to Multipoint Using the RFC 3550 Mixer Model 923 Shortcut name: Topo-Mixer 925 A Mixer is a middlebox that aggregates multiple RTP streams that are 926 part of a session by generating one or more new RTP streams and, in 927 most cases, by manipulating the media data. One common application 928 for a Mixer is to allow a participant to receive a session with a 929 reduced amount of resources. 931 +-----+ 932 +---+ / \ +-----------+ +---+ 933 | A |<---/ \ | |<---->| B | 934 +---+ / Multi- \ | | +---+ 935 + cast +->| Mixer | 936 +---+ \ Network / | | +---+ 937 | C |<---\ / | |<---->| D | 938 +---+ \ / +-----------+ +---+ 939 +-----+ 941 Figure 13: Point to Multipoint Using the RFC 3550 Mixer Model 943 A Mixer can be viewed as a device terminating the RTP streams 944 received from other endpoints in the same RTP session. Using the 945 media data carried in the received RTP streams, a Mixer generates 946 derived RTP streams that are sent to the receiving endpoints. 948 The content that the Mixer provides is the mixed aggregate of what 949 the Mixer receives over the PtP or PtM paths, which are part of the 950 same Communication Session. 952 The Mixer creates the Media Source and the source RTP stream just 953 like an endpoint, as it mixes the content (often in the uncompressed 954 domain) and then encodes and packetizes it for transmission to a 955 receiving endpoint. The CSRC Count (CC) and CSRC fields in the RTP 956 header can be used to indicate the contributors to the newly 957 generated RTP stream. The SSRCs of the to-be-mixed streams on the 958 Mixer input appear as the CSRCs at the Mixer output. That output 959 stream uses a unique SSRC that identifies the Mixer's stream. The 960 CSRC should be forwarded between the different endpoints to allow for 961 loop detection and identification of sources that are part of the 962 Communication Session. Note that Section 7.1 of RFC 3550 requires 963 the SSRC space to be shared between domains for these reasons. This 964 also implies that any SDES information normally needs to be forwarded 965 across the mixer. 967 The Mixer is responsible for generating RTCP packets in accordance 968 with its role. It is an RTP receiver and should therefore send RTCP 969 receiver reports for the RTP streams it receives and terminates. In 970 its role as an RTP sender, it should also generate RTCP sender 971 reports for those RTP streams it sends. As specified in Section 7.3 972 of RFC 3550, a Mixer must not forward RTCP unaltered between the two 973 domains. 975 The Mixer depicted in Figure 13 is involved in three domains that 976 need to be separated: the any source multicast network (including 977 endpoints A and C), endpoint B, and endpoint D. Assuming all four 978 endpoints in the conference are interested in receiving content from 979 all other endpoints, the Mixer produces different mixed RTP streams 980 for B and D, as the one to B may contain content received from D, and 981 vice versa. However, the Mixer may only need one SSRC per media type 982 in each domain where it is the receiving entity and transmitter of 983 mixed content. 985 In the multicast domain, a Mixer still needs to provide a mixed view 986 of the other domains. This makes the Mixer simpler to implement and 987 avoids any issues with advanced RTCP handling or loop detection, 988 which would be problematic if the Mixer were providing non-symmetric 989 behavior. Please see Section 3.11 for more discussion on this topic. 990 The mixing operation, however, in each domain could potentially be 991 different. 993 A Mixer is responsible for receiving RTCP feedback messages and 994 handling them appropriately. The definition of "appropriate" depends 995 on the message itself and the context. In some cases, the reception 996 of a codec-control message by the Mixer may result in the generation 997 and transmission of RTCP feedback messages by the Mixer to the 998 endpoints in the other domain(s). In other cases, a message is 999 handled by the Mixer locally and therefore not forwarded to any other 1000 domain. 1002 When replacing the multicast network in Figure 13 (to the left of the 1003 Mixer) with individual unicast paths as depicted in Figure 14, the 1004 Mixer model is very similar to the one discussed in Section 3.9 1005 below. Please see the discussion in Section 3.9 about the 1006 differences between these two models. 1008 +---+ +------------+ +---+ 1009 | A |<---->| |<---->| B | 1010 +---+ | | +---+ 1011 | Mixer | 1012 +---+ | | +---+ 1013 | C |<---->| |<---->| D | 1014 +---+ +------------+ +---+ 1016 Figure 14: RTP Mixer with Only Unicast Paths 1018 We now discuss in more detail the different mixing operations that a 1019 mixer can perform and how they can affect RTP and RTCP behavior. 1021 3.6.1. Media Mixing Mixer 1023 The media mixing mixer is likely the one that most think of when they 1024 hear the term "mixer". Its basic mode of operation is that it 1025 receives RTP streams from several endpoints and selects the stream(s) 1026 to be included in a media-domain mix. The selection can be through 1027 static configuration or by dynamic, content dependent means such as 1028 voice activation. The mixer then creates a single outgoing RTP 1029 stream from this mix. 1031 The most commonly deployed media mixer is probably the audio mixer, 1032 used in voice conferencing, where the output consists of a mixture of 1033 all the input audio signals; this needs minimal signalling to be 1034 successfully set up. From a signal processing viewpoint, audio 1035 mixing is relatively straightforward and commonly possible for a 1036 reasonable number of endpoints. Assume, for example, that one wants 1037 to mix N streams from N different endpoints. The mixer needs to 1038 decode those N streams, typically into the sample domain, and then 1039 produce N or N+1 mixes. Different mixes are needed so that each 1040 contributing source gets a mix of all other sources except its own, 1041 as this would result in an echo. When N is lower than the number of 1042 all endpoints, one may produce a mix of all N streams for the group 1043 that are currently not included in the mix, thus N+1 mixes. These 1044 audio streams are then encoded again, RTP packetized and sent out. 1045 In many cases, audio level normalization, noise suppression, and 1046 similar signal processing steps are also required or desirable before 1047 the actual mixing process commences. 1049 In video, the term "mixing" has a different interpretation than 1050 audio. It is commonly used to refer to the process of spatially 1051 combining contributed video streams, which is also known as "tiling". 1052 The reconstructed, appropriately scaled down videos can be spatially 1053 arranged in a set of tiles, each tile containing the video from an 1054 endpoint (typically showing a human participant). Tiles can be of 1055 different sizes, so that, for example, a particularly important 1056 participant, or the loudest speaker, is being shown on in larger tile 1057 than other participants. A self-view picture can be included in the 1058 tiling, which can either be locally produced or be a feedback from a 1059 mixer-received and reconstructed video image. Such remote loopback 1060 allows for confidence monitoring, i.e., it enables the participant to 1061 see himself/herself in the same quality as other participants see 1062 him/her. The tiling normally operates on reconstructed video in the 1063 sample domain. The tiled image is encoded, packetized, and sent by 1064 the mixer to the receiving endpoints. It is possible that a 1065 middlebox with media mixing duties contains only a single mixer of 1066 the aforementioned type, in which case all participants necessarily 1067 see the same tiled video, even if it is being sent over different RTP 1068 streams. More common, however, are mixing arrangement where an 1069 individual mixer is available for each outgoing port of the 1070 middlebox, allowing individual compositions for each receiving 1071 endpoint (a feature commonly referred to as personalized layout). 1073 One problem with media mixing is that it consumes both large amounts 1074 of media processing resources (for the decoding and mixing process in 1075 the uncompressed domain) and encoding resources (for the encoding of 1076 the mixed signal). Another problem is the quality degradation 1077 created by decoding and re-encoding the media, which is the result of 1078 the lossy nature of most commonly used media codecs. A third problem 1079 is the latency introduced by the media mixing, which can be 1080 substantial and annoyingly noticeable in case of video, or in case of 1081 audio if that mixed audio is lip-sychronized with high latency video. 1082 The advantage of media mixing is that it is straightforward for the 1083 endpoints to handle the single media stream (which includes the mixed 1084 aggregate of many sources), as they don't need to handle multiple 1085 decodings, local mixing and composition. In fact, mixers were 1086 introduced in pre-RTP times so that legacy, single stream receiving 1087 endpoints (that, in some protocol environments, actually didn't need 1088 to be aware of the multipoint nature of the conference) could 1089 successfully participate in what a user would recognize as a 1090 multiparty video conference. 1092 +-A---------+ +-MIXER----------------------+ 1093 | +-RTP1----| |-RTP1------+ +-----+ | 1094 | | +-Audio-| |-Audio---+ | +---+ | | | 1095 | | | AA1|--------->|---------+-+-|DEC|->| | | 1096 | | | |<---------|MA1 <----+ | +---+ | | | 1097 | | | | |(BA1+CA1)|\| +---+ | | | 1098 | | +-------| |---------+ +-|ENC|<-| B+C | | 1099 | +---------| |-----------+ +---+ | | | 1100 +-----------+ | | | | 1101 | | M | | 1102 +-B---------+ | | E | | 1103 | +-RTP2----| |-RTP2------+ | D | | 1104 | | +-Audio-| |-Audio---+ | +---+ | I | | 1105 | | | BA1|--------->|---------+-+-|DEC|->| A | | 1106 | | | |<---------|MA2 <----+ | +---+ | | | 1107 | | +-------| |(AA1+CA1)|\| +---+ | | | 1108 | +---------| |---------+ +-|ENC|<-| A+C | | 1109 +-----------+ |-----------+ +---+ | | | 1110 | | M | | 1111 +-C---------+ | | I | | 1112 | +-RTP3----| |-RTP3------+ | X | | 1113 | | +-Audio-| |-Audio---+ | +---+ | E | | 1114 | | | CA1|--------->|---------+-+-|DEC|->| R | | 1115 | | | |<---------|MA3 <----+ | +---+ | | | 1116 | | +-------| |(AA1+BA1)|\| +---+ | | | 1117 | +---------| |---------+ +-|ENC|<-| A+B | | 1118 +-----------+ |-----------+ +---+ +-----+ | 1119 +----------------------------+ 1121 Figure 15: Session and SSRC details for Media Mixer 1123 From an RTP perspective media mixing can be a very simple process, as 1124 can be seen in Figure 15. The mixer presents one SSRC towards the 1125 receiving endpoint, e.g., MA1 to Peer A, where the associated stream 1126 is the media mix of the other endpoints. As each peer, in this 1127 example, receives a different version of a mix from the mixer, there 1128 is no actual relation between the different RTP sessions in terms of 1129 actual media or transport level information. There are, however, 1130 common relationships between RTP1-RTP3, namely SSRC space and 1131 identity information. When A receives the MA1 stream which is a 1132 combination of BA1 and CA1 streams, the mixer may include CSRC 1133 information in the MA1 stream to identify the contributing source BA1 1134 and CA1, allowing the receiver to identify the contributing sources 1135 even if this were not possible through the media itself or through 1136 other signaling means. 1138 The CSRC has, in turn, utility in RTP extensions, like the Mixer to 1139 Client audio levels RTP header extension [RFC6465]. If the SSRCs 1140 from the endpoint to mixer paths are used as CSRCs in another RTP 1141 session, then RTP1, RTP2 and RTP3 become one joint session as they 1142 have a common SSRC space. At this stage, the mixer also needs to 1143 consider which RTCP information it needs to expose in the different 1144 paths. In the above scenario, a mixer would normally expose nothing 1145 more than the Source Description (SDES) information and RTCP BYE for 1146 a CSRC leaving the session. The main goal would be to enable the 1147 correct binding against the application logic and other information 1148 sources. This also enables loop detection in the RTP session. 1150 3.6.2. Media Switching 1152 Media switching mixers are used in limited functionality scenarios 1153 where no, or only very limited, concurrent presentation of multiple 1154 sources is required by the application, to more complex multi-stream 1155 usages with receiver mixing or tiling, including combined with 1156 simulcast and/or scalability between source and mixer. An RTP Mixer 1157 based on media switching avoids the media decoding and encoding 1158 operations in the mixer, as it conceptually forwards the encoded 1159 media stream as it was being sent to the mixer. It does not avoid, 1160 however, the decryption and re-encryption cycle as it rewrites RTP 1161 headers. Forwarding media (in contrast to reconstructing-mixing- 1162 encoding media) reduces the amount of computational resources needed 1163 in the mixer and increases the media quality (both in terms of 1164 fidelity and reduced latency). 1166 A media switching mixer maintains a pool of SSRCs representing 1167 conceptual or functional RTP streams that the mixer can produce. 1168 These RTP streams are created by selecting media from one of the RTP 1169 streams received by the mixer and forwarded to the peer using the 1170 mixer's own SSRCs. The mixer can switch between available sources if 1171 that is required by the concept for the source, like the currently 1172 active speaker. Note that the mixer, in most cases, still needs to 1173 perform a certain amount of media processing, as many media formats 1174 do not allow to "tune into" the stream at arbitrary points in their 1175 bitstream. 1177 To achieve a coherent RTP stream from the mixer's SSRC, the mixer 1178 needs to rewrite the incoming RTP packet's header. First the SSRC 1179 field must be set to the value of the Mixer's SSRC. Second, the 1180 sequence number must be the next in the sequence of outgoing packets 1181 it sent. Third, the RTP timestamp value needs to be adjusted using 1182 an offset that changes each time one switches media source. Finally, 1183 depending on the negotiation of the RTP payload type, the value 1184 representing this particular RTP payload configuration may have to be 1185 changed if the different endpoint-to-mixer paths have not arrived on 1186 the same numbering for a given configuration. This also requires 1187 that the different endpoints support a common set of codecs, 1188 otherwise media transcoding for codec compatibility would still be 1189 required. 1191 We now consider the operation of a media switching mixer that 1192 supports a video conference with six participating endpoints (A-F) 1193 where the two most recent speakers in the conference are shown to 1194 each receiving endpoint. The mixer has thus two SSRCs sending video 1195 to each peer, and each peer is capable of locally handling two video 1196 streams simultaneously. 1198 +-A---------+ +-MIXER----------------------+ 1199 | +-RTP1----| |-RTP1------+ +-----+ | 1200 | | +-Video-| |-Video---+ | | | | 1201 | | | AV1|------------>|---------+-+------->| S | | 1202 | | | |<------------|MV1 <----+-+-BV1----| W | | 1203 | | | |<------------|MV2 <----+-+-EV1----| I | | 1204 | | +-------| |---------+ | | T | | 1205 | +---------| |-----------+ | C | | 1206 +-----------+ | | H | | 1207 | | | | 1208 +-B---------+ | | M | | 1209 | +-RTP2----| |-RTP2------+ | A | | 1210 | | +-Video-| |-Video---+ | | T | | 1211 | | | BV1|------------>|---------+-+------->| R | | 1212 | | | |<------------|MV3 <----+-+-AV1----| I | | 1213 | | | |<------------|MV4 <----+-+-EV1----| X | | 1214 | | +-------| |---------+ | | | | 1215 | +---------| |-----------+ | | | 1216 +-----------+ | | | | 1217 : : : : 1218 : : : : 1219 +-F---------+ | | | | 1220 | +-RTP6----| |-RTP6------+ | | | 1221 | | +-Video-| |-Video---+ | | | | 1222 | | | FV1|------------>|---------+-+------->| | | 1223 | | | |<------------|MV11 <---+-+-AV1----| | | 1224 | | | |<------------|MV12 <---+-+-EV1----| | | 1225 | | +-------| |---------+ | | | | 1226 | +---------| |-----------+ +-----+ | 1227 +-----------+ +----------------------------+ 1229 Figure 16: Media Switching RTP Mixer 1231 The Media Switching RTP mixer can, similarly to the Media Mixing 1232 Mixer, reduce the bit-rate required for media transmission towards 1233 the different peers by selecting and forwarding only a sub-set of RTP 1234 streams it receives from the sending endpoints. In cases the mixer 1235 receives simulcast transmissions or a scalable encoding of the media 1236 source, the mixer has more degrees of freedom to select streams or 1237 sub-sets of stream to forward to a receiving endpoint, both based on 1238 transport or endpoint restrictions as well as application logic. 1240 To ensure that a media receiver in an endpoint can correctly decode 1241 the media in the RTP stream after a switch, a codec that uses 1242 temporal prediction needs to start its decoding from independent 1243 refresh points, or points in the bitstream offering similar 1244 functionality (like "dirty refresh points"). For some codecs, for 1245 example frame based speech and audio codecs, this is easily achieved 1246 by starting the decoding at RTP packet boundaries, as each packet 1247 boundary provides a refresh point (assuming proper packetization on 1248 the encoder side). For other codecs, particularly in video, refresh 1249 points are less common in the bitstream or may not be present at all 1250 without an explicit request to the respective encoder. The Full 1251 Intra Request [RFC5104] RTCP codec control message has been defined 1252 for this purpose. 1254 In this type of mixer one could consider to fully terminate the RTP 1255 sessions between the different endpoint and mixer paths. The same 1256 arguments and considerations as discussed in Section 3.9 need to be 1257 taken into consideration and apply here. 1259 3.7. Selective Forwarding Middlebox 1261 Another method for handling media in the RTP mixer is to "project", 1262 or make available, all potential RTP sources (SSRCs) into a per- 1263 endpoint, independent RTP session. The middlebox can select which of 1264 the potential sources that are currently actively transmitting media 1265 will be sent to each of the endpoints. This is similar to the media 1266 switching Mixer but has some important differences in RTP details. 1268 +-A---------+ +-Middlebox-----------------+ 1269 | +-RTP1----| |-RTP1------+ +-----+ | 1270 | | +-Video-| |-Video---+ | | | | 1271 | | | AV1|------------>|---------+-+------>| | | 1272 | | | |<------------|BV1 <----+-+-------| S | | 1273 | | | |<------------|CV1 <----+-+-------| W | | 1274 | | | |<------------|DV1 <----+-+-------| I | | 1275 | | | |<------------|EV1 <----+-+-------| T | | 1276 | | | |<------------|FV1 <----+-+-------| C | | 1277 | | +-------| |---------+ | | H | | 1278 | +---------| |-----------+ | | | 1279 +-----------+ | | M | | 1280 | | A | | 1281 +-B---------+ | | T | | 1282 | +-RTP2----| |-RTP2------+ | R | | 1283 | | +-Video-| |-Video---+ | | I | | 1284 | | | BV1|------------>|---------+-+------>| X | | 1285 | | | |<------------|AV1 <----+-+-------| | | 1286 | | | |<------------|CV1 <----+-+-------| | | 1287 | | | | : : : |: : : : : : : : :| | | 1288 | | | |<------------|FV1 <----+-+-------| | | 1289 | | +-------| |---------+ | | | | 1290 | +---------| |-----------+ | | | 1291 +-----------+ | | | | 1292 : : : : 1293 : : : : 1294 +-F---------+ | | | | 1295 | +-RTP6----| |-RTP6------+ | | | 1296 | | +-Video-| |-Video---+ | | | | 1297 | | | FV1|------------>|---------+-+------>| | | 1298 | | | |<------------|AV1 <----+-+-------| | | 1299 | | | | : : : |: : : : : : : : :| | | 1300 | | | |<------------|EV1 <----+-+-------| | | 1301 | | +-------| |---------+ | | | | 1302 | +---------| |-----------+ +-----+ | 1303 +-----------+ +---------------------------+ 1305 Figure 17: Selective Forwarding Middlebox 1307 In the six endpoint conference depicted above in (Figure 17) one can 1308 see that endpoint A is aware of five incoming SSRCs, BV1-FV1. If 1309 this middlebox intends to have a similar behavior as in Section 3.6.2 1310 where the mixer provides the endpoints with the two latest speaking 1311 endpoints, then only two out of these five SSRCs need concurrently 1312 transmit media to A. As the middlebox selects the source in the 1313 different RTP sessions that transmit media to the endpoints, each RTP 1314 stream requires rewriting of certain RTP header fields when being 1315 projected from one session into another. In particular, the sequence 1316 number needs to be consecutively incremented based on the packet 1317 actually being transmitted in each RTP session. Therefore, the RTP 1318 sequence number offset will change each time a source is turned on in 1319 a RTP session. The timestamp (possibly offset) stays the same. 1321 As the RTP sessions are independent, the SSRC numbers used can also 1322 be handled independently, thereby bypassing the requirement for SSRC 1323 collision detection and avoidance. On the other hand, tools such as 1324 remapping tables between the RTP sessions are required. For example, 1325 the RTP stream that is being sent by endpoint B to the middlebox 1326 (BV1) may use an SSRC value of 12345678. When that RTP stream is 1327 sent to endpoint F by the middlebox, it can use any SSRC value, e.g. 1328 87654321. As a result, each endpoint may have a different view of 1329 the application usage of a particular SSRC. Any RTP level identity 1330 information, such as SDES items also needs to update the SSRC 1331 referenced, if the included SDES items are intended to be global. 1332 Thus the application must not use SSRC as references to RTP streams 1333 when communicating with other peers directly. This also affects loop 1334 detection which will fail to work, as there is no common namespace 1335 and identities across the different legs in the communication session 1336 on RTP level. Instead this responsibility falls onto higher layers. 1338 The middlebox is also responsible to receive any RTCP codec control 1339 requests coming from an endpoint, and decide if it can act on the 1340 request locally or needs to translate the request into the RTP 1341 session that contains the media source. Both endpoints and the 1342 middlebox need to implement conference related codec control 1343 functionalities to provide a good experience. Commonly used are Full 1344 Intra Request to request from the media source to provide switching 1345 points between the sources, and Temporary Maximum Media Bit-rate 1346 Request (TMMBR) to enable the middlebox to aggregate congestion 1347 control responses towards the media source so to enable it to adjust 1348 its bit-rate (obviously only in case the limitation is not in the 1349 source to middlebox link). 1351 The selective forwarding middlebox has been introduced in recently 1352 developed videoconferencing systems in conjunction with, and to 1353 capitalize on, scalable video coding as well as simulcasting. An 1354 example of scalable video coding is Annex G of H.264, but other 1355 codecs, including H.264 AVC and VP8 also exhibit scalability, albeit 1356 only in the temporal dimension. In both scalable coding and 1357 simulcast cases the video signal is represented by a set of two or 1358 more bitstreams, providing a corresponding number of distinct 1359 fidelity points. The middlebox selects which parts of a scalable 1360 bitstream (or which bitstream, in the case of simulcasting) to 1361 forward to each of the receiving endpoints. The decision may be 1362 driven by a number of factors, such as available bit rate, desired 1363 layout, etc. Contrary to transcoding MCUs, these "Selective 1364 Forwarding Units" (SFUs) have extremely low delay, and provide 1365 features that are typically associated with high-end systems 1366 (personalized layout, error localization) without any signal 1367 processing at the middlebox. They are also capable of scaling to a 1368 large number of concurrent users, and--due to their very low delay-- 1369 can also be cascaded. 1371 This version of the middlebox also puts different requirements on the 1372 endpoint when it comes to decoder instances and handling of the RTP 1373 streams providing media. As each projected SSRC can, at any time, 1374 provide media, the endpoint either needs to be able to handle as many 1375 decoder instances as the middlebox received, or have efficient 1376 switching of decoder contexts in a more limited set of actual decoder 1377 instances to cope with the switches. The application also gets more 1378 responsibility to update how the media provided is to be presented to 1379 the user. 1381 Note that this topology could potentially be seen as a media 1382 translator which include an on/off logic as part of its media 1383 translation. The main difference would be a common global SSRC space 1384 in the case of the Media Translator and the mapped one used in the 1385 above. It also has mixer aspects, as the streams it provides are not 1386 basically translated version, but instead they have conceptual 1387 property assigned to them. Thus this topology appears to be some 1388 hybrid between the translator and mixer model. 1390 The differences between selective forwarding middlebox and a 1391 switching mixer (Section 3.6.2) are minor, and they share most 1392 properties. The above requirement on having a large number of 1393 decoding instances or requiring efficient switching of decoder 1394 contexts, are one point of difference. The other is how the 1395 identification is performed, where the Mixer uses CSRC to provide 1396 information on what is included in a particular RTP stream that 1397 represent a particular concept. Selective forwarding gets the source 1398 information through the SSRC, and instead have to use other mechanism 1399 to make clear the streams current purpose. 1401 3.8. Point to Multipoint Using Video Switching MCUs 1403 Shortcut name: Topo-Video-switch-MCU 1404 +---+ +------------+ +---+ 1405 | A |------| Multipoint |------| B | 1406 +---+ | Control | +---+ 1407 | Unit | 1408 +---+ | (MCU) | +---+ 1409 | C |------| |------| D | 1410 +---+ +------------+ +---+ 1412 Figure 18: Point to Multipoint Using a Video Switching MCU 1414 This PtM topology was popular in early implementations of multipoint 1415 videoconferencing systems due to its simplicity, and the 1416 corresponding middlebox design has been known as a "video switching 1417 MCU". The more complex RTCP-terminating MCUs, discussed in the next 1418 section, became the norm, however, when technology allowed 1419 implementations at acceptable costs. 1421 A video switching MCU forwards to a participant a single media 1422 stream, selected from the available streams. The criteria for 1423 selection are often based on voice activity in the audio-visual 1424 conference, but other conference management mechanisms (like 1425 presentation mode or explicit floor control) are known to exist as 1426 well. 1428 The video switching MCU may also perform media translation to modify 1429 the content in bit-rate, encoding, or resolution. However, it still 1430 may indicate the original sender of the content through the SSRC. In 1431 this case, the values of the CC and CSRC fields are retained. 1433 If not terminating RTP, the RTCP Sender Reports are forwarded for the 1434 currently selected sender. All RTCP Receiver Reports are freely 1435 forwarded between the endpoints. In addition, the MCU may also 1436 originate RTCP control traffic in order to control the session and/or 1437 report on status from its viewpoint. 1439 The video switching MCU has most of the attributes of a Translator. 1440 However, its stream selection is a mixing behavior. This behavior 1441 has some RTP and RTCP issues associated with it. The suppression of 1442 all but one RTP stream results in most participants seeing only a 1443 subset of the sent RTP streams at any given time, often a single RTP 1444 stream per conference. Therefore, RTCP Receiver Reports only report 1445 on these RTP streams. Consequently, the endpoints emitting RTP 1446 streams that are not currently forwarded receive a view of the 1447 session that indicates their RTP streams disappear somewhere en 1448 route. This makes the use of RTCP for congestion control, or any 1449 type of quality reporting, very problematic. 1451 To avoid the aforementioned issues, the MCU needs to implement two 1452 features. First, it needs to act as a Mixer (see Section 3.6) and 1453 forward the selected RTP stream under its own SSRC and with the 1454 appropriate CSRC values. Second, the MCU needs to modify the RTCP 1455 RRs it forwards between the domains. As a result, it is recommended 1456 that one implement a centralized video switching conference using a 1457 Mixer according to RFC 3550, instead of the shortcut implementation 1458 described here. 1460 3.9. Point to Multipoint Using RTCP-Terminating MCU 1462 Shortcut name: Topo-RTCP-terminating-MCU 1464 +---+ +------------+ +---+ 1465 | A |<---->| Multipoint |<---->| B | 1466 +---+ | Control | +---+ 1467 | Unit | 1468 +---+ | (MCU) | +---+ 1469 | C |<---->| |<---->| D | 1470 +---+ +------------+ +---+ 1472 Figure 19: Point to Multipoint Using Content Modifying MCUs 1474 In this PtM scenario, each endpoint runs an RTP point-to-point 1475 session between itself and the MCU. This is a very commonly deployed 1476 topology in multipoint video conferencing. The content that the MCU 1477 provides to each participant is either: 1479 a. a selection of the content received from the other endpoints, or 1481 b. the mixed aggregate of what the MCU receives from the other PtP 1482 paths, which are part of the same Communication Session. 1484 In case (a), the MCU may modify the content in terms of bit-rate, 1485 encoding format, or resolution. No explicit RTP mechanism is used to 1486 establish the relationship between the original RTP stream of the 1487 media being sent RTP stream the MCU sends. In other words, the 1488 outgoing RTP streams typically use a different SSRC, and may well use 1489 a different payload type (PT), even if this different PT happens to 1490 be mapped to the same media type. This is a result of the 1491 individually negotiated RTP session for each endpoint. 1493 In case (b), the MCU is the Media Source and generates the Source RTP 1494 Stream as it mixes the received content and then encodes and 1495 packetizes it for transmission to an endpoint. According to RTP 1496 [RFC3550], the SSRC of the contributors are to be signalled using the 1497 CSRC/CC mechanism. In practice, today, most deployed MCUs do not 1498 implement this feature. Instead, the identification of the endpoints 1499 whose content is included in the Mixer's output is not indicated 1500 through any explicit RTP mechanism. That is, most deployed MCUs set 1501 the CSRC Count (CC) field in the RTP header to zero, thereby 1502 indicating no available CSRC information, even if they could identify 1503 the original sending endpoints as suggested in RTP. 1505 The main feature that sets this topology apart from what RFC 3550 1506 describes is the breaking of the common RTP session across the 1507 centralized device, such as the MCU. This results in the loss of 1508 explicit RTP-level indication of all participants. If one were using 1509 the mechanisms available in RTP and RTCP to signal this explicitly, 1510 the topology would follow the approach of an RTP Mixer. The lack of 1511 explicit indication has at least the following potential problems: 1513 1. Loop detection cannot be performed on the RTP level. When 1514 carelessly connecting two misconfigured MCUs, a loop could be 1515 generated. 1517 2. There is no information about active media senders available in 1518 the RTP packet. As this information is missing, receivers cannot 1519 use it. It also deprives the client of information related to 1520 currently active senders in a machine-usable way, thus preventing 1521 clients from indicating currently active speakers in user 1522 interfaces, etc. 1524 Note that many/most deployed MCUs (and video conferencing endpoints) 1525 rely on signalling layer mechanisms for the identification of the 1526 contributing sources, for example, a SIP conferencing package 1527 [RFC4575]. This alleviates, to some extent, the aforementioned 1528 issues resulting from ignoring RTP's CSRC mechanism. 1530 3.10. Split Component Terminal 1532 Shortcut name: Topo-Split-Terminal 1534 In some applications, for example in some telepresence systems, 1535 terminals may be not integrated into a single functional unit, but 1536 composed of more than one subunits. For example, a telepresence room 1537 terminal employing multiple cameras and monitors may consist of 1538 multiple video conferencing subunits, each capable of handling a 1539 single camera and monitor. Another example would be a video 1540 conferencing terminal in which audio is handled by one subunit, and 1541 video by another. Each of these subunits uses its own physical 1542 network interface (for example: Ethernet jack) and network address. 1544 The various (media processing) subunits need (logically and 1545 physically) to be interconnected by control functionality, but their 1546 media plane functionality may be split. This type of terminals is 1547 referred to as split component terminals. Historically, the earliest 1548 split component terminals were perhaps the (independent) audio and 1549 video conference software tools used over the MBONE in the late 1550 1990s. 1552 An example for such a split component terminal is depicted in 1553 Figure 20. Within split component terminal A, at least audio and 1554 video subunits are addressed by their own network addresses. In some 1555 of these systems, the control stack subunit may also have its own 1556 network address. 1558 From an RTP viewpoint, each of the subunits terminates RTP, and acts 1559 as an endpoint in the sense that each subunit includes its own, 1560 independent RTP stack. However, as the subunits are semantically 1561 part of the same terminal, it is appropriate that this semantic 1562 relationship is expressed in RTCP protocol elements, namely in the 1563 CNAME. 1565 +---------------------+ 1566 | Endpoint A | 1567 | Local Area Network | 1568 | +------------+ | 1569 | +->| Audio |<+-RTP---\ 1570 | | +------------+ | \ +------+ 1571 | | +------------+ | +-->| | 1572 | +->| Video |<+-RTP-------->| B | 1573 | | +------------+ | +-->| | 1574 | | +------------+ | / +------+ 1575 | +->| Control |<+-SIP---/ 1576 | +------------+ | 1577 +---------------------+ 1579 Figure 20: Split Component Terminal 1581 It is further sensible that the subunits share a common clock from 1582 which RTP and RTCP clocks are derived, to facilitate synchronization 1583 and avoid clock drift. 1585 To indicate that audio and video Source Streams generated by 1586 different sub-units share a common clock, and can be synchronized, 1587 the RTP streams generated from those Source Streams need to include 1588 the same CNAME in their RTCP SDES packets. The use of a common CNAME 1589 for RTP flows carried in different transport-layer flows is entirely 1590 normal for RTP and RTCP senders, and fully compliant RTP endpoints, 1591 middle-boxes, and other tools should have no problem with this. 1593 However, outside of the split component terminal scenario (and 1594 perhaps a multi-homed endpoint scenario, which is not further 1595 discussed herein), the use of a common CNAME in RTP streams sent from 1596 separate endpoints (as opposed to a common CNAME for RTP streams sent 1597 on different transport layer flows between two endpoints) is rare. 1598 It has been reported that at least some third party tools like some 1599 network monitors do not handle endpoints that use of a common CNAME 1600 across multiple transport layer flows gracefully: they report an 1601 error condition that two separate endpoints are using the same CNAME. 1602 Depending on the sophistication of the support staff, such erroneous 1603 reports can lead to support issues. 1605 Aforementioned support issue can sometimes be avoided if each of the 1606 subunits of a split component terminal is configured to use a 1607 different CNAME, with the synchronization between the RTP streams 1608 being indicated by some non-RTP signaling channel rather than using a 1609 common CNAME sent in RTCP. This complicates the signaling, 1610 especially in cases where there are multiple SSRCs in use with 1611 complex synchronization requirements, as is the same in many current 1612 telepresence systems. Unless one uses RTCP terminating topologies 1613 such as Topo-RTCP-terminating-MCU, sessions involving more than one 1614 video subunit with a common CNAME are close to unavoidable. 1616 The different RTP streams comprising a split terminal system can form 1617 a single RTP session or they can form multiple RTP sessions, 1618 depending on the visibility of their SSRC values in RTCP reports. If 1619 the receiver of the RTP streams sent by the split terminal sends 1620 reports relating to all of the RTP flows (i.e., to each SSRC) in each 1621 RTCP report then a single RTP session is formed. Alternatively, if 1622 the receiver of the RTP streams sent by the split terminal does not 1623 send cross-reports in RTCP, then the audio and video form separate 1624 RTP sessions. 1626 For example, in the Figure 20, B will send RTCP reports to each of 1627 the sub-units of A. If the RTCP packets that B sends to the audio 1628 sub-unit of A include reports on the reception quality of the video 1629 as well as the audio, and similarly if the RTCP packets that B sends 1630 to the video sub-unit of A include reports on the reception quality 1631 of the audio as well as video, then a single RTP session is formed. 1632 However, if the RTCP packets B sends to the audio sub-unit of A only 1633 report on the received audio, and the RTCP packet B sends to the 1634 video sub-unit of A only report on the received video, then there are 1635 two separate RTP sessions. 1637 Forming a single RTP session across the RTP streams sent by the 1638 different sub-units of a split terminal gives each sub-unit 1639 visibility into reception quality of RTP streams sent by the other 1640 sub-units. This information can help diagnose reception quality 1641 problems, but at the cost of increased RTCP bandwidth use. 1643 RTP streams sent by the sub-units of a split terminal need to use the 1644 same CNAME in their RTCP packets if they are to be synchronized, 1645 irrespective of whether a single RTP session is formed or not. 1647 3.11. Non-Symmetric Mixer/Translators 1649 Shortcut name: Topo-Asymmetric 1651 It is theoretically possible to construct an MCU that is a Mixer in 1652 one direction and a Translator in another. The main reason to 1653 consider this would be to allow topologies similar to Figure 13, 1654 where the Mixer does not need to mix in the direction from B or D 1655 towards the multicast domains with A and C. Instead, the RTP streams 1656 from B and D are forwarded without changes. Avoiding this mixing 1657 would save media processing resources that perform the mixing in 1658 cases where it isn't needed. However, there would still be a need to 1659 mix B's media towards D. Only in the direction B -> multicast domain 1660 or D -> multicast domain would it be possible to work as a 1661 Translator. In all other directions, it would function as a Mixer. 1663 The Mixer/Translator would still need to process and change the RTCP 1664 before forwarding it in the directions of B or D to the multicast 1665 domain. One issue is that A and C do not know about the mixed-media 1666 stream the Mixer sends to either B or D. Therefore, any reports 1667 related to these streams must be removed. Also, receiver reports 1668 related to A and C's RTP streams would be missing. To avoid A and C 1669 thinking that B and D aren't receiving A and C at all, the Mixer 1670 needs to insert locally generated reports reflecting the situation 1671 for the streams from A and C into B and D's Sender Reports. In the 1672 opposite direction, the Receiver Reports from A and C about B's and 1673 D's stream also need to be aggregated into the Mixer's Receiver 1674 Reports sent to B and D. Since B and D only have the Mixer as source 1675 for the stream, all RTCP from A and C must be suppressed by the 1676 Mixer. 1678 This topology is so problematic and it is so easy to get the RTCP 1679 processing wrong, that it is not recommended for implementation. 1681 3.12. Combining Topologies 1683 Topologies can be combined and linked to each other using Mixers or 1684 Translators. However, care must be taken in handling the SSRC/CSRC 1685 space. A Mixer does not forward RTCP from sources in other domains, 1686 but instead generates its own RTCP packets for each domain it mixes 1687 into, including the necessary Source Description (SDES) information 1688 for both the CSRCs and the SSRCs. Thus, in a mixed domain, the only 1689 SSRCs seen will be the ones present in the domain, while there can be 1690 CSRCs from all the domains connected together with a combination of 1691 Mixers and Translators. The combined SSRC and CSRC space is common 1692 over any Translator or Mixer. It is important to facilitate loop 1693 detection, something that is likely to be even more important in 1694 combined topologies due to the mixed behavior between the domains. 1695 Any hybrid, like the Topo-Video-switch-MCU or Topo-Asymmetric, 1696 requires considerable thought on how RTCP is dealt with. 1698 4. Topology Properties 1700 The topologies discussed in Section 3 have different properties. 1701 This section describes these properties. Note that, even if a 1702 certain property is supported within a particular topology concept, 1703 the necessary functionality may be optional to implement. 1705 4.1. All to All Media Transmission 1707 To recapitulate, multicast, and in particular Any Source Multicast 1708 (ASM), provides the functionality that everyone may send to, or 1709 receive from, everyone else within the session. Source-specific 1710 Multicast (SSM) can provide a similar functionality by having anyone 1711 intending to participate as sender to send its media to the SSM 1712 distribution source. The SSM distribution source forwards the media 1713 to all receivers subscribed to the multicast group. Mesh, MCUs, 1714 Mixers, SFMs and Translators may all provide that functionality at 1715 least on some basic level. However, there are some differences in 1716 which type of reachability they provide. 1718 Closest to true IP-multicast-based, all-to-all transmission comes 1719 perhaps the transport Translator function called "relay" in 1720 Section 3.5, as well as the Mesh with joint RTP sessions. Media 1721 Translators, Mesh with independent RTP Sessions, Mixers, SFUs and the 1722 MCU variants do not provide a fully meshed forwarding on the 1723 transport level; instead, they only allow limited forwarding of 1724 content from the other session participants. 1726 The "all to all media transmission" requires that any media 1727 transmitting endpoint considers the path to the least capable 1728 receiving endpoint. Otherwise, the media transmissions may overload 1729 that path. Therefore, a sending endpoint needs to monitor the path 1730 from itself to any of the receiving endpoints, to detect the 1731 currently least capable receiver, and adapt its sending rate 1732 accordingly. As multiple endpoints may send simultaneously, the 1733 available resources may vary. RTCP's Receiver Reports help 1734 performing this monitoring, at least on a medium time scale. 1736 The resource consumption for performing all to all transmission 1737 varies depending with the topology. Both ASM and SSM have the 1738 benefit that only one copy of each packet traverses a particular 1739 link. Using a relay causes the transmission of one copy of a packet 1740 per endpoint-to-relay path and packet transmitted. However, in most 1741 cases the links carrying the multiple copies will be the ones close 1742 to the relay (which can be assumed to be part of the network 1743 infrastructure with good connectivity to the backbone), rather than 1744 the endpoints (which may be behind slower access links). The Mesh 1745 causes N-1 streams of transmitted packets to traverse the first hop 1746 link from the endpoint, in an N endpoint mesh. How long the 1747 different paths are common, is highly situation dependent. 1749 The transmission of RTCP by design adapts to any changes in the 1750 number of participants due to the transmission algorithm, defined in 1751 the RTP specification [RFC3550], and the extensions in AVPF [RFC4585] 1752 (when applicable). That way, the resources utilized for RTCP stay 1753 within the bounds configured for the session. 1755 4.2. Transport or Media Interoperability 1757 All Translators, Mixers, and RTCP-terminating MCU, and Mesh with 1758 individual RTP sessions, allow changing the media encoding or the 1759 transport to other properties of the other domain, thereby providing 1760 extended interoperability in cases where the endpoints lack a common 1761 set of media codecs and/or transport protocols. Selective Forwarding 1762 Middleboxes can adopt the transport, and (at least) selectively 1763 forward the encoded streams that match a receiving endpoint's 1764 capability. It requires an additional translator to change the media 1765 encoding if the encoded streams do not match the receiving endpoint's 1766 capabilities. 1768 4.3. Per Domain Bit-Rate Adaptation 1770 Endpoints are often connected to each other with a heterogeneous set 1771 of paths. This makes congestion control in a Point to Multipoint set 1772 problematic. For the ASM, SSM, Mesh with common RTP session, and 1773 Transport Relay scenario, each individual sending endpoint has to 1774 adapt to the receiving endpoint behind the least capable path, 1775 yielding suboptimal quality for the endpoints behind the more capable 1776 paths. This is no longer an issue when Media Translators, Mixers, 1777 SFM or MCUs are involved, as each endpoint only needs to adapt to the 1778 slowest path within its own domain. The Translator, Mixer, SFM, or 1779 MCU topologies all require their respective outgoing RTP streams to 1780 adjust the bit-rate, packet-rate, etc., to adapt to the least capable 1781 path in each of the other domains. That way one can avoid lowering 1782 the quality to the least-capable endpoint in all the domains at the 1783 cost (complexity, delay, equipment) of the Mixer, SFM or Translator, 1784 and potentially media sender (multicast/layered encoding and sending 1785 the different representations). 1787 4.4. Aggregation of Media 1789 In the all-to-all media property mentioned above and provided by ASM, 1790 SSM, Mesh with common RTP session, and relay, all simultaneous media 1791 transmissions share the available bit-rate. For endpoints with 1792 limited reception capabilities, this may result in a situation where 1793 even a minimal acceptable media quality cannot be accomplished, 1794 because multiple RTP streams need to share the same resources. One 1795 solution to this problem is to provide for a Mixer, or MCU to 1796 aggregate the multiple RTP streams into a single one, where the 1797 single RTP stream takes up less resources in terms of bit-rate. This 1798 aggregation can be performed according to different methods. Mixing 1799 or selection are two common methods. Selection is almost always 1800 possible and easy to implement. Mixing requires resources in the 1801 mixer, and may be relatively easy and not impairing the quality too 1802 badly (audio) or quite difficult (video tiling, which is not only 1803 computationally complex but also reduces the pixel count per stream, 1804 with corresponding loss in perceptual quality). 1806 4.5. View of All Session Participants 1808 The RTP protocol includes functionality to identify the session 1809 participants through the use of the SSRC and CSRC fields. In 1810 addition, it is capable of carrying some further identity information 1811 about these participants using the RTCP Source Descriptors (SDES). 1812 In topologies that provide a full all-to-all functionality, i.e. ASM, 1813 Mesh with common RTP session, Relay a compliant RTP implementation 1814 offers the functionality directly as specified in RTP. In topologies 1815 that do not offer all-to-all communication, it is necessary that RTCP 1816 is handled correctly in domain bridging function. RTP includes 1817 explicit specification text for Translators and Mixers, and for SFMs 1818 the required functionality can be derived from that text. However, 1819 the MCU described in Section 3.8 cannot offer the full functionality 1820 for session participant identification through RTP means. The 1821 topologies that create independent RTP sessions per endpoint or pair 1822 of endpoints, like Back-to-Back RTP session, MESH with independent 1823 RTP sessions, and the RTCP terminating MCU RTCP terminating MCU 1824 (Section 3.9), with an exception of SFM, do not support RTP based 1825 identification of session participants. In all those cases, other 1826 non-RTP based mechanisms need to be implemented if such knowledge is 1827 required or desirable. When it comes to SFM the SSRC name space is 1828 not necessarily joint, instead identification will require knowledge 1829 of SSRC/CSRC mappings that the SFM performed, see Section 3.7. 1831 4.6. Loop Detection 1833 In complex topologies with multiple interconnected domains, it is 1834 possible to unintentionally form media loops. RTP and RTCP support 1835 detecting such loops, as long as the SSRC and CSRC identities are 1836 maintained and correctly set in forwarded packets. Loop detection 1837 will work in ASM, SSM, Mesh with joint RTP session, and Relay. It is 1838 likely that loop detection works for the video switching MCU 1839 Section 3.8, at least as long as it forwards the RTCP between the 1840 endpoints. However, the Back-to-Back RTP sessions, Mesh with 1841 independent RTP sessions, SFM, will definitely break the loop 1842 detection mechanism. 1844 4.7. Consistency between header extensions and RTCP 1846 Some RTP header extensions have relevance not only end-to-end, but 1847 also hop-to-hop, meaning at least some of the middleboxes in the path 1848 are aware of their potential presence through signaling, intercept 1849 and interpret such header extensions and potentially also rewrite or 1850 generate them. Modern header extensions generally follow RFC 5285 1851 [RFC5285], which allows for all of the above. Examples for such 1852 header extensions include the mid (media ID) in [draft-ietf-mmusic- 1853 sdp-bundle-negotiation-12] [I-D.ietf-mmusic-sdp-bundle-negotiation]. 1854 At the time of writing there was also a proposal for how to include 1855 any SDES into an RTP header extension [draft-westerlund-avtext-dses- 1856 hdr-ext] [I-D.westerlund-avtext-sdes-hdr-ext]. 1858 When such header extensions are in use, any middlebox that 1859 understands it must ensure consistency between the extensions it sees 1860 and/or generates, and the RTCP it receives and generates. For 1861 example, the mid of bundle is sent in an RTP header extension and 1862 also in an RTCP SDES message. This apparent redundancy was 1863 introduced as unaware middleboxes may choose to discard RTP header 1864 extensions. Obviously, inconsistency between the media ID sent in 1865 the RTP header extension and in the RTCP SDES message could lead to 1866 undesirable results, and, therefore, consistency is needed. 1867 Middleboxes unaware of the nature of a header extension, as specified 1868 in RFC 5285 [RFC5285], are free to forward or discard header 1869 extensions. 1871 5. Comparison of Topologies 1873 The table below attempts to summarize the properties of the different 1874 topologies. The legend to the topology abbreviations are: Topo- 1875 Point-to-Point (PtP), Topo-ASM (ASM), Topo-SSM (SSM), Topo-Trns- 1876 Translator (TT), Topo-Media-Translator (including Transport 1877 Translator) (MT), Topo-Mesh with joint session (MJS), Topo-Mesh with 1878 individual sessions (MIS), Topo-Mixer (Mix), Topo-Asymmetric (ASY), 1879 Topo-Video-switch-MCU (VSM), and Topo-RTCP-terminating-MCU (RTM), 1880 Selective Forwarding Middlebox (SFM). In the table below, Y 1881 indicates Yes or full support, N indicates No support, (Y) indicates 1882 partial support, and N/A indicates not applicable. 1884 Property PtP ASM SSM TT MT MJS MIS Mix ASY VSM RTM SFM 1885 --------------------------------------------------------------------- 1886 All to All media N Y (Y) Y Y Y (Y) (Y) (Y) (Y) (Y) (Y) 1887 Interoperability N/A N N Y Y Y Y Y Y N Y Y 1888 Per Domain Adaptation N/A N N N Y N Y Y Y N Y Y 1889 Aggregation of media N N N N N N N Y (Y) Y Y N 1890 Full Session View Y Y Y Y Y Y N Y Y (Y) N Y 1891 Loop Detection Y Y Y Y Y Y N Y Y (Y) N N 1893 Please note that the Media Translator also includes the transport 1894 Translator functionality. 1896 6. Security Considerations 1898 The use of Mixers, SFMs and Translators has impact on security and 1899 the security functions used. The primary issue is that both Mixers, 1900 SFMs and Translators modify packets, thus preventing the use of 1901 integrity and source authentication, unless they are trusted devices 1902 that take part in the security context, e.g., the device can send 1903 Secure Realtime Transport Protocol (SRTP) and Secure Realtime 1904 Transport Control Protocol (SRTCP) [RFC3711] packets to endpoints in 1905 the Communication Session. If encryption is employed, the media 1906 Translator, SFM and Mixer need to be able to decrypt the media to 1907 perform its function. A transport Translator may be used without 1908 access to the encrypted payload in cases where it translates parts 1909 that are not included in the encryption and integrity protection, for 1910 example, IP address and UDP port numbers in a media stream using SRTP 1911 [RFC3711]. However, in general, the Translator, SFM or Mixer needs 1912 to be part of the signalling context and get the necessary security 1913 associations (e.g., SRTP crypto contexts) established with its RTP 1914 session participants. 1916 Including the Mixer, SFM and Translator in the security context 1917 allows the entity, if subverted or misbehaving, to perform a number 1918 of very serious attacks as it has full access. It can perform all 1919 the attacks possible (see RFC 3550 and any applicable profiles) as if 1920 the media session were not protected at all, while giving the 1921 impression to the human session participants that they are protected. 1923 Transport Translators have no interactions with cryptography that 1924 works above the transport layer, such as SRTP, since that sort of 1925 Translator leaves the RTP header and payload unaltered. Media 1926 Translators, on the other hand, have strong interactions with 1927 cryptography, since they alter the RTP payload. A media Translator 1928 in a session that uses cryptographic protection needs to perform 1929 cryptographic processing to both inbound and outbound packets. 1931 A media Translator may need to use different cryptographic keys for 1932 the inbound and outbound processing. For SRTP, different keys are 1933 required, because an RFC 3550 media Translator leaves the SSRC 1934 unchanged during its packet processing, and SRTP key sharing is only 1935 allowed when distinct SSRCs can be used to protect distinct packet 1936 streams. 1938 When the media Translator uses different keys to process inbound and 1939 outbound packets, each session participant needs to be provided with 1940 the appropriate key, depending on whether they are listening to the 1941 Translator or the original source. (Note that there is an 1942 architectural difference between RTP media translation, in which 1943 participants can rely on the RTP Payload Type field of a packet to 1944 determine appropriate processing, and cryptographically protected 1945 media translation, in which participants must use information that is 1946 not carried in the packet.) 1948 When using security mechanisms with Translators, SFMs and Mixers, it 1949 is possible that the Translator, SFM or Mixer could create different 1950 security associations for the different domains they are working in. 1951 Doing so has some implications: 1953 First, it might weaken security if the Mixer/Translator accepts a 1954 weaker algorithm or key in one domain than in another. Therefore, 1955 care should be taken that appropriately strong security parameters 1956 are negotiated in all domains. In many cases, "appropriate" 1957 translates to "similar" strength. If a key management system does 1958 allow the negotiation of security parameters resulting in a different 1959 strength of the security, then this system should notify the 1960 participants in the other domains about this. 1962 Second, the number of crypto contexts (keys and security related 1963 state) needed (for example, in SRTP [RFC3711]) may vary between 1964 Mixers, SFMs and Translators. A Mixer normally needs to represent 1965 only a single SSRCs per domain and therefore needs to create only one 1966 security association (SRTP crypto context) per domain. In contrast, 1967 a Translator needs one security association per participant it 1968 translates towards, in the opposite domain. Considering Figure 11, 1969 the Translator needs two security associations towards the multicast 1970 domain, one for B and one for D. It may be forced to maintain a set 1971 of totally independent security associations between itself and B and 1972 D respectively, so as to avoid two-time pad occurrences. These 1973 contexts must also be capable of handling all the sources present in 1974 the other domains. Hence, using completely independent security 1975 associations (for certain keying mechanisms) may force a Translator 1976 to handle N*DM keys and related state; where N is the total number of 1977 SSRCs used over all domains and DM is the total number of domains. 1979 The multicast based (ASM and SSM), Relay and Mesh with common RTP 1980 session are all topologies with multiple endpoints that require 1981 shared knowledge about the different crypto contexts for the 1982 endpoints. These multi-party topologies have special requirements on 1983 the key-management as well as the security functions. Specifically 1984 source-authentication in these environments has special requirements. 1986 There exist a number of different mechanisms to provide keys to the 1987 different participants. One example is the choice between group keys 1988 and unique keys per SSRC. The appropriate keying model is impacted 1989 by the topologies one intends to use. The final security properties 1990 are dependent on both the topologies in use and the keying 1991 mechanisms' properties, and need to be considered by the application. 1992 Exactly which mechanisms are used is outside of the scope of this 1993 document. Please review RTP Security Options [RFC7201] to get a 1994 better understanding of most of the available options. 1996 7. IANA Considerations 1998 This document makes no request of IANA. 2000 Note to RFC Editor: this section may be removed on publication as an 2001 RFC. 2003 8. Acknowledgements 2005 The authors would like to thank Mark Baugher, Bo Burman, Umesh 2006 Chandra, Alex Eleftheriadis, Roni Even, Ladan Gharai, Geoff Hunt, 2007 Keith Lantz, Jonathan Lennox, Scarlet Liuyan, Suhas Nandakumar, and 2008 Colin Perkins for their help in reviewing and improving this 2009 document. 2011 9. References 2013 9.1. Normative References 2015 [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. 2016 Jacobson, "RTP: A Transport Protocol for Real-Time 2017 Applications", STD 64, RFC 3550, July 2003. 2019 [RFC4585] Ott, J., Wenger, S., Sato, N., Burmeister, C., and J. Rey, 2020 "Extended RTP Profile for Real-time Transport Control 2021 Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585, July 2022 2006. 2024 9.2. Informative References 2026 [I-D.ietf-avtcore-rtp-multi-stream-optimisation] 2027 Lennox, J., Westerlund, M., Wu, W., and C. Perkins, 2028 "Sending Multiple Media Streams in a Single RTP Session: 2029 Grouping RTCP Reception Statistics and Other Feedback", 2030 draft-ietf-avtcore-rtp-multi-stream-optimisation-05 (work 2031 in progress), February 2015. 2033 [I-D.ietf-mmusic-sdp-bundle-negotiation] 2034 Holmberg, C., Alvestrand, H., and C. Jennings, 2035 "Negotiating Media Multiplexing Using the Session 2036 Description Protocol (SDP)", draft-ietf-mmusic-sdp-bundle- 2037 negotiation-16 (work in progress), January 2015. 2039 [I-D.westerlund-avtext-sdes-hdr-ext] 2040 Westerlund, M., Even, R., and M. Zanaty, "RTP Header 2041 Extension for RTCP Source Description Items", draft- 2042 westerlund-avtext-sdes-hdr-ext-03 (work in progress), 2043 November 2014. 2045 [RFC1112] Deering, S., "Host extensions for IP multicasting", STD 5, 2046 RFC 1112, August 1989. 2048 [RFC3022] Srisuresh, P. and K. Egevang, "Traditional IP Network 2049 Address Translator (Traditional NAT)", RFC 3022, January 2050 2001. 2052 [RFC3569] Bhattacharyya, S., "An Overview of Source-Specific 2053 Multicast (SSM)", RFC 3569, July 2003. 2055 [RFC3711] Baugher, M., McGrew, D., Naslund, M., Carrara, E., and K. 2056 Norrman, "The Secure Real-time Transport Protocol (SRTP)", 2057 RFC 3711, March 2004. 2059 [RFC4575] Rosenberg, J., Schulzrinne, H., and O. Levin, "A Session 2060 Initiation Protocol (SIP) Event Package for Conference 2061 State", RFC 4575, August 2006. 2063 [RFC4607] Holbrook, H. and B. Cain, "Source-Specific Multicast for 2064 IP", RFC 4607, August 2006. 2066 [RFC5104] Wenger, S., Chandra, U., Westerlund, M., and B. Burman, 2067 "Codec Control Messages in the RTP Audio-Visual Profile 2068 with Feedback (AVPF)", RFC 5104, February 2008. 2070 [RFC5117] Westerlund, M. and S. Wenger, "RTP Topologies", RFC 5117, 2071 January 2008. 2073 [RFC5285] Singer, D. and H. Desineni, "A General Mechanism for RTP 2074 Header Extensions", RFC 5285, July 2008. 2076 [RFC5760] Ott, J., Chesterfield, J., and E. Schooler, "RTP Control 2077 Protocol (RTCP) Extensions for Single-Source Multicast 2078 Sessions with Unicast Feedback", RFC 5760, February 2010. 2080 [RFC5766] Mahy, R., Matthews, P., and J. Rosenberg, "Traversal Using 2081 Relays around NAT (TURN): Relay Extensions to Session 2082 Traversal Utilities for NAT (STUN)", RFC 5766, April 2010. 2084 [RFC6285] Ver Steeg, B., Begen, A., Van Caenegem, T., and Z. Vax, 2085 "Unicast-Based Rapid Acquisition of Multicast RTP 2086 Sessions", RFC 6285, June 2011. 2088 [RFC6465] Ivov, E., Marocco, E., and J. Lennox, "A Real-time 2089 Transport Protocol (RTP) Header Extension for Mixer-to- 2090 Client Audio Level Indication", RFC 6465, December 2011. 2092 [RFC7201] Westerlund, M. and C. Perkins, "Options for Securing RTP 2093 Sessions", RFC 7201, April 2014. 2095 Authors' Addresses 2097 Magnus Westerlund 2098 Ericsson 2099 Farogatan 6 2100 SE-164 80 Kista 2101 Sweden 2103 Phone: +46 10 714 82 87 2104 Email: magnus.westerlund@ericsson.com 2106 Stephan Wenger 2107 Vidyo 2108 433 Hackensack Ave 2109 Hackensack, NJ 07601 2110 USA 2112 Email: stewe@stewe.org