idnits 2.17.1 draft-ietf-avtcore-idms-05.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 3 instances of too long lines in the document, the longest one being 71 characters in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (June 13, 2012) is 4335 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Outdated reference: A later version (-01) exists of draft-williams-avtcore-clksrc-00 ** Obsolete normative reference: RFC 4566 (Obsoleted by RFC 8866) -- Possible downref: Non-RFC (?) normative reference: ref. 'TS183063' -- No information found for draft-gross-leap-seconds - is the name correct? Summary: 2 errors (**), 0 flaws (~~), 2 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 AVTCore R. van Brandenburg 3 Internet-Draft H. Stokking 4 Intended status: Standards Track O. van Deventer 5 Expires: December 15, 2012 TNO 6 F. Boronat 7 M. Montagud 8 Universitat Politecnica de 9 Valencia 10 K. Gross 11 AVA Networks 12 June 13, 2012 14 RTCP for inter-destination media synchronization 15 draft-ietf-avtcore-idms-05 17 Abstract 19 This document gives information on an RTCP Packet Type and RTCP XR 20 Block Type including associated SDP parameters for Inter-Destination 21 Media Synchronization (IDMS). The RTCP XR Block Type, registered 22 with IANA based on an ETSI specification, is used to collect media 23 play-out information from participants in a group playing-out 24 (watching, listening, etc.) a specific RTP media stream. The RTCP 25 packet type specified by this document is used to distribute a common 26 target play-out point to which all the distributed receivers, sharing 27 a media experience, can synchronize. 29 Typical use cases in which IDMS is usefull are social TV, shared 30 service control (i.e. applications where two or more geographically 31 separated users are watching a media stream together), distance 32 learning, networked video walls, networked loudspeakers, etc. 34 Status of this Memo 36 This Internet-Draft is submitted in full conformance with the 37 provisions of BCP 78 and BCP 79. 39 Internet-Drafts are working documents of the Internet Engineering 40 Task Force (IETF). Note that other groups may also distribute 41 working documents as Internet-Drafts. The list of current Internet- 42 Drafts is at http://datatracker.ietf.org/drafts/current/. 44 Internet-Drafts are draft documents valid for a maximum of six months 45 and may be updated, replaced, or obsoleted by other documents at any 46 time. It is inappropriate to use Internet-Drafts as reference 47 material or to cite them other than as "work in progress." 48 This Internet-Draft will expire on December 15, 2012. 50 Copyright Notice 52 Copyright (c) 2012 IETF Trust and the persons identified as the 53 document authors. All rights reserved. 55 This document is subject to BCP 78 and the IETF Trust's Legal 56 Provisions Relating to IETF Documents 57 (http://trustee.ietf.org/license-info) in effect on the date of 58 publication of this document. Please review these documents 59 carefully, as they describe your rights and restrictions with respect 60 to this document. Code Components extracted from this document must 61 include Simplified BSD License text as described in Section 4.e of 62 the Trust Legal Provisions and are provided without warranty as 63 described in the Simplified BSD License. 65 Table of Contents 67 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 68 1.1. Inter-Destination Media Synchronization . . . . . . . . . 4 69 1.2. Applicability of RTCP to IDMS . . . . . . . . . . . . . . 4 70 1.3. Applicability of SDP to IDMS . . . . . . . . . . . . . . . 5 71 1.4. This document and ETSI TISPAN . . . . . . . . . . . . . . 5 72 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 73 3. Overview of IDMS operation . . . . . . . . . . . . . . . . . . 5 74 4. Inter-Destination Media Synchronization use cases . . . . . . 7 75 5. Architecture for Inter-Destination Media Synchronization . . . 8 76 5.1. Media Synchronization Application Server (MSAS) . . . . . 8 77 5.2. Synchronization Client (SC) . . . . . . . . . . . . . . . 9 78 5.3. Communication between MSAS and SCs . . . . . . . . . . . . 9 79 6. RTCP XR Block Type for IDMS . . . . . . . . . . . . . . . . . 9 80 7. RTCP Packet Type for IDMS (IDMS report) . . . . . . . . . . . 11 81 8. Timing and NTP Considerations . . . . . . . . . . . . . . . . 13 82 9. SDP Parameter for RTCP XR IDMS Block Type . . . . . . . . . . 14 83 10. SDP Parameter for RTCP IDMS Packet Type . . . . . . . . . . . 15 84 11. Compatibility with ETSI TISPAN . . . . . . . . . . . . . . . . 15 85 12. On the use of presentation timestamps . . . . . . . . . . . . 17 86 13. Security Considerations . . . . . . . . . . . . . . . . . . . 18 87 14. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 18 88 15. Contributors . . . . . . . . . . . . . . . . . . . . . . . . . 19 89 16. References . . . . . . . . . . . . . . . . . . . . . . . . . . 19 90 16.1. Normative References . . . . . . . . . . . . . . . . . . . 19 91 16.2. Informative References . . . . . . . . . . . . . . . . . . 20 92 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 21 94 1. Introduction 96 1.1. Inter-Destination Media Synchronization 98 Inter-Destination Media Synchronization (IDMS) refers to the play-out 99 of media streams at two or more geographically distributed locations 100 in a time synchronized manner. It can be applied to both unicast and 101 multicast media streams and can be applied to any type and/or 102 combination of streaming media, such as audio, video and text 103 (subtitles).[Ishibashi2006] and [Boronat2009] provide an overview of 104 technologies and algorithms for IDMS. 106 IDMS requires the exchange of information on media receipt and play- 107 out times among participants in an IDMS session. It may also require 108 signaling for the initiation and maintenance of IDMS sessions and 109 groups of receivers. 111 The presented RTCP specification for IDMS is independent of the used 112 synchronization algorithm, which is out-of-scope of this document. 114 1.2. Applicability of RTCP to IDMS 116 Currently, a large share of real-time applications make use of RTP 117 and RTCP [RFC3550]. RTP provides end-to-end network transport 118 functions suitable for applications requiring real-time data 119 transport, such as audio, video or data, over multicast or unicast 120 network services. The timestamps, sequence numbers, and payload 121 (content) type identification mechanisms provided by RTP packets are 122 very useful for reconstructing the original media timing, and for 123 reordering and detecting packet loss at the client side. 125 The data transport is augmented by a control protocol (RTCP) to allow 126 monitoring of the data delivery in a manner that is scalable to large 127 multicast networks, and to provide minimal control and identification 128 functionality. RTP receivers and senders provide reception quality 129 feedback by sending out RTCP Receiver Report (RR) and Sender Report 130 (SR) packets [RFC3550], respectively, which may be augmented by 131 eXtended Reports (XR) [RFC3611]. Both RTP and RTCP are intended to 132 be tailored through modifications in order to include profile- 133 specific information required by particular applications, and the 134 guidelines on doing so are specified in [RFC5868]. 136 IDMS involves the collection, summarizing and distribution of RTP 137 packet arrival and play-out times. As information on RTP packet 138 arrival times and play-out times can be considered reception quality 139 feedback information, RTCP is well suited for carrying out IDMS, 140 which may facilitate the implementation and deployment in typical 141 multimedia applications. 143 1.3. Applicability of SDP to IDMS 145 RTCP XR [RFC3611] defines the Extended Report (XR) packet type for 146 the RTP Control Protocol (RTCP), and defines how the use of XR 147 packets can be signaled by an application using the Session 148 Description Protocol (SDP) [RFC4566]. 150 SDP signaling is used to set up and maintain a synchronization group 151 between Synchronization Clients (SCs). This document describes two 152 SDP parameters for doing this, one for the RTCP XR block type and one 153 for the new RTCP packet type. 155 1.4. This document and ETSI TISPAN 157 ETSI TISPAN [TS183063] has specified architecture and protocol for 158 IDMS using RTCP XR exchange and SDP signaling. For more information 159 on how this document relates to [TS183063], see Section 11. 161 2. Terminology 163 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 164 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 165 document are to be interpreted as described in RFC 2119 [RFC2119] and 166 indicate requirement levels for compliant implementations. 168 3. Overview of IDMS operation 170 This section provides a brief example of how the RTCP functionality 171 is used for achieving IDMS. The section is tutorial in nature and 172 does not contain any normative statements. 174 Alice's . . . . . . .tv:abc.com . . . . . . . . . Bob's 175 TV (Sync Client) (Sync Server) Laptop (Sync Client) 176 | | | 177 | Media Session | | 178 |<=====================>| | 179 | Invite(URL,Sync-group ID) | 180 |------------------------------------------------->| 181 | | Media Session Set-up | 182 | |<========================>| 183 | | | 184 | Call set-up | 185 |<================================================>| 186 | | | 187 | RTP Packet | RTP Packet | 188 |<----------------------|------------------------->| 189 | RR + IDMS XR | | 190 |---------------------->| RR + IDMS XR | 191 | |<-------------------------| 192 | RTCP IDMS packet | RTCP IDMS packet | 193 |<----------------------|------------------------->| 194 | | | 196 Alice is watching TV in her living room. At some point she sees that 197 a football game of Bob's favorite team is on. She sends him an 198 invite to watch the program together. Embedded in the invitation is 199 the link to the media server and a unique sync-group identifier. 201 Bob, who is also at home, receives the invite on his laptop. He 202 accepts Alice's invitation and the RTP client on his laptop sets up a 203 session to the media server. A VoIP connection to Alice's TV is also 204 set up, so that Alice and Bob can talk while watching the game 205 together. 207 As is common with RTP, both the RTP client in Alice's TV as well as 208 the one in Bob's laptop send periodic RTCP Receiver Reports (RR) to 209 the media server. However, in order to make sure Alice and Bob see 210 the events in the football game at (approximately) the same time, 211 their clients also periodically send an IDMS XR block to the sync 212 server function of the media server. Included in the XR blocks are 213 timestamps on when both Alice and Bob received (or played out) a 214 particular RTP packet. 216 The sync server function in the media server calculates a reference 217 client from the received IDMS XR blocks (e.g. by selecting whichever 218 client received the packet the latest as the reference client). It 219 then sends an RTCP IDMS packet containing the play-out information of 220 this reference client to the sync clients of both Alice and Bob. 222 In this case Bob's connection has the longest delay and the reference 223 client therefore includes a delay similar to the one experienced by 224 Bob. Upon reception of this information, Alice's RTP client can 225 choose what to do with this information. In this case it decreases 226 its play-out rate temporarily until the play-out time matches with 227 the reference client play-out (and thus matches Bob's play-out). 228 Another option for Alice's TV would be to simply pause playback until 229 it catches up. The exact implementation of the synchronization 230 algorithm is up to the client. 232 Upon reception of the reference client RTCP IDMS packet, Bob's client 233 does not have to do anything since it is already synchronized to the 234 reference client (since it is based on Bob's delay). Note that other 235 synchronization algorithms may introduce even more delay than the one 236 experienced by the most delayed client, e.g. to account for delay 237 variations, for new clients joining an existing synchronization 238 group, etc. 240 4. Inter-Destination Media Synchronization use cases 242 There are a large number of use cases in which IDMS might be useful. 243 This section will highlight some of them. It should be noted that 244 this section is in no way meant to be exhaustive. 246 A first usage scenario for IDMS is Social TV. Social TV is the 247 combination of media content consumption by two or more users at 248 different devices and locations combined with the real-time 249 communication between those users. An example of Social TV is when 250 two or more users are watching the same television broadcast at 251 different devices and locations, while communicating with each other 252 using text, audio and/or video. A skew in their media play-out 253 processes can have adverse effects on their experience. A well-known 254 use case here is one friend experiencing a goal in a football match 255 well before or after other friend(s). 257 Another potential use case for IDMS is a networked video wall. A 258 video wall consists of multiple computer monitors, video projectors, 259 or television sets tiled together contiguously or overlapped in order 260 to form one large screen. Each of the screens reproduces a portion 261 of the larger picture. In some implementations, each screen may be 262 individually connected to the network and receive its portion of the 263 overall image from a network-connected video server or video scaler. 264 Screens are refreshed at 60 hertz (every 16-2/3 milliseconds) or 265 potentially faster. If the refresh is not synchronized, the effect 266 of multiple screens acting as one is broken. 268 A third usage scenario is that of the networked loudspeakers, in 269 which two or more speakers are connected to the network individually. 270 Such situations can for example be found in large conference rooms, 271 legislative chambers, classrooms (especially those supporting 272 distance learning) and other large-scale environments such as 273 stadiums. Since humans are more susceptible to differences in audio 274 delay, this use case needs even more accuracy than the video wall use 275 case. Depending on the exact application, the need for accuracy can 276 then be in the range of microseconds. 278 5. Architecture for Inter-Destination Media Synchronization 280 The architecture for IDMS, which is based on a sync-maestro 281 architecture [Boronat2009], is sketched below. The Synchronization 282 Client (SC) and Media Synchronization Application Server (MSAS) 283 entities are shown as additional functionality for the RTP receiver 284 and sender respectively. 286 It should be noted that a master/slave type of architecture is also 287 supported by having one of the SC devices also act as an MSAS. In 288 this case the MSAS functionality is thus embedded in an RTP receiver 289 instead of an RTP sender. 291 +-----------------------+ +-----------------------+ 292 | | SR + | | 293 | RTP Receiver | RTCP | RTP Sender | 294 | | IDMS | | 295 | +-----------------+ | <----- | +-----------------+ | 296 | | | | | | | | 297 | | Synchronization | | | | Media | | 298 | | Client | | | | Synchronization | | 299 | | (SC) | | | | Application | | 300 | | | | | | Server | | 301 | | | | RR+XR | | (MSAS) | | 302 | | | | -----> | | | | 303 | +-----------------+ | | +-----------------+ | 304 | | | | 305 +-----------------------+ +-----------------------+ 307 5.1. Media Synchronization Application Server (MSAS) 309 An MSAS collects RTP packet arrival times and play-out times from one 310 or more SC(s) in a synchronization group. The MSAS summarizes and 311 distributes this information to the SCs in the synchronization group 312 as synchronization settings, e.g. by determining the SC with the most 313 lagged play-out and using its reported RTP packet arrival time and 314 play-out time as a summary. 316 5.2. Synchronization Client (SC) 318 An SC reports on RTP packet arrival times and play-out times of a 319 media stream. It can receive summaries of such information, and use 320 that to adjust its play-out buffer. 322 5.3. Communication between MSAS and SCs 324 Two different message types are used for the communication between 325 MSAS and SCs. For the SC->MSAS message containing the play-out 326 information of a particular client, an RTCP XR Block Type is used 327 (see Section 6). For the MSAS->SC message containing the 328 synchronization settings instructions, a new RTCP Packet Type is 329 defined (see Section 7). 331 6. RTCP XR Block Type for IDMS 333 This section describes the RTCP XR Block Type for reporting IDMS 334 information on an RTP media stream. Its definition is based on 335 [RFC3611]. The RTCP XR is used to provide feedback information on 336 receipt times and presentation times of RTP packets to e.g. a Sender 337 [RFC3611], a Feedback Target [RFC5760] or a Third Party Monitor 338 [RFC3611]. 340 0 1 2 3 341 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 342 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 343 |V=2|P| Resrv | PT=XR=207 | length | 344 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 345 | SSRC of packet sender | 346 +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ 347 | BT=12 | SPST |Resrv|P| block length=7 | 348 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 349 | PT | Resrv | 350 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 351 | Media Stream Correlation Identifier | 352 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 353 | SSRC of media source | 354 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 355 | Packet Received NTP timestamp, most significant word | 356 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 357 | Packet Received NTP timestamp, least significant word | 358 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 359 | Packet Received RTP timestamp | 360 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 361 | Packet Presented NTP timestamp | 362 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 364 The first 64 bits form the header of the RTCP XR, as defined in 365 [RFC3611]. The SSRC of packet sender identifies the sender of the 366 specific RTCP packet. 368 The IDMS report block consists of 8 32-bit words, with the following 369 fields: 371 Block Type (BT): 8 bits. It identifies the block format. Its value 372 SHALL be set to 12. 374 Synchronization Packet Sender Type (SPST): 4 bits. This field 375 identifies the role of the packet sender for this specific eXtended 376 Report. It can have the following values: 378 SPST=0 Reserved For future use. 380 SPST=1 The packet sender is an SC. It uses this XR to report 381 synchronization status information. Timestamps relate to the SC 382 input. 384 SPST=2 This setting is reserved in order to preserve compatibility 385 with ETSI TISPAN [TS183063]. See Section 11 for more information. 387 SPST=3-15 Reserved For future use. 389 Reserved bits (Resrv): 3 bits. These bits are reserved for future 390 definition. In the absence of such a definition, the bits in this 391 field MUST be set to zero and MUST be ignored by the receiver. 393 Packet Presented NTP timestamp flag (P): 1 bit. Bit set to 1 if the 394 Packet Presented NTP timestamp field contains a value, 0 if it is 395 empty. If this flag is set to zero, then the Packet Presented NTP 396 timestamp SHALL be ignored. 398 Block Length: 16 bits. This field indicates the length of the block 399 in 32 bit words minus one and SHALL be set to 7, as this RTCP Block 400 Type has a fixed length. 402 Payload Type (PT): 7 bits. This field identifies the format of the 403 media payload, according to [RFC3551]. The media payload is 404 associated with an RTP timestamp clock rate. This clock rate 405 provides the time base for the RTP timestamp counter. 407 Reserved bits (Resrv): 25 bits. These bits are reserved for future 408 use and SHALL be set to 0. 410 Media Stream Correlation Identifier: 32 bits. This identifier is 411 used to correlate synchronized media streams. The value 0 (all bits 412 are set "0") indicates that this field is empty. The value 2^32-1 413 (all bits are set "1") is reserved for future use. If the RTCP 414 Packet Sender is an SC (SPST=1) or an MSAS (SPST=2), then the Media 415 Stream Correlation Identifier maps on the Synchronization Group 416 Identifier (SyncGroupId) to which the report applies. 418 SSRC: 32 bits. The SSRC of the media source SHALL be set to the 419 value of the SSRC identifier carried in the RTP header [RFC3550] of 420 the RTP packet to which the XR relates. 422 Packet Received NTP timestamp: 64 bits. This timestamp reflects the 423 wall clock time at the moment of arrival of the first octet of the 424 RTP packet to which the XR relates. It is formatted based on the NTP 425 timestamp format as specified in [RFC5905]. See Section 8 for more 426 information on how this field is used. 428 Packet Received RTP timestamp: 32 bits. This timestamp has the value 429 of the RTP timestamp carried in the RTP header [RFC3550] of the RTP 430 packet to which the XR relates. Several consecutive RTP packets will 431 have equal timestamps if they are (logically) generated at once, 432 e.g., belong to the same video frame. It may well be the case that 433 one receiver reports on the first RTP packet having a certain RTP 434 timestamp and a second receiver reports on the last RTP packet having 435 that same RTP timestamp. This would lead to an error in the 436 synchronization algorithm due to the faulty interpretation of 437 considering both reports to be on the same RTP packet. To solve 438 this, an SC SHOULD report on RTP packets in which a certain RTP 439 timestamp shows up for the first time. 441 Packet Presented NTP timestamp: 32 bits. This timestamp reflects the 442 wall clock time at the moment the rendered frame contained in the 443 first byte of the associated RTP packet is presented to the user. It 444 is based on the time format used by NTP and consists of the least 445 significant 16 bits of the NTP seconds part and the most significant 446 16 bits of the NTP fractional second part. If this field is empty, 447 then it SHALL be set to 0 and the Packet Presented NTP timestamp flag 448 (P) SHALL be set to 0. Presented here means the moment the data is 449 played out to the user of the system, i.e. sound played out through 450 speakers, video images being displayed on some display, etc. The 451 accuracy resulting from the synchronization algorithm will only be as 452 good as the accuracy with which the receivers can determine the delay 453 between receiving packets and presenting them to the end-user. 455 7. RTCP Packet Type for IDMS (IDMS report) 457 This section specifies the RTCP Packet Type for indicating 458 synchronization settings instructions to the receivers of the RTP 459 media stream. Its definition is based on [RFC3550]. 461 0 1 2 3 462 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 463 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 464 |V=2|P| Resrv | PT=TBD | length | 465 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 466 | SSRC of packet sender | 467 +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ 468 | SSRC of media source | 469 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 470 | Media Stream Correlation Identifier | 471 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 472 | Packet Received NTP timestamp, most significant word | 473 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 474 | Packet Received NTP timestamp, least significant word | 475 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 476 | Packet Received RTP timestamp | 477 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 478 | Packet Presented NTP timestamp, most significant word | 479 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 480 | Packet Presented NTP timestamp, least significant word | 481 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 483 The first 64 bits form the header of the RTCP Packet Type, as defined 484 in [RFC3550]. The SSRC of packet sender identifies the sender of the 485 specific RTCP packet. 487 The RTCP IDMS packet consists of 7 32-bit words, with the following 488 fields: 490 SSRC: 32 bits. The SSRC of the media source SHALL be set to the 491 value of the SSRC identifier carried in the RTP header [RFC3550] of 492 the RTP packet to which the RTCP IDMS packet relates. 494 Media Stream Correlation Identifier: 32 bits. This identifier is 495 used to correlate synchronized media streams. The value 0 (all bits 496 are set "0") indicates that this field is empty. The value 2^32-1 497 (all bits are set "1") is reserved for future use. The Media Stream 498 Correlation Identifier maps on the SyncGroupId of the group to which 499 this packet is sent. 501 Packet Received NTP timestamp: 64 bits. This timestamp reflects the 502 wall clock time at the reference client at the moment it received the 503 first octet of the RTP packet to which this packet relates. It can 504 be used by the synchronization algorithm on the receiving SC to 505 adjust its play-out timing in order to achieve synchronization, e.g. 506 to set the required play-out delay. The timestamp is formatted based 507 on the NTP timestamp format as specified in [RFC5905]. See Section 8 508 for more information on how this field is used. 510 Packet Received RTP timestamp: 32 bits. This timestamp has the value 511 of the RTP timestamp carried in the RTP header [RFC3550] of the RTP 512 packet to which the XR relates. This SHOULD relate to the first 513 arriving RTP packet containing this particular RTP timestamp, in case 514 multiple RTP packets contain the same RTP timestamp. 516 Packet Presented NTP timestamp: 64 bits. This timestamp reflects the 517 wall clock time at the reference client at the moment it presented 518 the rendered frame contained in the first octet of the associated RTP 519 packet to the user. The timestamp is formatted based on the NTP 520 timestamp format as specified in [RFC5905]. If this field is empty, 521 then it SHALL be set to 0. This field MAY be left empty if none or 522 only one of the receivers reported on presentation timestamps. 523 Presented here means the moment the data is played out to the user of 524 the system. 526 In some use cases (e.g. phased array transducers), the level of 527 control an MSAS might need to have over the exact moment of play-out 528 is so precise that a 32bit Presented Timestamp will not suffice. For 529 this reason, this RTCP Packet Type for IDMS includes a 64bit 530 Presented Timestamp field. Since an MSAS will in practice always add 531 some extra delay to the delay reported by the most lagged receiver 532 (to account for packet jitter), it suffices for the IDMS XR Block 533 Type with which the SCs report on their play-out to have a 32bit 534 Presented Timestamp field. 536 8. Timing and NTP Considerations 538 To achieve IDMS, the different receivers involved need synchronized 539 clocks as a common timeline for synchronization. Depending on the 540 synchronization accuracy required, different clock synchronization 541 methods can be used. For social TV, synchronization accuracy should 542 be achieved on the order of hundreds of milliseconds. In that case, 543 correct use of NTP on receivers will in most situations achieve the 544 required accuracy. As a guideline, to deal with clock drift of 545 receivers, receivers should synchronize their clocks at the beginning 546 of a synchronized session. In case of high required accuracy, the 547 synchronized clocks of different receivers should not drift beyond 548 the accuracy required for the synchronization mechanism. In 549 practice, this can mean that receivers need to synchronize their 550 clocks repeatedly during a synchronization session. 552 Because of the stringent synchronization requirements for achieving 553 good audio in some use cases, a high accuracy will be needed. In 554 this case, use of the global NTP system may not be sufficient. For 555 improved accuracy, a local NTP server could be set up, or some other 556 more accurate clock synchronization mechanism can be used, such as 557 GPS time or the Precision Time Protocol [IEEE-1588]. 559 [I-D.draft-williams-avtcore-clksrc] defines a set of SDP parameters 560 for signaling the clock synchronization source or sources available 561 to and used by the individual receivers. Using these paramenters, an 562 SC can indicate which synchronization source is being used at the 563 moment, the last time the SC synchronized with this source and the 564 synchronization frequency. An SC can also indicate any other 565 synchronization sources available to it. This allows multiple SCs in 566 an IDMS session to use the same or a similar clock source for their 567 session. 569 Applications performing IDMS may or may not be able to choose a 570 synchronization method for the system clock, because this may be a 571 system-wide setting which the application cannot change. How 572 applications deal with this is up to the implementation. The 573 application might control the system clock, or it might use a 574 separate application clock or even a separate IDMS session clock. It 575 might also report on the system clock and the synchronization method 576 used, without being able to change it. 578 [I-D.draft-gross-leap-second] presents some guidelines on how RTP 579 senders and receivers should deal with leap seconds. When relying on 580 NTP for clock synchronization, IDMS is particularly sensitive to leap 581 second induced timing discrepancies. It is therefore recommended to 582 take the guideline specified in [I-D.draft-gross-leap-second] into 583 account when implementing IDMS. 585 9. SDP Parameter for RTCP XR IDMS Block Type 587 The SDP parameter sync-group is used to signal the use of the RTCP XR 588 block for IDMS. It is also used to carry an identifier of the 589 synchronization group to which clients belong or will belong. This 590 SDP parameter extends rtcp-xr-attrib as follows, using Augmented 591 Backus-Naur Form [RFC5234]. 593 rtcp-xr-attrib = "a=" "rtcp-xr" ":" [xr-format *(SP xr-format)] CRLF 594 ; Original definition from [RFC3611], section 5.1 596 xr-format =/ grp-sync ; Extending xr-format for Inter-Destination 597 Media Synchronization 599 grp-sync = "grp-sync" [",sync-group=" SyncGroupId] 600 SyncGroupId = 1*DIGIT ; Numerical value from 0 through 4294967294 602 DIGIT = %x30-39 604 SyncGroupId is a 32-bit unsigned integer represented in decimal. 605 SyncGroupId identifies a group of SCs for IDMS. It maps on the Media 606 Stream Correlation Identifier as described in Section 6 and 607 Section 7. The value SyncGroupId=0 represents an empty SyncGroupId. 608 The value 4294967294 (2^32-1) is reserved for future use. 610 The following is an example of the SDP attribute for IDMS 612 a=rtcp-xr:grp-sync,sync-group=42 614 10. SDP Parameter for RTCP IDMS Packet Type 616 The SDP parameter rtcp-idms is used to signal the use of the RTCP 617 IDMS Packet Type for IDMS. It is also used to carry an identifier of 618 the synchronization group to which clients belong or will belong. 619 The SDP parameter is used as a media-level attribute during session 620 setup. This SDP parameter is defined as follows, using Augmented 621 Backus-Naur Form [RFC5234]. 623 rtcp-idms = "a=" "rtcp-idms" ":" [sync-grp] CRLF 625 sync-grp = "sync-group=" SyncGroupId 627 SyncGroupId = 1*DIGIT ; Numerical value from 0 through 4294967294 629 DIGIT = %x30-39 631 SyncGroupId is a 32-bit unsigned integer and represented in decimal. 632 SyncGroupId identifies a group of SCs for IDMS. The value 633 SyncGroupId=0 represents an empty SyncGroupId. The value 4294967294 634 (2^32-1) is reserved for future use. 636 The following is an example of the SDP attribute for IDMS. 638 a=rtcp-idms:sync-group=42 640 11. Compatibility with ETSI TISPAN 642 The SDP usage for IDMS follows the rules defined in RFC3611 in 643 section 5 on SDP signalling, with the exception of what is stated 644 here. The IDMS usage of RTCP is a (loosely coupled) collaborative 645 parameter, in the sense that receivers sent their status information 646 and in response (asynchronously) the MSAS sents synchronization 647 instructions. Both the sync-group parameter (defined by ETSI TISPAN) 648 and the rtcp-idms parameter (defined in this document) thus indicate 649 the ability to sent and the ability to receive indicated RTCP 650 messages. This section defines how these SDP parameters should be 651 used, and thus also explains how compatibility with the TISPAN 652 solution is arranged for. 654 As described in Section 1.4, ETSI TISPAN has described its mechanism 655 for IDMS in [TS183063]. One of the main differences between the 656 TISPAN document and this document is the fact that the TISPAN 657 solution uses an RTCP XR block for both the SC->MSAS message and the 658 MSAS->SC message (by selecting SPST-type 2), while this document 659 specifies a new RTCP Packet Type for the MSAS->SC message. The 660 message from MSAS to SC is not in any way a report on how a receiver 661 sees a session, and therefore a separate RTCP packet type is more 662 appropriate than the XR block solution chosen in ETSI TISPAN. To 663 achieve compatibility, MSAS implementations SHOULD implement both the 664 TISPAN RTCP block and the new RTCP IDMS report for MSAS->SC messages. 665 SCs MAY implement support for both types of messages. For the 666 MSAS->SC signaling, it is recommended to use the RTCP IDMS report 667 defined in this document. The TISPAN RTCP XR block with SPST=2 MAY 668 be used for purposes of compatibility with the TISPAN solution, but 669 MUST NOT be used if all nodes involved support the new RTCP IDMS 670 report. 672 Most of the times, the IDMS SDP parameters will be used in the offer/ 673 answer context. Receivers will indicate in their SDP which RTCP 674 messages they support. 676 For a unicast situation, three situations are possible in offer/ 677 answer context: 679 - If a receiver indicates at least the rtcp-idms SDP parameter, the 680 MSAS SHOULD reply with only the rtcp-idms parameter and use only 681 the RTCP IDMS report for MSAS->SC communication 683 - If a receiver indicates only the sync-group SDP parameter, and the 684 MSAS also supports this, it SHOULD reply with only the sync-group 685 parameter and use only the RTCP XR block with SPST=2 for MSAS->SC 686 communication 688 - If a receiver indicates only the sync-group SDP parameter, and the 689 MSAS does not support this, the media sender MUST ignore the 690 parameter. This receiver will not become part of the 691 synchronization session 693 Note that it is possible that for a certain synchronization group, 694 that the MSAS sends RTCP IDMS packets to one receiver and RTCP XR 695 IDMS blocks with SPST=2 to another receiver. 697 In a multicast situation using the offer/answer context, it will work 698 a bit differently. The negotiation is the same as in the unicast 699 situation. But, the MSAS will multicast all RTCP messages to all 700 receivers. So: 702 - If all receivers support the RTCP IDMS report, the MSAS SHOULD 703 only sent the RTCP IDMS report for MSAS->SC messages 705 - If not all receivers support the RTCP IDMS report, but all 706 receivers support the TISPAN solution, the MSAS SHOULD only sent 707 the RTCP XR block with SPST=2 for MSAS->SC messages. 709 - If some receivers support only the RTCP IDMS report and other 710 receivers support only the TISPAN solution, the MSAS SHOULD sent 711 both the RTCP IDMS report and the RTCP XR block with SPST=2 for 712 MSAS->SC messages. This is less efficient, since the information 713 sent is duplicated, but this is the only way to include all 714 receivers in a synchronization session in this scenario. 716 In certain multicast situations, there is no offer/answer context. 717 In that case, the MSAS SHOULD use only the RTCP IDMS packet type and 718 thus use only the SDP parameter rtcp-idms. Receivers that do not 719 support the RTCP IDMS packet will just ignore both the SDP parameter 720 and the RTCP IDMS packets, and will thus not join the synchronization 721 session. For compatability with the TISPAN solution, the MSAS MAY 722 choose to use the RTCP XR IDMS block type instead, using the SDP 723 parameter sync-group. The media sender SHOULD NOT use both 724 parameters at the same time in this case of no offer/answer context. 726 12. On the use of presentation timestamps 728 A receiver can report on different timing events, i.e. on packet 729 arrival times and on play-out times. A receiver SHALL report on 730 arrival times and a receiver MAY report on play-out times. RTP 731 packet arrival times are relatively easy to report on. Normally, the 732 processing and play-out of the same media stream by different 733 receivers will take roughly the same amount of time. Synchronizing 734 on packet arrival times, may lead to some accuracy loss, but it will 735 be adequate for many applications, such as social TV. 737 Also, if the receivers are in some way controlled, e.g. having the 738 same buffer settings and decoding times, high accuracy can be 739 achieved. However, if all receivers in a synchronization session 740 have the ability to report on, and thus synchronize on, actual play- 741 out times, or packet presentation times, this may be more accurate. 742 It is up to applications and implementations of this RTCP extension 743 whether to implement and use this. 745 13. Security Considerations 747 The security considerations described in [RFC3611] apply to this 748 document as well. 750 The specified RTCP XR Block Type in this document is used to collect, 751 summarize and distribute information on packet reception- and play- 752 out-times of streaming media. The information may be used to 753 orchestrate the media play-out at multiple devices. 755 Errors in the information, either accidental or malicious, may lead 756 to undesired behavior. For example, if one device erroneously 757 reports a two-hour delayed play-out, then another device in the same 758 synchronization group could decide to delay its play-out by two hours 759 as well, in order to keep its play-out synchronized. A user would 760 likely interpret this two hour delay as a malfunctioning service. 762 Therefore, the application logic of both Synchronization Clients and 763 Media Synchronization Application Servers should check for 764 inconsistent information. Differences in play-out time exceeding 765 configured limits (e.g. more than ten seconds) could be an indication 766 of such inconsistent information. 768 No new mechanisms are introduced in this document to ensure 769 confidentiality. Encryption procedures, such as those being 770 suggested for a Secure RTP (SRTP) at the time that this document was 771 written, can be used when confidentiality is a concern to end hosts. 773 14. IANA Considerations 775 This document defines a new RTCP packet type called IDMS report in 776 the IANA registry of RTP parameters, based on the specification in 777 Section 10. 779 Further, this document defines a new SDP parameter "rtcp-idms" within 780 the existing IANA registry of SDP Parameters. 782 The SDP attribute "rtcp-idms" defined by this document is registered 783 with the IANA registry of SDP Parameters as follows: 785 SDP Attribute ("att-field"): 787 Attribute name: rtcp-idms 789 Long form: RTCP report block for IDMS 791 Type of name: att-field 793 Type of attribute: media level 795 Subject to charset: no 797 Purpose: see sections 7 and 10 of this document 799 Reference: this document 801 Values: see this document 803 15. Contributors 805 The following people have participated as co-authors or provided 806 substantial contributions to this document: Omar Niamut, Fabian 807 Walraven, Ishan Vaishnavi, Rufael Mekuria and Rob Koenen. 809 16. References 811 16.1. Normative References 813 [I-D.draft-williams-avtcore-clksrc] 814 Williams, A., van Brandenburg, R., Stokking, H., and K. 815 Gross, "RTP Clock Source Signalling, 816 draft-williams-avtcore-clksrc-00", March 2012. 818 [RFC2119] Bradner, S., "Key Words for use in RFCs to Indicate 819 Requirement Levels, RFC 2119", March 1997. 821 [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. 822 Jacobson, "RTP: A Transport Protocol for Real-Time 823 Applications, RFC3550", July 2003. 825 [RFC3551] Schulzrinne, H. and S. Casner, "RTP Profile for Audio and 826 Video conferences with Minimal Control, RFC3551", 827 July 2003. 829 [RFC3611] Friedman, T., Ed., Caceres, R., Ed., and A. Clark, Ed., 830 "RTP Control Protocol Extended Reports (RTCP XR), 831 RFC3611", November 2003. 833 [RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session 834 Description Protocol, RFC4566", July 2006. 836 [RFC5234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax 837 Specifications, RFC5234", January 2008. 839 [RFC5760] Ott, J., Chesterfield, J., and E. Schooler, "RTP Control 840 Protocol (RTCP) Extensions for Single-Source Multicast 841 Sessions with Unicast Feedback, RFC5760", February 2010. 843 [RFC5905] Mills, D., Martin, J., Ed., Burbank, J., and W. Kasch, 844 "Network Time Protocol Version 4: Protocol and Algorithms 845 Specifications, RFC5905", February 2010. 847 [TS183063] 848 "IMS-based IPTV stage 3 specification, TS 183 063 v3.4.1", 849 June 2010. 851 16.2. Informative References 853 [Boronat2009] 854 Boronat, F., Lloret, J., and M. Garcia, "Multimedia group 855 and inter-stream synchronization techniques: a comparative 856 study, Elsevier Information Systems 34 (2009), pp. 108- 857 131". 859 [I-D.draft-gross-leap-second] 860 Gross, K. and R. Brandenburg, van, "RTP and Leap Seconds, 861 draft-gross-leap-seconds-01". 863 [IEEE-1588] 864 "1588-2008 - IEEE Standard for a Precision Clock 865 Synchronization Protocol for Networked Measurement and 866 Control Systems", 2008. 868 [Ishibashi2006] 869 Ishibashi, Y., Nagasaka, M., and N. Fujiyoshi, "Subjective 870 Assessment of Fairness among users in multipoint 871 communications, Proceedings of the 2006 ACM SIGCHI 872 internation conference on Advances in computer 873 entertainment technology, 2006". 875 [RFC5868] Ott, J. and C. Perkins, "Guidelines for Extending the RTP 876 Control Protocol (RTCP), RFC5968", September 2010. 878 Authors' Addresses 880 Ray van Brandenburg 881 TNO 882 Brassersplein 2 883 Delft 2612CT 884 the Netherlands 886 Phone: +31-88-866-7000 887 Email: ray.vanbrandenburg@tno.nl 889 Hans Stokking 890 TNO 891 Brassersplein 2 892 Delft 2612CT 893 the Netherlands 895 Phone: +31-88-866-7000 896 Email: hans.stokking@tno.nl 898 M. Oskar van Deventer 899 TNO 900 Brassersplein 2 901 Delft 2612CT 902 the Netherlands 904 Phone: +31-88-866-7000 905 Email: oskar.vandeventer@tno.nl 907 Fernando Boronat 908 Universitat Politecnica de Valencia 909 IGIC Institute, Universitat Politecnica de Valencia-Campus de Gandia (UPV), C/ Paraninfo, 1, Grao de Gandia, C/ Paraninfo, 1, Grao de Gandia 910 Valencia 46730 911 Spain 913 Phone: +34 962 849 341 914 Email: fboronat@dcom.upv.es 915 Mario Montagud 916 Universitat Politecnica de Valencia 917 IGIC Institute, Universitat Politecnica de Valencia-Campus de Gandia (UPV), C/ Paraninfo, 1, Grao de Gandia, C/ Paraninfo, 1, Grao de Gandia 918 Valencia 46730 919 Spain 921 Phone: +34 962 849 341 922 Email: mamontor@posgrado.upv.es 924 Kevin Gross 925 AVA Networks 927 Phone: +1-303-447-0517 928 Email: Kevin.Gross@AVAnw.com