idnits 2.17.1 draft-ietf-avtcore-idms-06.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 3 instances of too long lines in the document, the longest one being 71 characters in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 16, 2012) is 4302 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Outdated reference: A later version (-01) exists of draft-williams-avtcore-clksrc-00 ** Obsolete normative reference: RFC 4566 (Obsoleted by RFC 8866) -- Possible downref: Non-RFC (?) normative reference: ref. 'TS183063' -- No information found for draft-gross-leap-seconds - is the name correct? Summary: 2 errors (**), 0 flaws (~~), 2 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 AVTCore R. van Brandenburg 3 Internet-Draft H. Stokking 4 Intended status: Standards Track O. van Deventer 5 Expires: January 17, 2013 TNO 6 F. Boronat 7 M. Montagud 8 Universitat Politecnica de 9 Valencia 10 K. Gross 11 AVA Networks 12 July 16, 2012 14 Inter-destination Media Synchronization using the RTP Control Protocol 15 (RTCP) 16 draft-ietf-avtcore-idms-06 18 Abstract 20 This document gives information on an RTP Control Protocol (RTCP) 21 Packet Type and RTCP Extended Report (XR) Block Type including 22 associated Session Description Protocol (SDP) parameters for Inter- 23 Destination Media Synchronization (IDMS). This document mandates the 24 use of the RTCP XR Block Type 12, which is used to collect media 25 playout information from participants in a group playing out 26 (watching, listening, etc.) a specific RTP media stream. This RTCP 27 XR Block Type is specified and registered with IANA by ETSI. The 28 RTCP packet type specified by this document is used to distribute a 29 common target playout point to which all the distributed receivers, 30 sharing a media experience, can synchronize. 32 Typical use cases in which IDMS is usefull are social TV, shared 33 service control (i.e. applications where two or more geographically 34 separated users are watching a media stream together), distance 35 learning, networked video walls, networked loudspeakers, etc. 37 Status of this Memo 39 This Internet-Draft is submitted in full conformance with the 40 provisions of BCP 78 and BCP 79. 42 Internet-Drafts are working documents of the Internet Engineering 43 Task Force (IETF). Note that other groups may also distribute 44 working documents as Internet-Drafts. The list of current Internet- 45 Drafts is at http://datatracker.ietf.org/drafts/current/. 47 Internet-Drafts are draft documents valid for a maximum of six months 48 and may be updated, replaced, or obsoleted by other documents at any 49 time. It is inappropriate to use Internet-Drafts as reference 50 material or to cite them other than as "work in progress." 52 This Internet-Draft will expire on January 17, 2013. 54 Copyright Notice 56 Copyright (c) 2012 IETF Trust and the persons identified as the 57 document authors. All rights reserved. 59 This document is subject to BCP 78 and the IETF Trust's Legal 60 Provisions Relating to IETF Documents 61 (http://trustee.ietf.org/license-info) in effect on the date of 62 publication of this document. Please review these documents 63 carefully, as they describe your rights and restrictions with respect 64 to this document. Code Components extracted from this document must 65 include Simplified BSD License text as described in Section 4.e of 66 the Trust Legal Provisions and are provided without warranty as 67 described in the Simplified BSD License. 69 Table of Contents 71 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 72 2. Rationale . . . . . . . . . . . . . . . . . . . . . . . . . . 4 73 2.1. Applicability of RTCP to IDMS . . . . . . . . . . . . . . 4 74 2.2. Applicability of SDP to IDMS . . . . . . . . . . . . . . . 5 75 2.3. This document and ETSI TISPAN . . . . . . . . . . . . . . 5 76 3. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 77 4. Overview of IDMS operation . . . . . . . . . . . . . . . . . . 5 78 5. Inter-Destination Media Synchronization use cases . . . . . . 7 79 6. Architecture for Inter-Destination Media Synchronization . . . 8 80 6.1. Media Synchronization Application Server (MSAS) . . . . . 9 81 6.2. Synchronization Client (SC) . . . . . . . . . . . . . . . 9 82 6.3. Communication between MSAS and SCs . . . . . . . . . . . . 9 83 7. RTCP XR Block Type for IDMS . . . . . . . . . . . . . . . . . 9 84 8. RTCP Packet Type for IDMS (IDMS Settings) . . . . . . . . . . 12 85 9. Timing and NTP Considerations . . . . . . . . . . . . . . . . 14 86 10. SDP Parameter for RTCP XR IDMS Block Type . . . . . . . . . . 15 87 11. SDP Parameter for RTCP IDMS Packet Type . . . . . . . . . . . 16 88 12. Compatibility with ETSI TISPAN . . . . . . . . . . . . . . . . 17 89 13. SDP rules . . . . . . . . . . . . . . . . . . . . . . . . . . 17 90 13.1. Offer/Answer rules . . . . . . . . . . . . . . . . . . . . 17 91 13.2. Declarative cases . . . . . . . . . . . . . . . . . . . . 19 92 14. On the use of presentation timestamps . . . . . . . . . . . . 19 93 15. Security Considerations . . . . . . . . . . . . . . . . . . . 19 94 16. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20 95 17. Contributors . . . . . . . . . . . . . . . . . . . . . . . . . 21 96 18. References . . . . . . . . . . . . . . . . . . . . . . . . . . 21 97 18.1. Normative References . . . . . . . . . . . . . . . . . . . 21 98 18.2. Informative References . . . . . . . . . . . . . . . . . . 22 99 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 22 101 1. Introduction 103 Inter-Destination Media Synchronization (IDMS) refers to the playout 104 of media streams at two or more geographically distributed locations 105 in a time synchronized manner. It can be applied to both unicast and 106 multicast media streams and can be applied to any type and/or 107 combination of streaming media, such as audio, video and text 108 (subtitles).[Ishibashi2006] and [Boronat2009] provide an overview of 109 technologies and algorithms for IDMS. 111 IDMS requires the exchange of information on media receipt and 112 playout times among participants in an IDMS session. It may also 113 require signaling for the initiation and maintenance of IDMS sessions 114 and groups of receivers. 116 The presented RTCP specification for IDMS is independent of the used 117 synchronization algorithm, which is out-of-scope of this document. 119 2. Rationale 121 2.1. Applicability of RTCP to IDMS 123 Currently, a large share of real-time applications make use of RTP 124 and RTCP [RFC3550]. RTP provides end-to-end network transport 125 functions suitable for applications requiring real-time data 126 transport, such as audio, video or data, over multicast or unicast 127 network services. The timestamps, sequence numbers, and payload 128 (content) type identification mechanisms provided by RTP packets are 129 very useful for reconstructing the original media timing, and for 130 reordering and detecting packet loss at the client side. 132 The data transport is augmented by a control protocol (RTCP) to allow 133 monitoring of the data delivery in a manner that is scalable to large 134 groups, and to provide minimal control and identification 135 functionality. RTP receivers and senders provide reception quality 136 feedback by sending out RTCP Receiver Report (RR) and Sender Report 137 (SR) packets [RFC3550], respectively, which may be augmented by 138 eXtended Reports (XR) [RFC3611]. Both RTP and RTCP are intended to 139 be tailored through modifications in order to include profile- 140 specific information required by particular applications, and the 141 guidelines on doing so are specified in [RFC5868]. 143 IDMS involves the collection, summarizing and distribution of RTP 144 packet arrival and playout times. As information on RTP packet 145 arrival times and playout times can be considered reception quality 146 feedback information, RTCP is well suited for carrying out IDMS, 147 which may facilitate the implementation and deployment in typical 148 multimedia applications. 150 2.2. Applicability of SDP to IDMS 152 RTCP XR [RFC3611] defines the Extended Report (XR) packet type for 153 the RTP Control Protocol (RTCP), and defines how the use of XR 154 packets can be signaled by an application using the Session 155 Description Protocol (SDP) [RFC4566]. 157 SDP signaling is used to set up and maintain a synchronization group 158 between Synchronization Clients (SCs). This document describes two 159 SDP parameters for doing this, one for the RTCP XR block type and one 160 for the new RTCP packet type. 162 2.3. This document and ETSI TISPAN 164 ETSI TISPAN [TS183063] has specified architecture and protocol for 165 IDMS using RTCP XR exchange and SDP signaling. For more information 166 on how this document relates to [TS183063], see Section 12. 168 This document mandates the use of an ETSI specified RTCP XR block for 169 the purpose of collecting RTP packet arrival and playout times. For 170 completeness, that XR block is contained in this document in section 171 7. 173 3. Terminology 175 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 176 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 177 document are to be interpreted as described in RFC 2119 [RFC2119] and 178 indicate requirement levels for compliant implementations. 180 4. Overview of IDMS operation 182 This section provides a brief example of how the RTCP functionality 183 is used for achieving IDMS. The section is tutorial in nature and 184 does not contain any normative statements. 186 Alice's . . . . . . .tv:abc.com . . . . . . . . . Bob's 187 TV (Sync Client) (Sync Server) Laptop (Sync Client) 188 | | | 189 | Media Session | | 190 |<=====================>| | 191 | Invite(URL,Sync-group ID) | 192 |------------------------------------------------->| 193 | | Media Session Set-up | 194 | |<========================>| 195 | | | 196 | Call set-up | 197 |<================================================>| 198 | | | 199 | RTP Packet | RTP Packet | 200 |<----------------------|------------------------->| 201 | RR + IDMS XR | | 202 |---------------------->| RR + IDMS XR | 203 | |<-------------------------| 204 | RTCP IDMS packet | RTCP IDMS packet | 205 |<----------------------|------------------------->| 206 | | | 208 Figure 1: Example of a typical IDMS session 210 Alice is watching TV in her living room. At some point she sees that 211 a football game of Bob's favorite team is on. She sends him an 212 invite to watch the program together. Embedded in the invitation is 213 the link to the media server and a unique sync-group identifier. 215 Bob, who is also at home, receives the invite on his laptop. He 216 accepts Alice's invitation and the RTP client on his laptop sets up a 217 session to the media server. A VoIP connection to Alice's TV is also 218 set up, so that Alice and Bob can talk while watching the game 219 together. 221 As is common with RTP, both the RTP client in Alice's TV as well as 222 the one in Bob's laptop send periodic RTCP Receiver Reports (RR) to 223 the media server. However, in order to make sure Alice and Bob see 224 the events in the football game at (approximately) the same time, 225 their clients also periodically send an IDMS XR block to the sync 226 server function of the media server. Included in the XR blocks are 227 timestamps on when both Alice and Bob received (and, optionally, when 228 they played out) a particular RTP packet. 230 The sync server function in the media server calculates a reference 231 client from the received IDMS XR blocks (e.g. by selecting whichever 232 client received the packet the latest as the reference client). It 233 then sends an RTCP IDMS packet containing the playout information of 234 this reference client to the sync clients of both Alice and Bob. 236 In this case Bob's connection has the longest delay and the reference 237 client therefore includes a delay similar to the one experienced by 238 Bob. Upon reception of this information, Alice's RTP client can 239 choose what to do with this information. In this case it decreases 240 its playout rate temporarily until the playout time matches with the 241 reference client playout (and thus matches Bob's playout). Another 242 option for Alice's TV would be to simply pause playback until it 243 catches up. The exact implementation of the synchronization 244 algorithm is up to the client. 246 Upon reception of the reference client RTCP IDMS packet, Bob's client 247 does not have to do anything since it is already synchronized to the 248 reference client (since it is based on Bob's delay). Note that other 249 synchronization algorithms may introduce even more delay than the one 250 experienced by the most delayed client, e.g. to account for delay 251 variations, for new clients joining an existing synchronization 252 group, etc. 254 For this functionality to work correctly, it is nessecary that the 255 wall clocks of the receivers are synchronized with each other. Alice 256 and Bob both report when they receive, and optionally when they play 257 out, certain RTP packets. In order to correlate their reports to 258 each other, it is necessary that their wallclocks are synchronized. 260 5. Inter-Destination Media Synchronization use cases 262 There are a large number of use cases in which IDMS might be useful. 263 This section will highlight some of them. It should be noted that 264 this section is in no way meant to be exhaustive. 266 A first usage scenario for IDMS is Social TV. Social TV is the 267 combination of media content consumption by two or more users at 268 different devices and locations combined with the real-time 269 communication between those users. An example of Social TV is when 270 two or more users are watching the same television broadcast at 271 different devices and locations, while communicating with each other 272 using text, audio and/or video. A skew in their media playout 273 processes can have adverse effects on their experience. A well-known 274 use case here is one friend experiencing a goal in a football match 275 well before or after other friend(s). 277 Another potential use case for IDMS is a networked video wall. A 278 video wall consists of multiple computer monitors, video projectors, 279 or television sets tiled together contiguously or overlapped in order 280 to form one large screen. Each of the screens reproduces a portion 281 of the larger picture. In some implementations, each screen may be 282 individually connected to the network and receive its portion of the 283 overall image from a network-connected video server or video scaler. 284 Screens are refreshed at 60 hertz (every 16-2/3 milliseconds) or 285 potentially faster. If the refresh is not synchronized, the effect 286 of multiple screens acting as one is broken. 288 A third usage scenario is that of the networked loudspeakers, in 289 which two or more speakers are connected to the network individually. 290 Such situations can for example be found in large conference rooms, 291 legislative chambers, classrooms (especially those supporting 292 distance learning) and other large-scale environments such as 293 stadiums. Since humans are more susceptible to differences in audio 294 delay, this use case needs even more accuracy than the video wall use 295 case. Depending on the exact application, the need for accuracy can 296 then be in the range of microseconds. 298 6. Architecture for Inter-Destination Media Synchronization 300 The architecture for IDMS, which is based on a sync-maestro 301 architecture [Boronat2009], is sketched below. The Synchronization 302 Client (SC) and Media Synchronization Application Server (MSAS) 303 entities are shown as additional functionality for the RTP receiver 304 and sender respectively. 306 It should be noted that a master/slave type of architecture is also 307 supported by having one of the SC devices also act as an MSAS. In 308 this case the MSAS functionality is thus embedded in an RTP receiver 309 instead of an RTP sender. 311 +-----------------------+ +-----------------------+ 312 | | SR + | | 313 | RTP Receiver | RTCP | RTP Sender | 314 | | IDMS | | 315 | +-----------------+ | <----- | +-----------------+ | 316 | | | | | | | | 317 | | Synchronization | | | | Media | | 318 | | Client | | | | Synchronization | | 319 | | (SC) | | | | Application | | 320 | | | | | | Server | | 321 | | | | RR+XR | | (MSAS) | | 322 | | | | -----> | | | | 323 | +-----------------+ | | +-----------------+ | 324 | | | | 325 +-----------------------+ +-----------------------+ 327 6.1. Media Synchronization Application Server (MSAS) 329 An MSAS collects RTP packet arrival times and playout times from one 330 or more SC(s) in a synchronization group. The MSAS summarizes and 331 distributes this information to the SCs in the synchronization group 332 as synchronization settings, e.g. by determining the SC with the most 333 lagged playout and using its reported RTP packet arrival time and 334 playout time as a summary. 336 6.2. Synchronization Client (SC) 338 An SC reports on RTP packet arrival times and playout times of a 339 media stream. It can receive summaries of such information, and use 340 that to adjust its playout buffer. 342 6.3. Communication between MSAS and SCs 344 Two different message types are used for the communication between 345 MSAS and SCs. For the SC->MSAS message containing the playout 346 information of a particular client, an RTCP XR Block Type is used 347 (see Section 6). For the MSAS->SC message containing the 348 synchronization settings instructions, a new RTCP Packet Type is 349 defined (see Section 8). 351 7. RTCP XR Block Type for IDMS 353 For reporting IDMS information, SCs SHALL use the ETSI specified RTCP 354 XR Block Type 12. For completeness, that RTCP XR Block Type is 355 repeated here fully. 357 The definition of the XR block type is based on [RFC3611]. The RTCP 358 XR is used to provide feedback information on receipt times and 359 presentation times of RTP packets to e.g. a Sender [RFC3611], a 360 Feedback Target [RFC5760] or a Third Party Monitor [RFC3611]. 362 In most cases, a single RTP receiver will only be part of single IDMS 363 session, i.e. it will report on receipt and presentation times of RTP 364 packets from a single RTP stream in a certain synchronization group. 365 In some cases however, an RTP receiver may be a member of multiple 366 synchronization groups for the same RTP stream, e.g. watching a 367 single television program simultaneously with different groups. In 368 even further cases, a receiver may wish to synchronize different RTP 369 streams at the same time, either as part of the same synchronization 370 group or as part of multiple synchronization groups. These are all 371 valid scenario's for IDMS, and will require multiple reports by an 372 SC. 374 SCs SHOULD report on a recently received RTP packets. This document 375 does not define new rules on when to sent RTCP reports, but uses the 376 existing rules specified in [RFC3550] for sending RTCP reports. When 377 the RTCP reporting timer allows an SC to send an IDMS report, the SC 378 SHOULD report on the latest RTP packet received or played out 379 (depending on whether the SC reports on presentation timestamps or 380 receipt timestamps). For more details on which packet to report on, 381 see below under 'Packet Received RTP timestamp'. 383 0 1 2 3 384 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 385 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 386 |V=2|P| Resrv | PT=XR=207 | length | 387 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 388 | SSRC of packet sender | 389 +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ 390 | BT=12 | SPST |Resrv|P| block length=7 | 391 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 392 | PT | Resrv | 393 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 394 | Media Stream Correlation Identifier | 395 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 396 | SSRC of media source | 397 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 398 | Packet Received NTP timestamp, most significant word | 399 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 400 | Packet Received NTP timestamp, least significant word | 401 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 402 | Packet Received RTP timestamp | 403 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 404 | Packet Presented NTP timestamp | 405 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 407 The first 64 bits form the header of the RTCP XR, as defined in 408 [RFC3611]. The SSRC of packet sender identifies the sender of the 409 specific RTCP packet. 411 The IDMS report block consists of 8 32-bit words, with the following 412 fields: 414 Block Type (BT): 8 bits. It identifies the block format. Its value 415 SHALL be set to 12. 417 Synchronization Packet Sender Type (SPST): 4 bits. This field 418 identifies the role of the packet sender for this specific eXtended 419 Report. It can have the following values: 421 SPST=0 Reserved For future use. 423 SPST=1 The packet sender is an SC. It uses this XR to report 424 synchronization status information. Timestamps relate to the SC 425 input. 427 SPST=2 This setting is reserved in order to preserve compatibility 428 with ETSI TISPAN [TS183063]. See Section 12 for more information. 430 SPST=3-15 Reserved For future use. 432 Reserved bits (Resrv): 3 bits. These bits are reserved for future 433 definition. In the absence of such a definition, the bits in this 434 field MUST be set to zero and MUST be ignored by the receiver. 436 Packet Presented NTP timestamp flag (P): 1 bit. Bit set to 1 if the 437 Packet Presented NTP timestamp field contains a value, 0 if it is 438 empty. If this flag is set to zero, then the Packet Presented NTP 439 timestamp SHALL be ignored. 441 Block Length: 16 bits. This field indicates the length of the block 442 in 32 bit words minus one and SHALL be set to 7, as this RTCP Block 443 Type has a fixed length. 445 Payload Type (PT): 7 bits. This field identifies the format of the 446 media payload, according to [RFC3551]. This is the payload type of 447 the RTP packet reported upon. The media payload is associated with 448 an RTP timestamp clock rate. This clock rate provides the time base 449 for the RTP timestamp counter.This clock rate is nessecary for the 450 MSAS to relate reports from different SCs on different RTP timestamp 451 values. 453 Reserved bits (Resrv): 25 bits. These bits are reserved for future 454 use and SHALL be set to 0. 456 Media Stream Correlation Identifier: 32 bits. This identifier is 457 used to correlate synchronized media streams. The value 0 (all bits 458 are set "0") indicates that this field is empty. The value 2^32-1 459 (all bits are set "1") is reserved for future use. If the RTCP 460 Packet Sender is an SC (SPST=1) or an MSAS (SPST=2), then the Media 461 Stream Correlation Identifier field contains the Synchronization 462 Group Identifier (SyncGroupId) to which the report applies. 464 SSRC: 32 bits. The SSRC of the media source SHALL be set to the 465 value of the SSRC identifier carried in the RTP header [RFC3550] of 466 the RTP packet to which the XR relates. 468 Packet Received NTP timestamp: 64 bits. This timestamp reflects the 469 wall clock time at the moment of arrival of the first octet of the 470 RTP packet to which the XR relates. It is formatted based on the NTP 471 timestamp format as specified in [RFC5905]. See Section 9 for more 472 information on how this field is used. 474 Packet Received RTP timestamp: 32 bits. This timestamp has the value 475 of the RTP timestamp carried in the RTP header [RFC3550] of the RTP 476 packet to which the XR relates. Several consecutive RTP packets will 477 have equal timestamps if they are (logically) generated at once, 478 e.g., belong to the same video frame. It may well be the case that 479 one receiver reports on the first RTP packet having a certain RTP 480 timestamp and a second receiver reports on the last RTP packet having 481 that same RTP timestamp. This would lead to an error in the 482 synchronization algorithm due to the faulty interpretation of 483 considering both reports to be on the same RTP packet. When 484 reporting on an RTP packet which is one of several consecutive RTP 485 packets having equal timestamps, an SC SHOULD report on the RTP 486 packet it received with the lowest sequence number. Note that with 487 'lowest sequence number' here is meant the first in the sequence of 488 RTP packets just received, not from an earlier time before the last 489 wrap-around of RTP timestamps (unless this wrap-around occurs during 490 the sequence with equal RTP timestamps). 492 Packet Presented NTP timestamp: 32 bits. This timestamp reflects the 493 wall clock time at the moment the rendered frame contained in the 494 first byte of the associated RTP packet is presented to the user. It 495 is based on the time format used by NTP and consists of the least 496 significant 16 bits of the NTP seconds part and the most significant 497 16 bits of the NTP fractional second part. If this field is empty, 498 then it SHALL be set to 0 and the Packet Presented NTP timestamp flag 499 (P) SHALL be set to 0. Presented here means the moment the data is 500 played out to the user of the system, i.e. sound played out through 501 speakers, video images being displayed on some display, etc. The 502 accuracy resulting from the synchronization algorithm will only be as 503 good as the accuracy with which the receivers can determine the delay 504 between receiving packets and presenting them to the end-user. 506 8. RTCP Packet Type for IDMS (IDMS Settings) 508 This section specifies the RTCP Packet Type for indicating 509 synchronization settings instructions to the receivers of the RTP 510 media stream. Its definition is based on [RFC3550]. 512 0 1 2 3 513 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 514 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 515 |V=2|P| Resrv | PT=TBD | length | 516 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 517 | SSRC of packet sender | 518 +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ 519 | SSRC of media source | 520 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 521 | Media Stream Correlation Identifier | 522 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 523 | Packet Received NTP timestamp, most significant word | 524 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 525 | Packet Received NTP timestamp, least significant word | 526 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 527 | Packet Received RTP timestamp | 528 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 529 | Packet Presented NTP timestamp, most significant word | 530 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 531 | Packet Presented NTP timestamp, least significant word | 532 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 534 The first 64 bits form the header of the RTCP Packet Type, as defined 535 in [RFC3550]. The SSRC of packet sender identifies the sender of the 536 specific RTCP packet. 538 The RTCP IDMS packet consists of 7 32-bit words, with the following 539 fields: 541 SSRC: 32 bits. The SSRC of the media source SHALL be set to the 542 value of the SSRC identifier of the media source carried in the RTP 543 header [RFC3550] of the RTP packet to which the RTCP IDMS packet 544 relates. 546 Media Stream Correlation Identifier: 32 bits. This identifier is 547 used to correlate synchronized media streams. The value 0 (all bits 548 are set "0") indicates that this field is empty. The value 2^32-1 549 (all bits are set "1") is reserved for future use. The Media Stream 550 Correlation Identifier contains the SyncGroupId of the group to which 551 this packet is sent. 553 Packet Received NTP timestamp: 64 bits. This timestamp reflects the 554 wall clock time at the reference client at the moment it received the 555 first octet of the RTP packet to which this packet relates. It can 556 be used by the synchronization algorithm on the receiving SC to 557 adjust its playout timing in order to achieve synchronization, e.g. 558 to set the required playout delay. The timestamp is formatted based 559 on the NTP timestamp format as specified in [RFC5905]. See Section 9 560 for more information on how this field is used. Because RTP 561 timestamps do wrap around, the sender of this packet SHOULD use 562 recent values, i.e. choose NTP timestamps that reflect current time 563 and not too far in the future or in the past. 565 Packet Received RTP timestamp: 32 bits. This timestamp has the value 566 of the RTP timestamp carried in the RTP header [RFC3550] of the RTP 567 packet to which the XR relates. This SHOULD relate to the first 568 arriving RTP packet containing this particular RTP timestamp, in case 569 multiple RTP packets contain the same RTP timestamp. 571 Packet Presented NTP timestamp: 64 bits. This timestamp reflects the 572 wall clock time at the reference client at the moment it presented 573 the rendered frame contained in the first octet of the associated RTP 574 packet to the user. The timestamp is formatted based on the NTP 575 timestamp format as specified in [RFC5905]. If this field is empty, 576 then it SHALL be set to 0. This field MAY be left empty if none or 577 only one of the receivers reported on presentation timestamps. 578 Presented here means the moment the data is played out to the user of 579 the system. 581 In some use cases (e.g. phased array transducers), the level of 582 control an MSAS might need to have over the exact moment of playout 583 is so precise that a 32bit Presented Timestamp will not suffice. For 584 this reason, this RTCP Packet Type for IDMS includes a 64bit 585 Presented Timestamp field. Since an MSAS will in practice always add 586 some extra delay to the delay reported by the most lagged receiver 587 (to account for packet jitter), it suffices for the IDMS XR Block 588 Type with which the SCs report on their playout to have a 32bit 589 Presented Timestamp field. 591 9. Timing and NTP Considerations 593 To achieve IDMS, the different receivers involved need synchronized 594 (wall) clocks as a common timeline for synchronization. This 595 synchronized clock is used for reporting the Packet Received NTP 596 Timestamps and the Packet Presented NTP Timestamp, and for 597 interpretation of these fields in received IDMS reports. Depending 598 on the synchronization accuracy required, different clock 599 synchronization methods can be used. For social TV, synchronization 600 accuracy should be achieved on the order of hundreds of milliseconds. 601 In that case, correct use of NTP on receivers will in most situations 602 achieve the required accuracy. As a guideline, to deal with clock 603 drift of receivers, receivers should synchronize their clocks at the 604 beginning of a synchronized session. In case of high required 605 accuracy, the synchronized clocks of different receivers should not 606 drift beyond the accuracy required for the synchronization mechanism. 608 In practice, this can mean that receivers need to synchronize their 609 clocks repeatedly during a synchronization session. 611 Because of the stringent synchronization requirements for achieving 612 good audio in some use cases, a high accuracy will be needed. In 613 this case, use of the global NTP system may not be sufficient. For 614 improved accuracy, a local NTP server could be set up, or some other 615 more accurate clock synchronization mechanism can be used, such as 616 GPS time or the Precision Time Protocol [IEEE-1588]. 618 [I-D.draft-williams-avtcore-clksrc] defines a set of SDP parameters 619 for signaling the clock synchronization source or sources available 620 to and used by the individual receivers. SCs MAY use this draft to 621 indicate their clock synchronization source or sourced in use and 622 available. Using these paramenters, an SC can indicate which 623 synchronization source is being used at the moment, the last time the 624 SC synchronized with this source and the synchronization frequency. 625 An SC can also indicate any other synchronization sources available 626 to it. This allows multiple SCs in an IDMS session to use the same 627 or a similar clock source for their session. 629 Applications performing IDMS may or may not be able to choose a 630 synchronization method for the system clock, because this may be a 631 system-wide setting which the application cannot change. How 632 applications deal with this is up to the implementation. The 633 application might control the system clock, or it might use a 634 separate application clock or even a separate IDMS session clock. It 635 might also report on the system clock and the synchronization method 636 used, without being able to change it. 638 [I-D.draft-gross-leap-second] presents some guidelines on how RTP 639 senders and receivers should deal with leap seconds. When relying on 640 NTP for clock synchronization, IDMS is particularly sensitive to leap 641 second induced timing discrepancies. It is RECOMMENDED to take the 642 guideline specified in [I-D.draft-gross-leap-second] into account 643 when implementing IDMS. 645 10. SDP Parameter for RTCP XR IDMS Block Type 647 The SDP parameter sync-group is used to signal the use of the RTCP XR 648 block for IDMS, as specified by ETSI, included here for completeness. 649 It is also used to carry an identifier of the synchronization group 650 to which clients belong or will belong. This SDP parameter extends 651 rtcp-xr-attrib as follows, using Augmented Backus-Naur Form 652 [RFC5234]. 654 rtcp-xr-attrib = "a=" "rtcp-xr" ":" [xr-format *(SP xr-format)] CRLF 655 ; Original definition from [RFC3611], section 5.1 657 xr-format =/ grp-sync ; Extending xr-format for Inter-Destination 658 Media Synchronization 660 grp-sync = "grp-sync" [",sync-group=" SyncGroupId] 662 SyncGroupId = 1*10DIGIT ; Numerical value from 0 through 4294967294 664 DIGIT = %x30-39 666 SyncGroupId is a 32-bit unsigned integer represented in decimal. 667 SyncGroupId identifies a group of SCs for IDMS. It maps on the Media 668 Stream Correlation Identifier as described in Section 7 and 669 Section 8. The value SyncGroupId=0 represents an empty SyncGroupId. 670 The value 4294967294 (2^32-1) is reserved for future use. 672 The following is an example of the SDP attribute for IDMS 674 a=rtcp-xr:grp-sync,sync-group=42 676 11. SDP Parameter for RTCP IDMS Packet Type 678 The SDP parameter rtcp-idms is used to signal the use of the RTCP 679 IDMS Packet Type for IDMS. It is also used to carry an identifier of 680 the synchronization group to which clients belong or will belong. 681 The SDP parameter is used as a media-level attribute during session 682 setup. This means that in case of multiple related streams, IDMS is 683 performed on one of them. The other streams will be synchronized to 684 this first stream using existing inter-stream synchronization (i.e. 685 lip-sync) solutions, i.e. using Sender Reports based on a common 686 clock source. Basic guidelines for choosing the media stream for 687 IDMS is to choose audio above video, as humans are more sensitive to 688 degradation in audio quality then in video quality. When using 689 mutli-description or multi-view codecs, the IDMS control should be 690 performed on the base layer. 692 This SDP parameter is defined as follows, using Augmented Backus-Naur 693 Form [RFC5234]. 695 rtcp-idms = "a=" "rtcp-idms" ":" [sync-grp] CRLF 697 sync-grp = "sync-group=" SyncGroupId 699 SyncGroupId = 1*10DIGIT ; Numerical value from 0 through 4294967294 701 DIGIT = %x30-39 702 SyncGroupId is a 32-bit unsigned integer and represented in decimal. 703 SyncGroupId identifies a group of SCs for IDMS. The value 704 SyncGroupId=0 represents an empty SyncGroupId. The value 4294967294 705 (2^32-1) is reserved for future use. 707 The following is an example of the SDP attribute for IDMS. 709 a=rtcp-idms:sync-group=42 711 12. Compatibility with ETSI TISPAN 713 As described in Section 2.3, ETSI TISPAN has described its mechanism 714 for IDMS in [TS183063]. One of the main differences between the 715 TISPAN document and this document is the fact that the TISPAN 716 solution uses an RTCP XR block for both the SC->MSAS message and the 717 MSAS->SC message (by selecting SPST-type 2), while this document 718 specifies a new RTCP Packet Type for the MSAS->SC message. The 719 message from MSAS to SC is not in any way a report on how a receiver 720 sees a session, and therefore a separate RTCP packet type is more 721 appropriate than the XR block solution chosen in ETSI TISPAN. To 722 achieve compatibility, MSAS implementations SHOULD implement both the 723 TISPAN RTCP block and the new RTCP IDMS Settings packet for MSAS->SC 724 messages. SCs MAY implement support for both types of messages. For 725 the MSAS->SC signaling, it is recommended to use the RTCP IDMS 726 Settings packet defined in this document. The TISPAN RTCP XR block 727 with SPST=2 MAY be used for purposes of compatibility with the TISPAN 728 solution, but MUST NOT be used if all nodes involved support the new 729 RTCP IDMS Settings packet. 731 13. SDP rules 733 13.1. Offer/Answer rules 735 The SDP usage for IDMS follows the rules defined in RFC3611 in 736 section 5 on SDP signalling, with the exception of what is stated 737 here. The IDMS usage of RTCP is a (loosely coupled) collaborative 738 parameter, in the sense that receivers sent their status information 739 and in response (asynchronously) the MSAS sents synchronization 740 instructions. Both the sync-group parameter (defined by ETSI TISPAN) 741 and the rtcp-idms parameter (defined in this document) thus indicate 742 the ability to sent and the ability to receive indicated RTCP 743 messages. This section defines how these SDP parameters should be 744 used. 746 Most of the times, the IDMS SDP parameters will be used in the offer/ 747 answer context. Receivers will indicate in their SDP which RTCP 748 messages they support. 750 For a unicast situation, three situations are possible in offer/ 751 answer context: 753 - If a receiver indicates at least the rtcp-idms SDP parameter, the 754 MSAS SHOULD reply with only the rtcp-idms parameter and use only 755 the RTCP IDMS Settings packet for MSAS->SC communication 757 - If a receiver indicates only the sync-group SDP parameter, and the 758 MSAS also supports this, it SHOULD reply with only the sync-group 759 parameter and use only the RTCP XR block with SPST=2 for MSAS->SC 760 communication 762 - If a receiver indicates only the sync-group SDP parameter, and the 763 MSAS does not support this, the media sender MUST ignore the 764 parameter. This receiver will not become part of the 765 synchronization session 767 Note that it is possible that for a certain synchronization group, 768 that the MSAS sends RTCP IDMS packets to one receiver and RTCP XR 769 IDMS blocks with SPST=2 to another receiver. 771 In a multicast situation using the offer/answer context, it will work 772 a bit differently. The negotiation is the same as in the unicast 773 situation. But, the MSAS will multicast all RTCP messages to all 774 receivers. So: 776 - If all receivers support the RTCP IDMS Settings packet, the MSAS 777 SHOULD only sent the RTCP IDMS Settings packet for MSAS->SC 778 messages 780 - If not all receivers support the RTCP IDMS Settings packet, but 781 all receivers support the TISPAN solution, the MSAS SHOULD only 782 sent the RTCP XR block with SPST=2 for MSAS->SC messages. 784 - If some receivers support only the RTCP IDMS Settings packet and 785 other receivers support only the TISPAN solution, the MSAS SHOULD 786 sent both the RTCP IDMS Settings packet and the RTCP XR block with 787 SPST=2 for MSAS->SC messages. This is less efficient, since the 788 information sent is duplicated, but this is the only way to 789 include all receivers in a synchronization session in this 790 scenario. 792 In certain multicast situations, there is no offer/answer context, 793 but only a declarative modus. In that case, the MSAS SHOULD use only 794 the RTCP IDMS packet type and thus use only the SDP parameter rtcp- 795 idms. Receivers that do not support the RTCP IDMS packet will just 796 ignore both the SDP parameter and the RTCP IDMS packets, and will 797 thus not join the synchronization session. For compatability with 798 the TISPAN solution, the MSAS MAY choose to use the RTCP XR IDMS 799 block type instead, using the SDP parameter sync-group. The media 800 sender SHOULD NOT use both parameters at the same time in this case 801 of no offer/answer context. 803 13.2. Declarative cases 805 In certain multicast situations, there is no offer/answer context, 806 but only a declarative modus. In that case, the MSAS SHOULD use only 807 the RTCP IDMS packet type and thus use only the SDP parameter rtcp- 808 idms. Receivers that do not support the RTCP IDMS packet will just 809 ignore both the SDP parameter and the RTCP IDMS packets, and will 810 thus not join the synchronization session. For compatability with 811 the TISPAN solution, the MSAS MAY choose to use the RTCP XR IDMS 812 block type instead, using the SDP parameter sync-group. The media 813 sender SHOULD NOT use both parameters at the same time in this case 814 of no offer/answer context. 816 14. On the use of presentation timestamps 818 A receiver can report on different timing events, i.e. on packet 819 arrival times and on playout times. A receiver SHALL report on 820 arrival times and a receiver MAY report on playout times. RTP packet 821 arrival times are relatively easy to report on. Normally, the 822 processing and playout of the same media stream by different 823 receivers will take roughly the same amount of time. Synchronizing 824 on packet arrival times, may lead to some accuracy loss, but it will 825 be adequate for many applications, such as social TV. 827 Also, if the receivers are in some way controlled, e.g. having the 828 same buffer settings and decoding times, high accuracy can be 829 achieved. However, if all receivers in a synchronization session 830 have the ability to report on, and thus synchronize on, actual 831 playout times, or packet presentation times, this may be more 832 accurate. It is up to applications and implementations of this RTCP 833 extension whether to implement and use this. 835 15. Security Considerations 837 The security considerations described in [RFC3611] apply to this 838 document as well. 840 The specified RTCP XR Block Type in this document is used to collect, 841 summarize and distribute information on packet reception- and 842 playout-times of streaming media. The information may be used to 843 orchestrate the media playout at multiple devices. 845 Errors in the information, either accidental or malicious, may lead 846 to undesired behavior. For example, if one device erroneously 847 reports a two-hour delayed playout, then another device in the same 848 synchronization group could decide to delay its playout by two hours 849 as well, in order to keep its playout synchronized. A user would 850 likely interpret this two hour delay as a malfunctioning service. 852 Therefore, the application logic of both Synchronization Clients and 853 Media Synchronization Application Servers should check for 854 inconsistent information. Differences in playout time exceeding 855 configured limits (e.g. more than ten seconds) could be an indication 856 of such inconsistent information. 858 No new mechanisms are introduced in this document to ensure 859 confidentiality. Encryption procedures, such as those being 860 suggested for a Secure RTP (SRTP) at the time that this document was 861 written, can be used when confidentiality is a concern to end hosts. 863 16. IANA Considerations 865 This document defines a new RTCP packet type called IDMS Settings 866 packet in the IANA registry of RTP parameters, part of RTCP Control 867 Packet types (PT), based on the specification in Section 11. 869 Further, this document defines a new SDP parameter "rtcp-idms" within 870 the existing IANA registry of SDP Parameters, part of the "att-field 871 (media level only)". 873 The SDP attribute "rtcp-idms" defined by this document is registered 874 with the IANA registry of SDP Parameters as follows: 876 SDP Attribute ("att-field"): 878 Attribute name: rtcp-idms 880 Long form: RTCP report block for IDMS 882 Type of name: att-field 883 Type of attribute: media level 885 Subject to charset: no 887 Purpose: see sections 7 and 10 of this document 889 Reference: this document 891 Values: see this document 893 17. Contributors 895 The following people have participated as co-authors or provided 896 substantial contributions to this document: Omar Niamut, Fabian 897 Walraven, Ishan Vaishnavi, Rufael Mekuria and Rob Koenen. 899 18. References 901 18.1. Normative References 903 [I-D.draft-williams-avtcore-clksrc] 904 Williams, A., van Brandenburg, R., Stokking, H., and K. 905 Gross, "RTP Clock Source Signalling, 906 draft-williams-avtcore-clksrc-00", March 2012. 908 [RFC2119] Bradner, S., "Key Words for use in RFCs to Indicate 909 Requirement Levels, RFC 2119", March 1997. 911 [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. 912 Jacobson, "RTP: A Transport Protocol for Real-Time 913 Applications, RFC3550", July 2003. 915 [RFC3551] Schulzrinne, H. and S. Casner, "RTP Profile for Audio and 916 Video conferences with Minimal Control, RFC3551", 917 July 2003. 919 [RFC3611] Friedman, T., Ed., Caceres, R., Ed., and A. Clark, Ed., 920 "RTP Control Protocol Extended Reports (RTCP XR), 921 RFC3611", November 2003. 923 [RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session 924 Description Protocol, RFC4566", July 2006. 926 [RFC5234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax 927 Specifications, RFC5234", January 2008. 929 [RFC5760] Ott, J., Chesterfield, J., and E. Schooler, "RTP Control 930 Protocol (RTCP) Extensions for Single-Source Multicast 931 Sessions with Unicast Feedback, RFC5760", February 2010. 933 [RFC5905] Mills, D., Martin, J., Ed., Burbank, J., and W. Kasch, 934 "Network Time Protocol Version 4: Protocol and Algorithms 935 Specifications, RFC5905", February 2010. 937 [TS183063] 938 "IMS-based IPTV stage 3 specification, TS 183 063 v3.4.1", 939 June 2010. 941 18.2. Informative References 943 [Boronat2009] 944 Boronat, F., Lloret, J., and M. Garcia, "Multimedia group 945 and inter-stream synchronization techniques: a comparative 946 study, Elsevier Information Systems 34 (2009), pp. 108- 947 131". 949 [I-D.draft-gross-leap-second] 950 Gross, K. and R. Brandenburg, van, "RTP and Leap Seconds, 951 draft-gross-leap-seconds-01". 953 [IEEE-1588] 954 "1588-2008 - IEEE Standard for a Precision Clock 955 Synchronization Protocol for Networked Measurement and 956 Control Systems", 2008. 958 [Ishibashi2006] 959 Ishibashi, Y., Nagasaka, M., and N. Fujiyoshi, "Subjective 960 Assessment of Fairness among users in multipoint 961 communications, Proceedings of the 2006 ACM SIGCHI 962 internation conference on Advances in computer 963 entertainment technology, 2006". 965 [RFC5868] Ott, J. and C. Perkins, "Guidelines for Extending the RTP 966 Control Protocol (RTCP), RFC5968", September 2010. 968 Authors' Addresses 970 Ray van Brandenburg 971 TNO 972 Brassersplein 2 973 Delft 2612CT 974 the Netherlands 976 Phone: +31-88-866-7000 977 Email: ray.vanbrandenburg@tno.nl 979 Hans Stokking 980 TNO 981 Brassersplein 2 982 Delft 2612CT 983 the Netherlands 985 Phone: +31-88-866-7000 986 Email: hans.stokking@tno.nl 988 M. Oskar van Deventer 989 TNO 990 Brassersplein 2 991 Delft 2612CT 992 the Netherlands 994 Phone: +31-88-866-7000 995 Email: oskar.vandeventer@tno.nl 997 Fernando Boronat 998 Universitat Politecnica de Valencia 999 IGIC Institute, Universitat Politecnica de Valencia-Campus de Gandia (UPV), C/ Paraninfo, 1, Grao de Gandia, C/ Paraninfo, 1, Grao de Gandia 1000 Valencia 46730 1001 Spain 1003 Phone: +34 962 849 341 1004 Email: fboronat@dcom.upv.es 1005 Mario Montagud 1006 Universitat Politecnica de Valencia 1007 IGIC Institute, Universitat Politecnica de Valencia-Campus de Gandia (UPV), C/ Paraninfo, 1, Grao de Gandia, C/ Paraninfo, 1, Grao de Gandia 1008 Valencia 46730 1009 Spain 1011 Phone: +34 962 849 341 1012 Email: mamontor@posgrado.upv.es 1014 Kevin Gross 1015 AVA Networks 1017 Phone: +1-303-447-0517 1018 Email: Kevin.Gross@AVAnw.com