idnits 2.17.1 draft-ietf-avtcore-idms-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 29 instances of lines with control characters in the document. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (October 31, 2011) is 4558 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: 'RFC5760' is defined on line 989, but no explicit reference was found in the text ** Obsolete normative reference: RFC 4566 (Obsoleted by RFC 8866) Summary: 2 errors (**), 0 flaws (~~), 2 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 AVTCore R. van Brandenburg 3 Internet Draft H. Stokking 4 Intended status: Standards Track M. van Deventer 5 Expires: May 3, 2012 TNO Netherlands 6 F. Boronat 7 M. Montagud 8 Universidad Politecnica de Valencia 9 Kevin Gross 10 AVA Networks 11 October 31, 2011 13 RTCP for inter-destination media synchronization 14 draft-ietf-avtcore-idms-02.txt 16 Status of this Memo 18 This Internet-Draft is submitted in full conformance with the 19 provisions of BCP 78 and BCP 79. 21 Internet-Drafts are working documents of the Internet Engineering 22 Task Force (IETF), its areas, and its working groups. Note that other 23 groups may also distribute working documents as Internet-Drafts. 25 Internet-Drafts are draft documents valid for a maximum of six months 26 and may be updated, replaced, or obsoleted by other documents at any 27 time. It is inappropriate to use Internet-Drafts as reference 28 material or to cite them other than as "work in progress." 30 The list of current Internet-Drafts can be accessed at 31 http://www.ietf.org/ietf/1id-abstracts.txt 33 The list of Internet-Draft Shadow Directories can be accessed at 34 http://www.ietf.org/shadow.html 36 This Internet-Draft will expire on May 3, 2012. 38 Copyright Notice 40 Copyright (c) 2011 IETF Trust and the persons identified as the 41 document authors. All rights reserved. 43 This document is subject to BCP 78 and the IETF Trust's Legal 44 Provisions Relating to IETF Documents 45 (http://trustee.ietf.org/license-info) in effect on the date of 46 publication of this document. Please review these documents 47 carefully, as they describe your rights and restrictions with respect 48 to this document. Code Components extracted from this document must 49 include Simplified BSD License text as described in Section 4.e of 50 the Trust Legal Provisions and are provided without warranty as 51 described in the Simplified BSD License. 53 Abstract 55 This document gives information on an RTCP Packet Type and RTCP XR 56 Block Type including associated SDP parameters for inter-destination 57 media synchronization (IDMS). The RTCP XR Block Type, registered with 58 IANA based on an ETSI specification, is used to collect media play- 59 out information from participants in a group playing-out (watching, 60 listening, etc.) a specific RTP media stream. The RTCP packet type 61 specified by this document is used to distribute a summary of the 62 collected information so that the participants can synchronize play- 63 out. 65 Typical applications for IDMS are social TV, shared service control 66 (i.e. applications where two or more geographically separated users 67 are watching a media stream together), distance learning, networked 68 video walls, networked speakers, etc. 70 Table of Contents 72 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 73 1.1. Inter-destination Media Synchronization . . . . . . . . . . 3 74 1.2. Applicability of RTCP to IDMS . . . . . . . . . . . . . . . 3 75 1.3. Applicability of SDP to IDMS . . . . . . . . . . . . . . . 4 76 1.4. This document and ETSI TISPAN . . . . . . . . . . . . . . . 4 77 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . 4 78 3. Overview of IDMS operation . . . . . . . . . . . . . . . . . . 4 79 4. Inter-destination media synchronization use cases . . . . . . . 6 80 5. Architecture for inter-destination media synchronization . . . 7 81 5.1. Media Synchronization Application Server (MSAS) . . . . . . 7 82 5.2. Synchronization Client (SC) . . . . . . . . . . . . . . . . 8 83 5.3. Communication between MSAS and SCs . . . . . . . . . . . . 8 84 6. RTCP XR Block Type for IDMS . . . . . . . . . . . . . . . . . . 9 85 7. RTCP Packet Type for IDMS (IDMS report) . . . . . . . . . . . . 11 86 8. Timing and NTP Considerations . . . . . . . . . . . . . . . . . 13 87 8.1. Leap Seconds . . . . . . . . . . . . . . . . . . . . . . . 14 88 9. SDP Parameter for RTCP XR IDMS Block Type . . . . . . . . . . . 15 89 10. SDP Parameter for RTCP IDMS Packet Type . . . . . . . . . . . 15 90 11. SDP parameter for clock source . . . . . . . . . . . . . . . . 16 91 12. Compatibility with ETSI TISPAN . . . . . . . . . . . . . . . . 18 92 13. On the use of presentation timestamps . . . . . . . . . . . . 19 93 14. Security Considerations . . . . . . . . . . . . . . . . . . . 19 94 15. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20 95 16. Contributors . . . . . . . . . . . . . . . . . . . . . . . . . 21 96 17. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . 21 97 18. References . . . . . . . . . . . . . . . . . . . . . . . . . . 22 98 18.1. Normative References . . . . . . . . . . . . . . . . . . . 22 99 18.2. Informative References . . . . . . . . . . . . . . . . . . 22 101 1. Introduction 103 1.1. Inter-destination Media Synchronization 105 Inter-destination media synchronization (IDMS) refers to the play-out 106 of media streams at two or more geographically distributed locations 107 in a temporally synchronized manner. It can be applied to both 108 unicast and multicast media streams and can be applied to any type 109 and/or combination of streaming media, such as audio, video and text 110 (subtitles). [Ishibashi2006] and [Boronat2009] provide an overview of 111 technologies and algorithms for IDMS. 113 IDMS requires the exchange of information on media receipt and 114 playout times. It may also require signaling for the initiation and 115 maintenance of IDMS sessions and groups. 117 The presented RTCP specification for IDMS is independent of the used 118 synchronization algorithm, which is out-of-scope of this document. 120 1.2. Applicability of RTCP to IDMS 122 Currently, most multimedia applications make use of RTP and RTCP 123 [RFC3550]. RTP (Real-time Transport Protocol) provides end-to-end 124 network transport functions suitable for applications requiring real- 125 time data transport, such as audio, video or data, over multicast or 126 unicast network services. The timestamps and sequence number 127 mechanisms provided by RTP are very useful to reconstruct the 128 original media timing, reorder and detect some packet loss at the 129 receiver side. 131 The data transport is augmented by a control protocol (RTCP) to allow 132 monitoring of the data delivery in a manner that is scalable to large 133 multicast networks, and to provide minimal control and identification 134 functionality. 136 RTP receivers and senders provide reception quality feedback by 137 sending out RTCP Receiver Report (RR) and Sender Report (SR) packets 138 [RFC3550] respectively, which may be augmented by eXtended Reports 139 (XR) [RFC3611]. Thus, the feedback reporting features provided by 140 RTCP make QoS monitoring possible and can be used for troubleshooting 141 and fault tolerance management in multicast distribution services 142 such as IPTV. 144 These protocols are intended to be tailored through modification 145 and/or additions in order to include profile-specific information 146 required by particular applications, and the guidelines on doing so 147 are specified in [RFC5968]. 149 IDMS involves the collection, summarizing and distribution of RTP 150 packet arrival and play-out times. As information on RTP packet 151 arrival times and play-out times can be considered reception quality 152 feedback information, RTCP becomes a promising candidate for carrying 153 out IDMS, which may facilitate implementation in typical multimedia 154 applications. 156 1.3. Applicability of SDP to IDMS 158 RTCP XR [RFC3611] defines the Extended Report (XR) packet type for 159 the RTP Control Protocol (RTCP), and defines how the use of XR 160 packets can be signaled by an application using the Session 161 Description Protocol (SDP) [RFC4566]. 163 SDP signaling is used to set up and maintain a synchronization group 164 between Synchronization Clients (SCs). This document describes two 165 SDP parameters for doing this, one for the RTCP XR block type and one 166 for the new RTCP packet type. 168 This document also allows for a receiver to indicate a used clock 169 source for synchronizing the receiver clock used in the IDMS session. 170 This is also done using an SDP parameter, which is described in this 171 document. 173 1.4. This document and ETSI TISPAN 175 ETSI TISPAN [TS 183 063] has specified architecture and protocol for 176 IDMS using RTCP XR exchange and SDP signaling. 178 2. Terminology 180 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 181 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 182 document are to be interpreted as described in RFC 2119 [RFC2119] and 183 indicate requirement levels for compliant implementations. 185 3. Overview of IDMS operation 187 This section provides a brief example of how the IDMS RTCP 188 functionality is used. The section is tutorial in nature and does not 189 contain any normative statements. 191 Alice's . . . . . . .tv:abc.com . . . . . . . . . Bob's 192 TV (Sync Client) (Sync Server) Laptop (SC) 193 | | | 194 | Media Session | | 195 |<=====================>| | 196 | Invite(URL,Sync-group ID) | 197 |------------------------------------------------->| 198 | | Media Session Set-up | 199 | |<========================>| 200 | | | 201 | Call set-up | 202 |<================================================>| 203 | | | 204 | RTP Packet | RTP Packet | 205 |<----------------------|------------------------->| 206 | RR + IDMS XR | | 207 |---------------------->| RR + IDMS XR | 208 | |<-------------------------| 209 | RTCP IDMS packet | RTCP IDMS packet | 210 |<----------------------|------------------------->| 211 | | | 213 Alice is watching TV in her living room. At some point she sees that 214 a football game of Bob's favorite team is on. She sends him an invite 215 to watch the program together. Embedded in the invitation is the link 216 to the media server and a unique sync-group identifier. 218 Bob, who is also at home, receives the invite on his laptop. He 219 accepts Alice's invitation and the RTP client on his laptop sets up a 220 session to the media server. A VoIP connection to Alice's TV is also 221 set up, so that Alice and Bob can talk while watching the program. 223 As is common with RTP, both the RTP client in Alice's TV as well as 224 the one in Bob's laptop send periodic RTCP Receiver Reports (RR) to 225 the media server. However, in order to make sure Alice and Bob see 226 the events in the football game at the same time, their clients also 227 periodically send an IDMS XR block to the sync server function of the 228 media server. Included in the XR blocks are timestamps on when both 229 Alice and Bob have received (or played out) a particular RTP packet. 231 The sync server function in the media server calculates a reference 232 client from the received IDMS XR blocks (e.g. by selecting whichever 233 client received the packet the latest as the reference client). It 234 then sends an RTCP IDMS packet containing the play-out information of 235 this reference client to both Alice and Bob. 237 In this case Bob has the slowest connection and the reference client 238 therefore includes a delay similar to the one experienced by Bob. 239 Upon reception of this information, Alice's RTP client can choose 240 what to do with this information. In this case it decreases its play- 241 out rate temporarily until it matches with the reference client play- 242 out (and thus matches Bob's play-out). Another option for Alice's TV 243 would be to simply pause playback until it catches up. The exact 244 implementation of the synchronization algorithm is up to the client. 246 Upon reception of the reference client RTCP IDMS packet, Bob's client 247 does not have to do anything since it is already synchronized to the 248 reference client (since it is based on Bob's delay). Note that other 249 synchronization algorithms may introduce even more delay than the one 250 experienced by the most delayed client, e.g. to account for delay 251 variations, for new clients joining an existing synchronization 252 group, etc. 254 4. Inter-destination media synchronization use cases 256 There are a large number of use cases imaginable in which IDMS might 257 be useful. This section will highlight some of them. It should be 258 noted that this section is in no way meant to be exhaustive 260 A first usage scenario for IDMS is Social TV. Social TV is the 261 combination of media content consumption by two or more users at 262 different devices and locations and real-time communication between 263 those users. An example of Social TV, is when two or more users are 264 watching the same television broadcast at different devices and 265 locations, while communicating with each other using text, audio 266 and/or video. A skew in the media play-out of the two or more users 267 can have adverse effects on their experience. A well-known use case 268 here is one friend experiencing a goal in a football match well 269 before or after other friend(s). Thus IDMS is required to provide 270 play-out synchronization. 272 Another potential use case for IDMS is the video wall. A video wall 273 consists of multiple computer monitors, video projectors, or 274 television sets tiled together contiguously or overlapped in order to 275 form one large screen. Each of the screens reproduces a portion of 276 the larger picture. In some implementations, each screen may be 277 individually connected to the network and receive its portion of the 278 overall image from a network-connected video server or video scaler. 279 Screens are refreshed at 60 hertz (every 16-2/3 milliseconds) or 280 potentially faster. If the refresh is not synchronized, the effect of 281 multiple screens acting as one is broken. 283 A third usage scenario is that of the networked loudspeakers, in 284 which two or more speakers are connected to the network individually. 285 Such situations can for example be found in large conference rooms, 286 legislative chambers, classrooms (especially those supporting 287 distance learning) and other large-scale environments such as 288 stadiums. Since humans are more susceptible to differences in audio 289 delay, this use case needs even more accuracy than the video wall use 290 case. Depending on the exact application, the need for accuracy can 291 then be in the range of microseconds. 293 5. Architecture for inter-destination media synchronization 295 The architecture for IDMS, which is based on a sync-maestro 296 architecture [Boronat2009], is sketched below. The Synchronization 297 Client (SC) and Media Synchronization Application Server (MSAS) 298 entities are shown as additional functionality for the RTP receiver 299 and sender respectively. 301 It should be noted that a master/slave type of architecture is also 302 supported by having one of the SC devices also act as an MSAS. In 303 this case the MSAS functionality is thus embedded in an RTP receiver 304 instead of an RTP sender. 306 +-----------------------+ +-----------------------+ 307 | | SR + | | 308 | RTP Receiver | RTCP | RTP Sender | 309 | | IDMS | | 310 | +-----------------+ | <----- | +-----------------+ | 311 | | | | | | | | 312 | | Synchronization | | | | Media | | 313 | | Client | | | | Synchronization | | 314 | | (SC) | | | | Application | | 315 | | | | | | Server | | 316 | | | | RR+XR | | (MSAS) | | 317 | | | | -----> | | | | 318 | +-----------------+ | | +-----------------+ | 319 | | | | 320 +-----------------------+ +-----------------------+ 322 5.1. Media Synchronization Application Server (MSAS) 324 An MSAS collects RTP packet arrival times and play-out times from one 325 or more SC(s) in a synchronization group. The MSAS summarizes and 326 distributes this information to the SCs in the synchronization group 327 as synchronization settings, e.g. by determining the SC with the most 328 lagged play-out and using its reported RTP packet arrival time and 329 play-out time as a summary. 331 5.2. Synchronization Client (SC) 333 An SC reports RTP packet arrival times and play-out times of a media 334 stream. It can receive summaries of such information, and use that to 335 adjust its play-out buffer. 337 5.3. Communication between MSAS and SCs 339 Two different message types are used for the communication between 340 MSAS and SCs. For the SC->MSAS message containing the play-out 341 information of a particular client, an RTCP XR Block Type is used 342 (see Section 6). For the MSAS->SC message containing the 343 synchronization settings instructions, a new RTCP Packet Type is 344 defined in Section 7. 346 6. RTCP XR Block Type for IDMS 348 This section describes the RTCP XR Block Type for reporting IDMS 349 information on an RTP media stream. Its definition is based on 350 [RFC3611]. The RTCP XR is used to provide feedback information on 351 receipt times and presentation times of RTP packets to e.g. a Sender 352 [RFC3611], a Feedback Target [RFC5576] or a Third Party Monitor 353 [RFC3611]. 355 0 1 2 3 356 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 357 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 358 |V=2|P| Resrv | PT=XR=207 | length | 359 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 360 | SSRC of packet sender | 361 +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ 362 | BT=12 | SPST |Resrv|P| block length=7 | 363 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 364 | PT | Resrv | 365 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 366 | Media Stream Correlation Identifier | 367 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 368 | SSRC of media source | 369 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 370 | Packet Received NTP timestamp, most significant word | 371 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 372 | Packet Received NTP timestamp, least significant word | 373 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 374 | Packet Received RTP timestamp | 375 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 376 | Packet Presented NTP timestamp | 377 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 379 The first 64 bits form the header of the RTCP XR, as defined in 380 [RFC3611]. The SSRC of packet sender identifies the sender of the 381 specific RTCP packet. 383 The IDMS report block consists of 7 32-bit words, with the following 384 fields: 386 Block Type (BT): 8 bits. It identifies the block format. Its value 387 SHALL be set to 12. 389 Synchronization Packet Sender Type (SPST): 4 bits. This field 390 identifies the role of the packet sender for this specific eXtended 391 Report. It can have the following values: 393 SPST=0 Reserved For future use. 395 SPST=1 The packet sender is an SC. It uses this XR to report 396 synchronization status information. Timestamps relate to the SC 397 input. 399 SPST=2 This setting is reserved in order to preserve compatibility 400 with ETSI TISPAN [TS 183 063]. See section 12. for more information. 402 SPST=3-15 Reserved For future use. 404 Reserved bits (Resrv): 3 bits. These bits are reserved for future 405 definition. In the absence of such a definition, the bits in this 406 field MUST be set to zero and MUST be ignored by the receiver. 408 Packet Presented NTP timestamp flag (P): 1 bit. Bit set to 1 if the 409 Packet Presented NTP timestamp field contains a value, 0 if it is 410 empty. If this flag is set to zero, then the Packet Presented NTP 411 timestamp shall not be inspected. 413 Block Length: 16 bits. This field indicates the length of the block 414 in 32 bit words and shall be set to 7, as this RTCP Block Type has a 415 fixed length. 417 Payload Type (PT): 7 bits. This field identifies the format of the 418 media payload, according to [RFC3551]. The media payload is 419 associated with an RTP timestamp clock rate. This clock rate provides 420 the time base for the RTP timestamp counter. 422 Reserved bits (Resrv): 25 bits. These bits are reserved for future 423 use and shall be set to 0. 425 Media Stream Correlation Identifier: 32 bits. This identifier is used 426 to correlate synchronized media streams. The value 0 (all bits are 427 set "0") indicates that this field is empty. The value 2^32-1 (all 428 bits are set "1") is reserved for future use. If the RTCP Packet 429 Sender is an SC (SPST=1), then the Media Stream Correlation 430 Identifier maps on the SyncGroupId to which the SC belongs. 432 SSRC: 32 bits. The SSRC of the media source shall be set to the value 433 of the SSRC identifier carried in the RTP header [RFC3550] of the RTP 434 packet to which the XR relates. 436 Packet Received NTP timestamp: 64 bits. This timestamp reflects the 437 wall clock time at the moment of arrival of the first octet of the 438 RTP packet to which the XR relates. It is formatted based on the NTP 439 timestamp format as specified in [RFC5905]. See section 8 for more 440 information on how this field is set. 442 Packet Received RTP timestamp: 32 bits. This timestamp has the value 443 of the RTP time stamp carried in the RTP header [RFC3550] of the RTP 444 packet to which the XR relates. Several consecutive RTP packets will 445 have equal timestamps if they are (logically) generated at once, 446 e.g., belong to the same video frame. It may well be the case that 447 one receiver reports on the first RTP packet having a certain RTP 448 timestamp and a second receiver reports on the last RTP packet having 449 that same RTP timestamp. This would lead to an error in the 450 synchronization algorithm due to the faulty interpretation of 451 considering both reports to be on the same RTP packet. To solve this, 452 an SC SHOULD report on RTP packets in which a certain RTP timestamp 453 shows up for the first time. 455 Packet Presented NTP timestamp: 32 bits. This timestamp reflects the 456 wall clock time at the moment the data contained in the first octet 457 of the associated RTP packet is presented to the user. It is based on 458 the time format used by NTP and consists of the least significant 16 459 bits of the NTP seconds part and the most significant 16 bits of the 460 NTP fractional second part. If this field is empty, then it SHALL be 461 set to 0 and the Packet Presented NTP timestamp flag (P) SHALL be set 462 to 0. Presented here means the moment the data is presented to the 463 user of the system, i.e. sound played out through speakers, video 464 images being displayed on some display, etc. The accuracy resulting 465 from the synchronization algorithm will only be as good as the 466 accuracy with which the receivers can determine the delay between 467 receiving packets and presenting them to the end-user. 469 7. RTCP Packet Type for IDMS (IDMS report) 471 This section specifies the RTCP Packet Type for indicating 472 synchronization settings instructions to a receiver of the RTP media 473 stream. Its definition is based on [RFC3550]. 475 0 1 2 3 476 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 477 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 478 |V=2|P| Resrv | PT=TBD | length | 479 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 480 | SSRC of packet sender | 481 +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ 482 | SSRC of media source | 483 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 484 | Media Stream Correlation Identifier | 485 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 486 | Packet Received NTP timestamp, most significant word | 487 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 488 | Packet Received NTP timestamp, least significant word | 489 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 490 | Packet Received RTP timestamp | 491 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 492 | Packet Presented NTP timestamp, most significant word | 493 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 494 | Packet Presented NTP timestamp, least significant word | 495 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 497 The first 64 bits form the header of the RTCP Packet Type, as defined 498 in [RFC3550]. The SSRC of packet sender identifies the sender of the 499 specific RTCP packet. 501 The RTCP IDMS packet consists of 7 32-bit words, with the following 502 fields: 504 SSRC: 32 bits. The SSRC of the media source shall be set to the value 505 of the SSRC identifier carried in the RTP header [RFC3550] of the RTP 506 packet to which the RTCP IDMS packet relates. 508 Media Stream Correlation Identifier: 32 bits. This identifier is used 509 to correlate synchronized media streams. The value 0 (all bits are 510 set "0") indicates that this field is empty. The value 2^32-1 (all 511 bits are set "1") is reserved for future use. The Media Stream 512 Correlation Identifier maps on the SyncGroupId of the group to which 513 this packet is sent. 515 Packet Received NTP timestamp: 64 bits. This timestamp reflects the 516 wall clock time at the reference client at the moment it received the 517 first octet of the RTP packet to which this packet relates. It can be 518 used by the synchronization algorithm on the receiving SC to set the 519 required playout delay. The timestamp is formatted based on the NTP 520 timestamp format as specified in [RFC5905]. See section 8 for more 521 information on how this field is set. 523 Packet Received RTP timestamp: 32 bits. This timestamp has the value 524 of the RTP time stamp carried in the RTP header [RFC3550] of the RTP 525 packet to which the XR relates. This SHOULD relate to the first RTP 526 packet containing this particular RTP timestamp, in case multiple RTP 527 packets contain the same RTP timestamp. 529 Packet Presented NTP timestamp: 64 bits. This timestamp reflects the 530 wall clock time at the reference client at the moment it presented 531 the data contained in the first octet of the associated RTP packet to 532 the user. The timestamp is formatted based on the NTP timestamp 533 format as specified in [RFC5905]. If this field is empty, then it 534 SHALL be set to 0. This field MAY be left empty if none or only one 535 of the receivers reported on presentation timestamps. Presented here 536 means the moment the data is presented to the user of the system. 538 In some use cases (e.g. phased array transducers), the level of 539 control a MSAS might need to have over the exact moment of playout is 540 so precise that a 32bit Presentation Timestamp might not suffice. For 541 this reason, this RTCP Packet Type for IDMS includes a 64bit 542 Presentation Timestamp field. Since an MSAS will in practice always 543 add some extra delay to the delay reported by the most lagged 544 receiver (to account for packet jitter), it suffices for the IDMS XR 545 Block Type with which the SCs report on their playout to have a 32bit 546 Presentation Timestamp field. 548 8. Timing and NTP Considerations 550 To achieve IDMS, the different receivers involved need synchronized 551 clocks as a common timeline for synchronization. Depending on the 552 synchronization accuracy required, different clock synchronization 553 methods can be used. For social TV, synchronization accuracy should 554 be achieved in order of hundreds of milliseconds. In that case, 555 correct use of NTP on receivers will in most situations achieve the 556 required accuracy. As a guideline, to deal with clock drift of 557 receivers, receivers should synchronize their clocks at the beginning 558 of a synchronized session. In case of high required accuracy, the 559 synchronized clocks of different receivers should not drift beyond 560 the accuracy required for the synchronization mechanism. In practice 561 this can mean that receivers need to synchronize their clocks 562 repeatedly during a synchronization session. 564 Because of the stringent synchronization requirements for achieving 565 good audio, a high accuracy will be needed. In this case, NTP usage 566 may not be sufficient. Either a local NTP server could be setup, or 567 some other more accurate clock synchronization mechanism could be 568 used, such as using GPS time or the Precision Time Protocol [IEEE- 569 1588]. 571 In this document, a new SDP parameter is introduced to signal the 572 clock synchronization source or sources used or able to be used (see 573 section 10). An SC can indicate which synchronization source is being 574 used at the moment and the last time the SC synchronized with this 575 source. An SC can also indicate any other synchronization sources 576 available to it. This allows multiple SCs in an IDMS session to use 577 the same or a similar clock synchronization source for their session. 579 Applications performing IDMS may or may not be able to choose a 580 synchronization method for the system clock. How applications deal 581 with this is up to the implementation. The application might control 582 the system clock, or it might use a separate application clock or 583 even a separate IDMS session clock. It might also report on the 584 system clock and the synchronization method used, without being able 585 to change it. 587 8.1. Leap Seconds 589 IDMS implementation is simplified by using a clock reference with a 590 timescale which does not include leap seconds. IEEE 1588, GPS and 591 other TAI (Inernational Atomic Time) references do not include leap 592 seconds. NTP time, operating system clocks and other UTC (Coordinated 593 Universal Time) references include leap seconds (though the ITU is 594 studying a proposal which could eventually eliminate leap seconds 595 from UTC). 597 Leap seconds are potentially scheduled at the end of the last day of 598 December and June each year. NTP inserts a leap second at the 599 beginning of the last second of the day. This results in the clock 600 freezing for one second immediately prior to the last second of the 601 affected day. Most system clocks insert the leap second at the end of 602 the last second. This results in repetition of the last second of the 603 day. Generating or using timestamps during the entire last second of 604 a day on which a leap second has been scheduled should therefore be 605 avoided. Note that the period to be avoided has a real-time duration 606 of two seconds. 608 It is also important that all participants correctly implement leap 609 seconds and have a working communications channel to receive 610 notification of leap second scheduling. Without prior knowledge of 611 leap second schedule, NTP servers and clients may be offset by 612 exactly one second with respect to their UTC reference. This 613 potential discrepancy begins when a leap second occurs and ends when 614 all participants receive a time update from a server or peer (which, 615 depending on the operating system and/or implementation, could be 616 anywhere from a few minutes to a week). Such a long-lived discrepancy 617 can be particularly disruptive to RTP and IDMS operation. 619 Apart from the long-lived discrepancy due to dependence on both 620 timing (e.g. NTP) updates and leap seconds scheduling updates, there 621 is also the potential for a short-lived timing discontinuity having 622 an effect on RTP and IDMS playout (even though leap seconds are quite 623 rare). 625 If a timescale with leap seconds is used for IDMS: 627 - SCs must take care not to generate any IDMS XR reports in the 628 immediate vicinity of the leap second. An MSAS must ignore any such 629 reports that may be inadvertently generated. 631 - RTP Senders using a leap-second-bearing reference must not generate 632 sender reports (SR) containing an originating NTP timestamp in the 633 vicinity of a leap second. Receivers should ignore timestamps in any 634 such reports inadvertently generated. 636 - Receivers working to a leap-second-bearing reference must be 637 careful to take leap seconds into account if a leap second occurs 638 between the time a RTP packet is originated and when it is to be 639 presented. 641 9. SDP Parameter for RTCP XR IDMS Block Type 643 The SDP parameter sync-group is used to signal the use of the RTCP XR 644 block for inter-destination media synchronization. It is also used to 645 carry an identifier for the synchronization group to which clients 646 belong or will belong. This SDP parameter extends rtcp-xr-attrib as 647 follows, using Augmented Backus-Naur Form [RFC5234]. 649 rtcp-xr-attrib = "a=" "rtcp-xr" ":" [xr-format *(SP xr-format)] CRLF 650 ; Original definition from [RFC3611], section 5.1 652 xr-format =/ grp-sync ; Extending xr-format for inter-destination 653 media synchronization 655 grp-sync = "grp-sync" [",sync-group=" SyncGroupId] 657 SyncGroupId = 1*DIGIT ; Numerical value from 0 till 4294967295 659 DIGIT = %x30-39 661 SyncGroupId is a 32-bit unsigned integer in network byte order and 662 represented in decimal. SyncGroupId identifies a group of SCs for 663 IDMS. It maps on the Media Stream Correlation Identifier as described 664 in sections 6 and 7. The value SyncGroupId=0 represents an empty 665 SyncGroupId. The value 4294967295 (2^32-1) is reserved for future 666 use. 668 The following is an example of the SDP attribute for IDMS 670 a=rtcp-xr:grp-sync,sync-group=42 672 10. SDP Parameter for RTCP IDMS Packet Type 674 The SDP parameter rtcp-idms is used to signal the use of the RTCP 675 IDMS Packet Type for IDMS. It is also used to carry an identifier for 676 the synchronization group to which clients belong or will belong. The 677 SDP parameter is used as a media-level attribute during session 678 setup. This SDP parameter is defined as follows, using Augmented 679 Backus-Naur Form [RFC5234]. 681 rtcp-idms = "a=" "rtcp-idms" ":" [sync-grp] CRLF 683 sync-grp = "sync-group=" SyncGroupId 684 SyncGroupId = 1*DIGIT ; Numerical value from 0 till 4294967295 686 DIGIT = %x30-39 688 SyncGroupId is a 32-bit unsigned integer in network byte order and 689 represented in decimal. SyncGroupId identifies a group of SCs for 690 IDMS. The value SyncGroupId=0 represents an empty SyncGroupId. The 691 value 4294967295 (2^32-1) is reserved for future use. 693 The following is an example of the SDP attribute for IDMS. 695 a=rtcp-idms:sync-group=42 697 11. SDP parameter for clock source 699 The SDP parameter clocksource is used to signal the source for clock 700 synchronization. This SDP parameter is specified as follows, using 701 Augmented Backus-Naur Form [RFC5234]. 703 clocksource = "a=" "clocksource" ":" source SP [last-synced] CRLF 705 source = local / ntp / gps / gal / ptp 707 local = "local" 709 ntp = "ntp" ["=" ntp-server] 711 ntp-server = host [ ":" port ] 713 host = hostname / IPv4address / IPv6reference 715 hostname = *( domainlabel "." ) toplabel [ "." ] 717 domainlabel = alphanum 719 / alphanum *( alphanum / "-" ) alphanum 721 toplabel = ALPHA / ALPHA *( alphanum / "-" ) alphanum 723 IPv4address = 1*3DIGIT "." 1*3DIGIT "." 1*3DIGIT "." 1*3DIGIT 725 IPv6reference = "[" IPv6address "]" 727 IPv6address = hexpart [ ":" IPv4address ] 729 hexpart = hexseq / hexseq "::" [ hexseq ] / "::" [ hexseq ] 731 hexseq = hex4 *( ":" hex4) 732 hex4 = 1*4HEXDIG 734 port = 1*DIGIT 736 gps = "gps" 738 gal = "gal" 740 ptp = "ptp" SP ptp-version [ ":" ptp-id] 742 ptp-version = "IEEE 1588-2002" / "IEEE 1588-2008" / "IEEE 802.1AS- 743 2011" 745 ptp-id = 1*alphanum 747 last-synced = date SP time UTCoffset 749 date = 2DIGIT "-" 2DIGIT "-" 4DIGIT 751 ; day month year (e.g., 02-06-1982) 753 time = 2DIGIT ":" 2DIGIT ":" 2DIGIT "." 3DIGIT 755 ; 00:00:00.000 - 23:59:59.999 757 UTCoffset = plusoffset / minusoffset 759 plusoffset = + 2DIGIT ":" 2DIGIT 761 ; +01:00 (+HH:MM) 763 minusoffset = - 2DIGIT ":" 2DIGIT 765 ; -00:00 (-HH:MM) 767 alphanum = ALPHA / DIGIT 769 EXAMPLE 771 a=clocksource:ntp=139.63.192.5:123 19-02-2011 21:03:20.345+01:00 773 A client MAY include this attribute multiple times. If multiple time 774 synchronization sources were used in the past, the client MUST only 775 report the 'last synced' parameter on the latest synchronization 776 performed. If a client supports a specific synchronization method, 777 but does not know any sources to use for synchronization, it SHOULD 778 indicate the method without specifying the source. A client MAY 779 indicate itself as source if it is a clock synchronization source, 780 but it SHOULD do so using a publicly reachable address. 782 The parameter can be used as both a session or media level attribute. 783 It will normally be a session level parameter, since it is not 784 directly media-related. In case of IDMS however, it can be used in 785 conjunction with the rtcp-idms SDP parameter, and then it SHOULD be 786 used as a media-level parameter as well. 788 The meaning of 'local' is that no clock synchronization is performed. 789 Allthough not re-used, the meaning of 'gps' (Global Positioning 790 System), 'gal' (Galileo Positioning System) and ptp (Precision Time 791 Protocol) follows the use of reference identifiers in NTP [RFC5905]. 792 When using ptp, the ptp-id MUST contain the proper identifier from 793 the used IEEE specification. 795 The 'last synced' parameter is used as an indication for the receiver 796 of the parameter on the accuracy of the clock. If the indicated last 797 synchronization time is very recent, this is an indication that the 798 clock can be trusted to be accurate, given the method of clock 799 synchronization used. If the indicated last synchronization time is 800 longer ago or in the future, either the clock synchronization has 801 been performed long ago, or the clock is synchronized to an incorrect 802 synchronization source. Either way, this shows that the clock used 803 can not be trusted to be accurate. 805 12. Compatibility with ETSI TISPAN 807 As described in section 1.4, ETSI TISPAN has also described a 808 mechanism for IDMS in [TS 183 063]. One of the main differences 809 between the TISPAN document and this document is the fact that the 810 TISPAN solution uses an RTPC XR block for both the SC->MSAS message 811 as well as for the MSAS->SC message (by selecting different SPST- 812 types), while this document specifies a new RTCP Packet Type for the 813 MSAS->SC message. The message from MSAS to SC is not in any way a 814 report on how a receiver sees a session, and therefore a separate 815 RTCP packet type is more appropriate then the XR block solution 816 chosen in ETSI TISPAN. 818 In order to maintain backward-compatibility, the RTCP XR block used 819 for SC->MSAS signaling specified in this document is fully compatible 820 with the TISPAN defined XR block. 822 For the MSAS->SC signaling, it is recommended to use the RTCP IDMS 823 Packet Type defined in this document. The TISPAN XR block with SPST=2 824 MAY be used for purposes of compatibility with the TISPAN solution, 825 but MUST NOT be used if all nodes involved support the new RTCP IDMS 826 Packet Type. 828 The above means that the IANA registry contains two SDP parameters 829 for the MSAS->SC signaling; one for the ETSI TISPAN solution and one 830 for the IETF solution. This also means that if all elements in the 831 SDP negotiation support the IETF solution they SHOULD use the new 832 RTCP IDMS Packet Type. 834 13. On the use of presentation timestamps 836 A receiver can report on different timing events, i.e. on packet 837 arrival times and on playout times. A receiver SHALL report on 838 arrival times and a receiver MAY report on playout times. RTP packet 839 arrival times are relatively easy to report on. Normally, the 840 processing and play-out of the same media stream by different 841 receivers will take roughly the same amount of time. By synchronizing 842 on packet arrival times, you may loose some accuracy, but it will be 843 adequate for many applications, such as social TV. Also, if the 844 receivers are in some way controlled, e.g. having the same buffer 845 settings and decoding times, high accuracy can be achieved. However, 846 if all receivers in a synchronization session have the ability to 847 report on, and thus synchronize on, actual playout times, or packet 848 presentation times, this may be more accurate. It is up to 849 applications and implementations of this RTCP extension whether to 850 implement and use this. 852 14. Security Considerations 854 The specified RTCP XR Block Type in this document is used to collect, 855 summarize and distribute information on packet reception- and playout 856 -times of streaming media. The information may be used to orchestrate 857 the media play-out at multiple devices. 859 Errors in the information, either accidental or malicious, may lead 860 to undesired behavior. For example, if one device erroneously reports 861 a two-hour delayed play-out, then another device in the same 862 synchronization group could decide to delay its play-out by two hours 863 as well, in order to keep its play-out synchronized. A user would 864 likely interpret this two hour delay as a malfunctioning service. 866 Therefore, the application logic of both Synchronization Clients and 867 Media Synchronization Application Servers should check for 868 inconsistent information. Differences in play-out time exceeding 869 configured limits (e.g. more than ten seconds) could be an indication 870 of such inconsistent information. 872 No new mechanisms are introduced in this document to ensure 873 confidentiality. Encryption procedures, such as those being suggested 874 for a Secure RTP (SRTP) at the time that this document was written, 875 can be used when confidentiality is a concern to end hosts. 877 15. IANA Considerations 879 New RTCP Packet Types and RTCP XR Block Types are subject to IANA 880 registration. For general guidelines on IANA considerations for RTCP 881 XR, refer to [RFC3611]. 883 [TS 183 063] assigns the block type value 12 in the RTCP XR Block 884 Type Registry to "Inter-destination Media Synchronization Block". [TS 885 183 063] also registers the SDP [RFC4566] parameter "grp-sync" for 886 the "rtcp-xr" attribute in the RTCP XR SDP Parameters Registry. 888 Further, this document defines a new RTCP packet type called IDMS 889 report. This new packet type is registered with the IANA registry of 890 RTP parameters, based on the specification in section 7. 892 Further, this document defines a new SDP parameter "rtcp-idms" within 893 the existing IANA registry of SDP Parameters. 895 The SDP attribute "rtcp-idms" defined by this document is registered 896 with the IANA registry of SDP Parameters as follows: 898 SDP Attribute ("att-field"): 900 Attribute name: rtcp-idms 902 Long form: RTCP report block for IDMS 904 Type of name: att-field 906 Type of attribute: media level 908 Subject to charset: no 910 Purpose: see sections 7 and 10 of this document 912 Reference: this document 914 Values: see this document 916 Further, this document defines a new SDP attribute, "clocksource", 917 within the existing IANA registry of SDP Parameters. 919 The SDP attribute "clocksource" defined by this document is 920 registered with the IANA registry of SDP Parameters as follows: 922 SDP Attribute ("att-field"): 924 Attribute name: clocksource 925 Long form: clock synchronization source 927 Type of name: att-field 929 Type of attribute: session level 931 Subject to charset: no 933 Purpose: see sections 8 and 11 of this document 935 Reference: this document 937 Values: see this document and registrations below 939 The attribute has an extensible parameter field and therefore a 940 registry for these parameters is required. This document creates an 941 IANA registry called the Clocksource Source Parameters Registry. It 942 contains the five parameters defined in Section 11: "local", "ntp", 943 "gps", "gal" and "ptp". 945 16. Contributors 947 The following people have participated as co-authors or provided 948 substantial contributions to this document: Omar Niamut, Fabian 949 Walraven, Ishan Vaishnavi, Rufael Mekuria 951 17. Conclusions 953 This document describes the RTCP XR block type for IDMS, the RTCP 954 IDMS report and the associated SDP parameters for inter-destination 955 media synchronization. It also describes an SDP parameter for 956 indicating which source is used for synchronizing a (systems) (wall) 957 clock. 959 18. References 961 18.1. Normative References 963 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 964 Requirement Levels", BCP 14, RFC 2119, March 1997. 966 [RFC5234] Crocker, D., Ed., and P. Overell, "Augmented BNF for 967 Syntax Specifications: ABNF", STD 68, RFC 5234, January 968 2008. 970 [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. 971 Jacobson, "RTP: A Transport Protocol for Real-Time 972 Applications", STD 64, RFC 3550, July 2003. 974 [RFC3551] Schulzrinne, H. and S. Casner, "RTP Profile for Audio and 975 Video Conferences with Minimal Control", STD 65, RFC 3551, 976 July 2003. 978 [RFC3611] Friedman, T., Ed., Caceres, R., Ed., and A. Clark, Ed., 979 "RTP Control Protocol Extended Reports (RTCP XR)", 980 RFC 3611, November 2003. 982 [RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session 983 Description Protocol", RFC 4566, July 2006. 985 [RFC5576] Lennox, J., Ott, J., and T. Schierl, "Source-Specific 986 Media Attributes in the Session Description Protocol 987 (SDP)", RFC 5576, June 2009. 989 [RFC5760] Ott, J., Chesterfield, J., and E. Schooler, "RTP Control 990 Protocol (RTCP) Extensions for Single-Source Multicast 991 Sessions with Unicast Feedback", RFC 5760, February 2010. 993 [RFC5905] Mills, D., Martin, J., Ed., Burbank, J., and W. Kasch, 994 "Network Time Protocol Version 4: Protocol and Algorithms 995 Specification", RFC 5905, June 2010. 997 [TS 183 063] ETSI TISPAN, "IMS-based IPTV stage 3 specification", TS 998 183 063 v3.4.1, June 2010. 1000 18.2. Informative References 1002 [RFC5968] Ott, J. and C. Perkins, "Guidelines for Extending the RTP 1003 Control Protocol (RTCP)", RFC 5968, September 2010. 1005 [Boronat2009] Boronat, F., et al, "Multimedia group and inter-stream 1006 synchronization techniques: A comparative study", Elsevier 1007 Information Systems 34 (2009), pp. 108-131 1009 [Ishibashi2006] Ishibashi Y. et al, "Subjective Assesment of Fairness 1010 among users in multipoint communications". Proceedings of 1011 the 2006 ACM SIGCHI international conference on Advances 1012 in computer entertainment technology, 2006. 1014 [IEEE-1588] IEEE Standards Association, "1588-2008 - IEEE Standard 1015 for a Precision Clock Synchronization Protocol for 1016 Networked Measurement and Control Systems", 2008 1018 Authors' Addresses 1020 Ray van Brandenburg 1021 TNO 1022 Brassersplein 2, Delft, the Netherlands 1024 Phone: +31 88 86 63609 1025 Email: ray.vanbrandenburg@tno.nl 1027 Hans M. Stokking 1028 TNO 1029 Brassersplein 2, Delft, the Netherlands 1031 Phone: +31 88 86 67278 1032 Email: hans.stokking@tno.nl 1034 M. Oskar van Deventer 1035 TNO 1036 Brassersplein 2, Delft, the Netherlands 1038 Phone: +31 88 86 67078 1039 Email: oskar.vandeventer@tno.nl 1041 Fernando Boronat 1042 IGIC Institute, Universidad Politecnica de Valencia-Campus de Gandia 1043 C/ Paraninfo, 1, Grao de Gandia, 46730, Valencia, Spain 1045 Phone: +34 962 849 341 1046 Email: fboronat@dcom.upv.es 1048 Mario Montagud 1049 IGIC Institute, Universidad Politecnica de Valencia-Campus de Gandia 1050 C/ Paraninfo, 1, Grao de Gandia, 46730, Valencia, Spain 1052 Phone: +34 962 849 341 1053 Email: mamontor@posgrado.upv.es 1055 Kevin Gross 1056 AVA Networks