idnits 2.17.1 draft-ietf-avtcore-idms-10.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (June 14, 2013) is 3966 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Outdated reference: A later version (-11) exists of draft-ietf-avtcore-clksrc-05 ** Obsolete normative reference: RFC 4566 (Obsoleted by RFC 8866) ** Obsolete normative reference: RFC 5226 (Obsoleted by RFC 8126) == Outdated reference: A later version (-08) exists of draft-ietf-avtcore-leap-second-02 Summary: 2 errors (**), 0 flaws (~~), 3 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 AVTCore R. van Brandenburg 3 Internet-Draft H. Stokking 4 Intended status: Standards Track O. van Deventer 5 Expires: December 16, 2013 TNO 6 F. Boronat 7 M. Montagud 8 Universitat Politecnica de Valencia 9 K. Gross 10 AVA Networks 11 June 14, 2013 13 Inter-destination Media Synchronization using the RTP Control Protocol 14 (RTCP) 15 draft-ietf-avtcore-idms-10 17 Abstract 19 This document defines a new RTP Control Protocol (RTCP) Packet Type 20 and an RTCP Extended Report (XR) Block Type to be used for achieving 21 Inter-Destination Media Synchronization (IDMS). IDMS is the process 22 of synchronizing playout across multiple geographically distributed 23 media receivers. Using the RTCP XR IDMS Report Block defined in this 24 document, media playout information from participants in a 25 synchronization group can be collected. Based on the collected 26 information, an RTCP IDMS Settings Packet can then be sent to 27 distribute a common target playout point to which all the distributed 28 receivers, sharing a media experience, can synchronize. 30 Typical use cases in which IDMS is useful are social TV, shared 31 service control (i.e. applications where two or more geographically 32 separated users are watching a media stream together), distance 33 learning, networked video walls, networked loudspeakers, etc. 35 Status of This Memo 37 This Internet-Draft is submitted in full conformance with the 38 provisions of BCP 78 and BCP 79. 40 Internet-Drafts are working documents of the Internet Engineering 41 Task Force (IETF). Note that other groups may also distribute 42 working documents as Internet-Drafts. The list of current Internet- 43 Drafts is at http://datatracker.ietf.org/drafts/current/. 45 Internet-Drafts are draft documents valid for a maximum of six months 46 and may be updated, replaced, or obsoleted by other documents at any 47 time. It is inappropriate to use Internet-Drafts as reference 48 material or to cite them other than as "work in progress." 49 This Internet-Draft will expire on December 16, 2013. 51 Copyright Notice 53 Copyright (c) 2013 IETF Trust and the persons identified as the 54 document authors. All rights reserved. 56 This document is subject to BCP 78 and the IETF Trust's Legal 57 Provisions Relating to IETF Documents 58 (http://trustee.ietf.org/license-info) in effect on the date of 59 publication of this document. Please review these documents 60 carefully, as they describe your rights and restrictions with respect 61 to this document. Code Components extracted from this document must 62 include Simplified BSD License text as described in Section 4.e of 63 the Trust Legal Provisions and are provided without warranty as 64 described in the Simplified BSD License. 66 Table of Contents 68 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 69 2. Rationale . . . . . . . . . . . . . . . . . . . . . . . . . . 3 70 2.1. Applicability of RTCP to IDMS . . . . . . . . . . . . . . 3 71 2.2. IDMS and ETSI . . . . . . . . . . . . . . . . . . . . . . 4 72 3. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 73 4. Inter-Destination Media Synchronization (IDMS) use cases . . 4 74 5. Overview of IDMS operation . . . . . . . . . . . . . . . . . 5 75 6. Architecture for Inter-Destination Media Synchronization . . 7 76 6.1. Media Synchronization Application Server (MSAS) . . . . . 7 77 6.2. Synchronization Client (SC) . . . . . . . . . . . . . . . 7 78 6.3. Communication between MSAS and SCs . . . . . . . . . . . 8 79 7. RTCP XR Block for IDMS (IDMS Report Block) . . . . . . . . . 8 80 8. RTCP Packet Type for IDMS (IDMS Settings) . . . . . . . . . . 11 81 9. Timing and NTP Considerations . . . . . . . . . . . . . . . . 13 82 10. On the use of presentation timestamps . . . . . . . . . . . . 14 83 11. SDP Signalling for RTCP IDMS Packet Type . . . . . . . . . . 14 84 12. SDP rules . . . . . . . . . . . . . . . . . . . . . . . . . . 15 85 12.1. Offer/Answer rules . . . . . . . . . . . . . . . . . . . 15 86 12.2. Declarative cases . . . . . . . . . . . . . . . . . . . 16 87 13. Security Considerations . . . . . . . . . . . . . . . . . . . 17 88 14. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 17 89 14.1. RTCP IDMS Packet Type . . . . . . . . . . . . . . . . . 18 90 14.2. RTCP XR IDMS Report Block . . . . . . . . . . . . . . . 18 91 14.3. RTCP-IDMS SDP Attribute . . . . . . . . . . . . . . . . 18 92 14.4. IDMS XR Block SPST Registry . . . . . . . . . . . . . . 19 93 14.5. Contact Information for Registrations . . . . . . . . . 19 94 15. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 19 95 16. References . . . . . . . . . . . . . . . . . . . . . . . . . 19 96 16.1. Normative References . . . . . . . . . . . . . . . . . . 19 97 16.2. Informative References . . . . . . . . . . . . . . . . . 20 98 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 21 100 1. Introduction 102 Inter-Destination Media Synchronization (IDMS) refers to the playout 103 of media streams at two or more geographically distributed locations 104 in a time synchronized manner. It can be applied to both unicast and 105 multicast media streams and can be applied to any type and/or 106 combination of streaming media, such as audio, video and text 107 (subtitles).[Ishibashi2006] and [Boronat2009] provide an overview of 108 technologies and algorithms for IDMS. 110 IDMS requires the exchange of information on media receipt and 111 playout times among participants in an IDMS session. It may also 112 require signaling for the initiation and maintenance of IDMS sessions 113 and groups of receivers. 115 The presented RTCP specification for IDMS is independent of the used 116 synchronization algorithm, which is out-of-scope of this document. 118 2. Rationale 120 2.1. Applicability of RTCP to IDMS 122 Currently, a large share of real-time applications make use of RTP 123 and RTCP [RFC3550]. RTP provides end-to-end network transport 124 functions suitable for applications requiring real-time data 125 transport, such as audio, video or data, over multicast or unicast 126 network services. The timestamps, sequence numbers, and payload 127 (content) type identification mechanisms provided by RTP packets are 128 very useful for reconstructing the original media timing, and for 129 reordering and detecting packet loss at the client side. 131 The data transport is augmented by a control protocol (RTCP) to allow 132 monitoring of the data delivery in a manner that is scalable to large 133 groups, and to provide minimal control and identification 134 functionality. RTP receivers and senders provide reception quality 135 feedback by sending out RTCP Receiver Report (RR) and Sender Report 136 (SR) packets [RFC3550], respectively, which may be augmented by 137 eXtended Reports (XR) [RFC3611]. Both RTP and RTCP are intended to 138 be tailored through modifications in order to include profile- 139 specific information required by particular applications, and the 140 guidelines on doing so are specified in [RFC5868]. 142 IDMS involves the collection, summarizing and distribution of RTP 143 packet arrival and playout times. As information on RTP packet 144 arrival times and playout times can be considered reception quality 145 feedback information, RTCP is well suited for carrying out IDMS, 146 which may facilitate the implementation and deployment in typical 147 multimedia applications. 149 2.2. IDMS and ETSI 151 A first version of IDMS for use with RTP/RTCP was standardized by 152 ETSI TISPAN in [TS183063], resulting in an IANA registration for an 153 RTCP IDMS XR Block. At some point in time, this work was brought as 154 input to the IETF AVTCORE WG for further standardization, leveraging 155 the RTP/RTCP expertise present in the AVTCORE WG. This document is 156 the result of that effort. 158 Although the IDMS protocol described in this document has evolved 159 significantly from the version that was originally specified by ETSI 160 TISPAN, it is still backwards compatible with the ETSI version. As 161 such, it had been decided in ETSI to update the TS 183 063 document 162 to reference this document as the normative specification of IDMS. 163 This update can be found in newer versions of TS 183 063 (>3.5.2). 164 In accordance, this document proposes to update the IANA registration 165 for the RTCP IDMS XR Block to point to this document. Finally, this 166 document proposes an IANA registry for SPST values, allowing ETSI to 167 register extensions to this document. 169 3. Terminology 171 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 172 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 173 document are to be interpreted as described in RFC 2119 [RFC2119] and 174 indicate requirement levels for compliant implementations. 176 4. Inter-Destination Media Synchronization (IDMS) use cases 178 There are a large number of use cases in which IDMS might be useful. 179 This section will highlight some of them. It should be noted that 180 this section is in no way meant to be exhaustive. 182 A first usage scenario for IDMS is social TV. Social TV is the 183 combination of media content consumption by two or more users at 184 different devices and locations combined with the real-time 185 communication between those users. An example of social TV is when 186 two or more users are watching the same television broadcast at 187 different devices and locations, while communicating with each other 188 using text, audio and/or video. A skew in their media playout 189 processes can have adverse effects on their experience. A well-known 190 use case here is one friend experiencing a goal in a football match 191 well before or after other friend(s). 193 Another potential use case for IDMS is a networked video wall. A 194 video wall consists of multiple computer monitors, video projectors, 195 or television sets tiled together contiguously or overlapped in order 196 to form one large screen. Each of the screens reproduces a portion 197 of the larger picture. In some implementations, each screen may be 198 individually connected to the network and receive its portion of the 199 overall image from a network-connected video server or video scaler. 200 Screens are refreshed at 60 hertz (every 16-2/3 milliseconds) or 201 potentially faster. If the refresh is not synchronized, the effect 202 of multiple screens acting as one is broken. 204 A third usage scenario is that of the networked loudspeakers, in 205 which two or more speakers are connected to the network individually. 206 Such situations can for example be found in large conference rooms, 207 legislative chambers, classrooms (especially those supporting 208 distance learning) and other large-scale environments such as 209 stadiums. Since humans are more susceptible to differences in audio 210 delay, this use case needs even more accuracy than the video wall use 211 case. Depending on the exact application, the need for accuracy can 212 then be in the range of microseconds. 214 5. Overview of IDMS operation 216 This section provides a brief example of how the RTCP functionality 217 is used for achieving IDMS. The section is tutorial in nature and 218 does not contain any normative statements. 220 Alice's . . . . . . .tv:abc.com . . . . . . . Bob's 221 TV (Sync Client) (Sync Server) Laptop (Sync Client) 222 | | | 223 | Media Session | | 224 |<=====================>| | 225 | Invite(URL,Sync-group ID) | 226 |------------------------------------------------->| 227 | | Media Session Set-up | 228 | |<========================>| 229 | | | 230 | Call set-up | 231 |<================================================>| 232 | | | 233 | RTP Packets | RTP Packets | 234 |<----------------------|------------------------->| 235 | RR + XR IDMS Report | | 236 |---------------------->| RR + XR IDMS Report | 237 | |<-------------------------| 238 | RTCP IDMS Settings | RTCP IDMS Settings | 239 |<----------------------|------------------------->| 240 | | | 241 Figure 1: Example of a typical IDMS session 243 Alice is watching TV in her living room. At some point she sees that 244 a football game of Bob's favorite team is on. She sends him an 245 invite to watch the program together. Embedded in the invitation is 246 the link to the media server and a unique sync-group identifier. 248 Bob, who is also at home, receives the invite on his laptop. He 249 accepts Alice's invitation and the RTP client on his laptop sets up a 250 session to the media server. A VoIP connection to Alice's TV is also 251 set up, so that Alice and Bob can talk while watching the game 252 together. 254 As is common with RTP, both the RTP client in Alice's TV as well as 255 the one in Bob's laptop send periodic RTCP Receiver Reports (RR) to 256 the media server. However, in order to make sure Alice and Bob see 257 the events in the football game at (approximately) the same time, 258 their clients also periodically send an RTCP XR IDMS Report Block to 259 the Sync Server function of the media server. Included in the XR 260 IDMS blocks are timestamps on when both Alice and Bob received (and, 261 optionally, when they played out) a particular RTP packet. 263 The Sync Server function in the media server calculates a reference 264 client from the received XR IDMS Report Blocks (e.g. by selecting the 265 most lagged client as the reference for IDMS). It then sends an RTCP 266 IDMS Settings packet containing the playout information of this 267 reference client to the sync clients of both Alice and Bob. 269 In this case Bob's connection has the longest delay and the reference 270 client therefore includes a delay similar to the one experienced by 271 Bob. Upon reception of this information, Alice's RTP client can 272 choose what to do with this information. In this case it decreases 273 its playout rate temporarily until the playout time matches with the 274 reference client playout (and thus matches Bob's playout). Another 275 option for Alice's TV would be to simply pause playback until it 276 catches up. The exact implementation of the synchronization 277 algorithm is up to the client. 279 Upon reception of the RTCP IDMS Settings packet, Bob's client does 280 not have to do anything since it is already synchronized to the 281 reference client (since it is based on Bob's delay). Note that other 282 synchronization algorithms may introduce even more delay than the one 283 experienced by the most delayed client, e.g. to account for delay 284 variations, for new clients joining an existing synchronization 285 group, etc. 287 For this functionality to work correctly, it is necessary that the 288 wall clocks of the receivers are synchronized with each other. Alice 289 and Bob both report when they receive, and optionally when they play 290 out, certain RTP packets. In order to correlate their reports to 291 each other, it is necessary that their wallclocks are synchronized. 293 6. Architecture for Inter-Destination Media Synchronization 295 The architecture for IDMS, which is based on a sync-maestro 296 architecture [Boronat2009], is sketched below. The Synchronization 297 Client (SC) and Media Synchronization Application Server (MSAS) 298 entities are shown as additional functionality for the RTP receiver 299 and sender respectively. 301 It should be noted that a master/slave type of architecture is also 302 supported by having one of the SC devices also act as an MSAS. In 303 this case the MSAS functionality is thus embedded in an RTP receiver 304 instead of an RTP sender. 306 +-----------------------+ +-----------------------+ 307 | | SR + | | 308 | RTP Receiver | RTCP | RTP Sender | 309 | | IDMS | | 310 | +-----------------+ | <----- | +-----------------+ | 311 | | | | | | | | 312 | | Synchronization | | | | Media | | 313 | | Client | | | | Synchronization | | 314 | | (SC) | | | | Application | | 315 | | | | | | Server | | 316 | | | | RR+XR | | (MSAS) | | 317 | | | | -----> | | | | 318 | +-----------------+ | | +-----------------+ | 319 | | | | 320 +-----------------------+ +-----------------------+ 322 IDMS Architecture Diagram 324 6.1. Media Synchronization Application Server (MSAS) 326 An MSAS collects RTP packet arrival times and playout times from one 327 or more SC(s) in a synchronization group. The MSAS summarizes and 328 distributes this information to the SCs in the synchronization group 329 as synchronization settings, e.g. by determining the SC with the most 330 lagged playout and using its reported RTP packet arrival time and 331 playout time as a summary. 333 6.2. Synchronization Client (SC) 334 An SC reports on RTP packet arrival times and playout times of a 335 media stream. It can receive summaries of such information, and use 336 that to adjust its playout buffer. 338 6.3. Communication between MSAS and SCs 340 Two different message types are used for the communication between 341 MSAS and SCs. For the SC->MSAS message containing the playout 342 information of a particular client, an RTCP XR IDMS Report Block is 343 used (see Section 7). For the MSAS->SC message containing the 344 synchronization settings instructions, a new RTCP IDMS Settings 345 Packet Type is defined (see Section 8). 347 7. RTCP XR Block for IDMS (IDMS Report Block) 349 This section specifies a new RTCP XR Block Type, the RTCP XR IDMS 350 Report Block, for reporting IDMS information to an MSAS. In 351 particular it is used to provide feedback information on receipt 352 times and presentation times of RTP packets. Its definition is based 353 on [RFC3550] and [RFC3611]. 355 In most cases, a single RTP receiver will only be part of a single 356 IDMS session, i.e. it will report on receipt and presentation times 357 of RTP packets from a single RTP stream in a certain synchronization 358 group. In some cases, however, an RTP receiver may be a member of 359 multiple synchronization groups for the same RTP stream, e.g. 360 watching a single television program simultaneously with different 361 groups. In even further cases, a receiver may wish to synchronize 362 different RTP streams at the same time, either as part of the same 363 synchronization group or as part of multiple synchronization groups. 364 These are all valid scenarios for IDMS, and will require multiple 365 reports by an SC. 367 SCs SHOULD report on a recently received RTP packets. This document 368 does not define new rules on when to send RTCP reports, but uses the 369 existing rules specified in [RFC3550] for sending RTCP reports. When 370 the RTCP reporting timer allows an SC to send an IDMS report, the SC 371 SHOULD report on an RTP packet received during the period since the 372 last RTCP XR IDMS Report Block was sent. For more details on which 373 packet to report on, see below under 'Packet Received RTP timestamp'. 375 0 1 2 3 376 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 377 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 378 | BT=12 | SPST |Resrv|P| block length=7 | 379 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 380 | PT | Resrv | 381 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 382 | Media Stream Correlation Identifier | 383 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 384 | SSRC of media source | 385 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 386 | Packet Received NTP timestamp, most significant word | 387 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 388 | Packet Received NTP timestamp, least significant word | 389 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 390 | Packet Received RTP timestamp | 391 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 392 | Packet Presented NTP timestamp | 393 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 395 The IDMS report block consists of 8 32-bit words, with the following 396 fields: 398 Block Type (BT): 8 bits. It identifies the block format. Its value 399 SHALL be set to 12. 401 Synchronization Packet Sender Type (SPST): 4 bits. This field 402 identifies the role of the packet sender for this specific eXtended 403 Report. It can have the following values, as maintained by IANA (see 404 Section 14.4): 406 SPST=0 Reserved for future use. 408 SPST=1 The packet sender is an SC. It uses this XR to report 409 synchronization status information. Timestamps relate to the SC 410 input. 412 SPST=2-4 Values defined by ETSI TISPAN (see [TS183063]). 414 SPST=5-15 Reserved for future use. 416 Reserved bits (Resrv): 3 bits. These bits are reserved for future 417 definition. In the absence of such a definition, the bits in this 418 field MUST be set to zero at transmission and MUST be ignored by the 419 receiver. 421 Packet Presented NTP timestamp flag (P): 1 bit. Bit set to 1 if the 422 Packet Presented NTP timestamp field contains a value, 0 if it is 423 empty. If this flag is set to zero, then the Packet Presented NTP 424 timestamp SHALL be ignored by the receiver. 426 Block Length: 16 bits. This field indicates the length of the block 427 in 32 bit words minus one and SHALL be set to 7, as this RTCP Block 428 Type has a fixed length. 430 Payload Type (PT): 7 bits. This field identifies the format of the 431 media payload, according to [RFC3551]. This is the payload type of 432 the RTP packet reported upon. The PT field is needed in the case 433 where the MSAS is neither the Media Server nor a receiver of the 434 media stream, i.e. it is implemented as a third-part entity. In such 435 cases, the MSAS needs the PT to determine the rate of advancement of 436 the timestamps of the RTP media stream to be able to relate reports 437 from different SCs on different RTP timestamp values. 439 Reserved bits (Resrv): 25 bits. These bits are reserved for future 440 use and SHALL be set to 0 at transmission and MUST be ignored by the 441 receiver. 443 Media Stream Correlation Identifier: 32 bits. This identifier is 444 used to correlate synchronized media streams. The value 0 (all bits 445 are set "0") indicates that this field is empty. The value 2^32-1 446 (all bits are set "1") is reserved for future use. If the RTCP 447 Packet Sender is an SC (SPST=1), then the Media Stream Correlation 448 Identifier field contains the Synchronization Group Identifier 449 (SyncGroupId) to which the report applies. 451 SSRC: 32 bits. The SSRC of the media source SHALL be set to the 452 value of the SSRC identifier carried in the RTP header [RFC3550] of 453 the RTP packet to which the XR IDMS relates. 455 Packet Received NTP timestamp: 64 bits. This timestamp reflects the 456 wall clock time at the moment of arrival of the first octet of the 457 RTP packet to which the XR IDMS relates. It is formatted based on 458 the NTP timestamp format as specified in [RFC5905]. See Section 9 459 for more information on how this field is used. 461 Packet Received RTP timestamp: 32 bits. This timestamp has the value 462 of the RTP timestamp carried in the RTP header [RFC3550] of the RTP 463 packet to which the XR IDMS relates. Several consecutive RTP packets 464 will have equal timestamps if they are (logically) generated at once, 465 e.g., belong to the same video frame. It may well be the case that 466 one receiver reports on the first RTP packet having a certain RTP 467 timestamp and a second receiver reports on the last RTP packet having 468 that same RTP timestamp. This would lead to an error in the 469 synchronization algorithm due to the faulty interpretation of 470 considering both reports to be on the same RTP packet. When 471 reporting on an RTP packet which is one of several consecutive RTP 472 packets having equal timestamps, an SC SHOULD report on the RTP 473 packet it received with the lowest sequence number. Note that with 474 'lowest sequence number' here is meant the first in the sequence of 475 RTP packets just received, not from an earlier time before the last 476 wrap-around of RTP timestamps (unless this wrap-around occurs during 477 the sequence with equal RTP timestamps). 479 Packet Presented NTP timestamp: 32 bits. This timestamp reflects the 480 wall clock time at the moment the rendered media unit (e.g. video 481 frame or audio sample) contained in the first byte of the associated 482 RTP packet is presented to the user. It is based on the time format 483 used by NTP and consists of the least significant 16 bits of the NTP 484 seconds part and the most significant 16 bits of the NTP fractional 485 second part. If this field is empty, then it SHALL be set to 0 and 486 the Packet Presented NTP timestamp flag (P) SHALL be set to 0. 487 Presented here means the moment the data is played out to the user of 488 the system, i.e. sound played out through speakers, video images 489 being displayed on some display, etc. The accuracy resulting from 490 the synchronization algorithm will only be as good as the accuracy 491 with which the SCs can determine the delay between receiving packets 492 and presenting them to the end-user. 494 8. RTCP Packet Type for IDMS (IDMS Settings) 496 This section specifies the RTCP Packet Type for indicating 497 synchronization settings instructions to the receivers of the RTP 498 media stream. Its definition is based on [RFC3550]. 500 0 1 2 3 501 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 502 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 503 |V=2|P| Resrv | PT=TBD | length | 504 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 505 | SSRC of packet sender | 506 +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ 507 | SSRC of media source | 508 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 509 | Media Stream Correlation Identifier | 510 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 511 | Packet Received NTP timestamp, most significant word | 512 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 513 | Packet Received NTP timestamp, least significant word | 514 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 515 | Packet Received RTP timestamp | 516 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 517 | Packet Presented NTP timestamp, most significant word | 518 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 519 | Packet Presented NTP timestamp, least significant word | 520 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 522 The first 64 bits form the header of the RTCP Packet Type, as defined 523 in [RFC3550]. The SSRC of packet sender identifies the sender of the 524 specific RTCP packet. 526 The RTCP IDMS packet consists of 7 32-bit words, with the following 527 fields: 529 PT: To be determined upon registration with IANA, inserted by the RFC 530 Editor upon registration with IANA. 532 SSRC: 32 bits. The SSRC of the media source SHALL be set to the 533 value of the SSRC identifier of the media source carried in the RTP 534 header [RFC3550] of the RTP packet to which the RTCP IDMS packet 535 relates. 537 Media Stream Correlation Identifier: 32 bits. This identifier is 538 used to correlate synchronized media streams. The value 0 (all bits 539 are set "0") indicates that this field is empty. The value 2^32-1 540 (all bits are set "1") is reserved for future use. The Media Stream 541 Correlation Identifier contains the SyncGroupId of the group to which 542 this packet is sent. 544 Packet Received NTP timestamp: 64 bits. This timestamp reflects the 545 wall clock time at the reference client at the moment it received the 546 first octet of the RTP packet to which this packet relates. It can 547 be used by the synchronization algorithm on the receiving SC to 548 adjust its playout timing in order to achieve synchronization, e.g. 549 to set the required playout delay. The timestamp is formatted based 550 on the NTP timestamp format as specified in [RFC5905]. See Section 9 551 for more information on how this field is used. Because RTP 552 timestamps do wrap around, the sender of this packet SHOULD use 553 recent values, i.e. choose NTP timestamps that reflect current time 554 and not too far in the future or in the past. 556 Packet Received RTP timestamp: 32 bits. This timestamp has the value 557 of the RTP timestamp carried in the RTP header [RFC3550] of the RTP 558 packet to which the XR IDMS relates. This SHOULD relate to the first 559 arriving RTP packet containing this particular RTP timestamp, in case 560 multiple RTP packets contain the same RTP timestamp. 562 Packet Presented NTP timestamp: 64 bits. This timestamp reflects the 563 wall clock time at the reference client at the moment it presented 564 the rendered media unit (e.g. video frame or audio sample) contained 565 in the first octet of the associated RTP packet to the user. The 566 timestamp is formatted based on the NTP timestamp format as specified 567 in [RFC5905]. If this field is empty, then it SHALL be set to 0. 568 This field MAY be left empty if none or only one of the receivers 569 reported on presentation timestamps. Presented here means the moment 570 the data is played out to the user of the system. 572 In some use cases (e.g. phased array transducers), the level of 573 control an MSAS might need to have over the exact moment of playout 574 is so precise that a 32bit Presented Timestamp will not suffice. For 575 this reason, this RTCP Packet Type for IDMS includes a 64bit 576 Presented Timestamp field. Since an MSAS will in practice always add 577 some extra delay to the delay reported by the most lagged receiver 578 (to account for packet jitter), it suffices for the IDMS XR Block 579 Type with which the SCs report on their playout to have a 32bit 580 Presented Timestamp field. 582 9. Timing and NTP Considerations 584 To achieve IDMS, the different receivers involved need synchronized 585 (wall) clocks as a common timeline for synchronization. This 586 synchronized clock is used for reporting the Packet Received NTP 587 Timestamps and the Packet Presented NTP Timestamp, and for 588 interpretation of these fields in received IDMS reports. Depending 589 on the synchronization accuracy required, different clock 590 synchronization methods can be used. For social TV, synchronization 591 accuracy should be achieved on the order of hundreds of milliseconds. 592 In that case, correct use of NTP on receivers will in most situations 593 achieve the required accuracy. As a guideline, to deal with clock 594 drift of receivers, receivers should synchronize their clocks at the 595 beginning of a synchronized session. In case of high required 596 accuracy, the synchronized clocks of different receivers should not 597 drift beyond the accuracy required for the synchronization mechanism. 598 In practice, this can mean that receivers need to synchronize their 599 clocks repeatedly during a synchronization session. 601 Because of the stringent synchronization requirements for achieving 602 good audio quality in some use cases, a high accuracy will be needed. 603 In this case, use of the global NTP system may not be sufficient. 604 For improved accuracy, a local NTP server could be set up, or some 605 other more accurate clock synchronization mechanism can be used, such 606 as GPS time or the Precision Time Protocol [IEEE-1588]. 608 [I-D.draft-ietf-avtcore-clksrc] defines a set of SDP parameters for 609 signaling the clock synchronization source or sources available to 610 and used by the individual receivers. SCs MAY use this draft to 611 indicate their clock synchronization source or sources in use and 612 available. Using these parameters, an SC can indicate which 613 synchronization source is being used at the moment, the last time the 614 SC synchronized with this source and the synchronization frequency. 615 An SC can also indicate any other synchronization sources available 616 to it. This allows multiple SCs in an IDMS session to use the same 617 or a similar clock source for their session. 619 Applications performing IDMS may or may not be able to choose a 620 synchronization method for the system clock, because this may be a 621 system-wide setting which the application cannot change. How 622 applications deal with this is up to the implementation. The 623 application might control the system clock, or it might use a 624 separate application clock or even a separate IDMS session clock. It 625 might also report on the system clock and the synchronization method 626 used, without being able to change it. 628 [I-D.draft-ietf-leap-seconds] presents some guidelines on how RTP 629 senders and receivers should deal with leap seconds. When relying on 630 NTP for clock synchronization, IDMS is particularly sensitive to leap 631 second induced timing discrepancies. It is RECOMMENDED to take the 632 guidelines specified in [I-D.draft-ietf-leap-seconds] into account 633 when implementing IDMS. 635 10. On the use of presentation timestamps 637 A receiver can report on different timing events, i.e. on packet 638 arrival times and on playout or presentation times. A receiver SHALL 639 report on arrival times and a receiver MAY report on playout times. 640 RTP packet arrival times are relatively easy to report on. Normally, 641 the processing and playout of the same media stream by different 642 receivers will take roughly the same amount of time. Synchronizing 643 on packet arrival times, may lead to some accuracy loss, but it will 644 be adequate for many applications, such as social TV. 646 Also, if the receivers are in some way controlled, e.g. having the 647 same buffer settings, decoding and rendering times, high accuracy can 648 be achieved. However, if all receivers in a synchronization session 649 have the ability to report on, and thus synchronize on, actual 650 playout times, or packet presentation times, this may be more 651 accurate. It is up to applications and implementations of this RTCP 652 extension whether to implement and use this. 654 11. SDP Signalling for RTCP IDMS Packet Type 656 The SDP attribute rtcp-idms is used to signal the use of the RTCP 657 IDMS Packet Type for IDMS and the associated RTCP XR Block for IDMS. 658 It is also used to carry an identifier of the synchronization group 659 to which clients belong or will belong. The SDP attribute is used as 660 a media-level attribute during session setup. This means that in 661 case of multiple related streams, IDMS is performed on one of them. 662 The other streams will be synchronized to this reference or master 663 stream using existing inter-stream synchronization (i.e. lip-sync) 664 solutions, i.e. using Sender Reports based on a common clock source. 665 Basic guidelines for choosing the media stream for IDMS is to choose 666 audio above video, as humans are more sensitive to degradation in 667 audio quality than in video quality. When using multi-description or 668 multi-view codecs, the IDMS control should be performed on the base 669 layer. 671 This SDP attribute is defined as follows, using Augmented Backus-Naur 672 Form [RFC5234]. 674 rtcp-idms = "a=" "rtcp-idms" ":" [sync-grp] CRLF 676 sync-grp = "sync-group=" SyncGroupId 678 SyncGroupId = 1*10DIGIT ; Numerical value from 0 through 4294967294 680 DIGIT = %x30-39 682 SyncGroupId is a 32-bit unsigned integer and represented in decimal. 683 SyncGroupId identifies a group of SCs for IDMS. The value 684 SyncGroupId=0 represents an empty SyncGroupId. The value 4294967295 685 (2^32-1) is reserved for future use. For a description on the value 686 of SyncGroupId to include, see Section 12. 688 The following is an example of the SDP attribute for IDMS. 690 a=rtcp-idms:sync-group=42 692 12. SDP rules 694 12.1. Offer/Answer rules 696 The SDP usage for IDMS follows the rules defined in [RFC4566] and 697 section 5 of [RFC3611] on SDP signalling, with the exception of what 698 is stated here. The IDMS usage of RTCP is a (loosely coupled) 699 collaborative attribute, in the sense that receivers send their 700 status information and, in response, the MSAS (asynchronously) sends 701 synchronization setting instructions. The rtcp-idms attribute thus 702 indicates the ability to send and receive indicated RTCP messages. 703 This section defines how this SDP attribute should be used with 704 regard to an offer/answer context. 706 It is expected that, in most cases, the rtcp-idms attribute will be 707 used in an offer/answer context where receivers will have pre- 708 determined, through some means outside the scope of this document, a 709 SyncGroupId before the media session is setup. However, it is also 710 supported that the MSAS assigns such a SyncGroupId, for example if 711 the MSAS contains group management functionality. Thus, both the 712 MSAS and the SCs can insert the attribute and the SyncGroupId. 713 Furthermore, it is allowed to insert the attribute for more than one 714 media stream, allowing an SC to become part of multiple 715 synchronization groups simultaneously. This effectively couples two 716 (or more) synchronization groups to each other. If the rtcp-idms 717 attribute is inserted more than once for a particular media session, 718 each SyncGroupId SHALL only be inserted once. 720 In order to join an IDMS session, the receiver (the SC) inserts the 721 rtcp-idms attribute as a media level attribute in the SDP offer. 722 This SDP offer can be an initial offer, if the media session is 723 starting as a synchronized session. The SDP offer can also be an 724 update to an existing media session, converting the session to an 725 IDMS session. If the receiver has a pre-determined SyncGroupId 726 value, it SHOULD use this value for setting the SyncGroupId parameter 727 in the rtcp-idms attribute. If the receiver does not know the 728 SyncGroupId to be used, it MAY leave the SyncGroupId parameter empty 729 by setting its value to 0. 731 The MSAS SHALL include the rtcp-idms attribute in its answer. If the 732 value of the SyncGroupId parameter in the offer was not empty (not 733 equal to 0), the MSAS SHOULD NOT change the SyncGroupId in its 734 answer. If the SyncGroupId was empty, the MSAS SHALL include the 735 proper SyncGroupId in its answer. If the MSAS receives an offer with 736 the value of the SyncGroupId parameter set to 0, and cannot determine 737 the proper SyncGroupId, it SHALL remove the attribute from its 738 answer. 740 An MSAS receiving an SDP offer without the rtcp-idms attribute can 741 also decide that IDMS is applicable to that media session. In such a 742 case, the MSAS MAY insert the rtcp-idms attribute, including a non- 743 empty SyncGroupId, as part of its answer. 745 A receiver receiving an rtcp-idms attribute as part of the SDP answer 746 from an MSAS, SHALL start sending IDMS XR reports (following all the 747 normal RTCP rules for sending RTCP XR blocks) and SHALL be ready to 748 start receiving IDMS Settings. As usual, if a receiver does not 749 support the attribute (e.g. in case of an MSAS-inserted IDMS 750 attribute), it SHALL ignore the attribute. 752 Different updates are applicable to such an IDMS session. Updates 753 can be sent ommitting the rtcp-idms attribute, thereby ending the 754 (involvement in) the synchronization session. Updates can also be 755 sent including the rtcp-idms attribute, but with a different 756 SyncGroupId. This indicates a switch in synchronization group. 757 Updates can also be sent including another rtcp-idms attribute, 758 indicating a membership of another synchronization group, effectively 759 merging the current group(s) with the new one. 761 12.2. Declarative cases 762 In certain situations, there is no offer/answer context, but only a 763 declarative modus. In this case, the MSAS just inserts the rtcp-idms 764 attribute and a valid SyncGroupId. Any receiver receiving the rtcp- 765 idms attribute in such a declarative case SHALL start sending IDMS XR 766 Report Blocks and SHALL be ready to start receiving RTCP IDMS 767 Settings packets. 769 13. Security Considerations 771 The security considerations described in [RFC3611] apply to this 772 document as well. 774 The RTCP XR IDMS Report Block defined in this document is used to 775 collect, summarize and distribute information on packet reception- 776 and playout-times of streaming media. The information may be used to 777 orchestrate the media playout at multiple devices. 779 Errors in the information, either accidental or malicious, may lead 780 to undesired behavior. For example, if one device erroneously or 781 maliciously reports a two-hour delayed playout, then another device 782 in the same synchronization group could decide to delay its playout 783 by two hours as well, in order to keep its playout synchronized. A 784 user would likely interpret this two hour delay as a malfunctioning 785 service. 787 Therefore, the application logic of both SCs and MSASs should check 788 for inconsistent information. Differences in playout time exceeding 789 configured limits (e.g. more than ten seconds) could be an indication 790 of such inconsistent information. 792 No new mechanisms are introduced in this document to ensure 793 confidentiality. Encryption procedures, such as those being 794 suggested for a Secure RTP (SRTP) at the time that this document was 795 written, can be used when confidentiality is a concern to end hosts. 797 14. IANA Considerations 799 This document defines a new RTCP packet type, the RTCP IDMS Packet 800 (IDMS Settings), within the existing Internet Assigned Numbers 801 Authority (IANA) registry of RTCP Control Packet Types. This 802 document also defines a new RTCP XR Block Type, the IDMS XR Report 803 Block, within the existing IANA registry of RTCP Extended Reports 804 (RTCP XR) Block Types. 806 Further, this document defines a new SDP attribute "rtcp-idms" within 807 the existing IANA registry of SDP Parameters, part of the "att-field 808 (media level only)". Finally, this document defines a new IANA 809 registry subordinate to the IANA RTCP Extended Reports (RTCP XR) 810 Block Type Registry: the IDMS XR Block SPST Registry. 812 14.1. RTCP IDMS Packet Type 814 This document assigns the packet type value TBD in the IANA 'RTCP 815 Control Packet types (PT) Registry' to the RTCP IDMS Packet Type. 817 [Note to RFC Editor: please replace TBD with the IANA-provided RTCP 818 Packet Type value for this packet type] 820 14.2. RTCP XR IDMS Report Block 822 This document updates the assignment of value 12 from the RTCP XR 823 Block Type for reporting IDMS information as per [TS183063] to the 824 RTCP XR IDMS Report Block defined in this document. 826 [Note to RFC Editor: this block type value is currently assigned to 827 [TS183063]. This document replaces [TS183063] as the normative 828 specification of the RTCP XR IDMS Report Block. Upon publication of 829 this document as RFC, [TS183063] will be changed to reflect this. 831 The RTCP XR IDMS Report Block contains an extensible SPST value field 832 and therefore a new registry for this field is required. This new 833 registry is defined in Section 14.4. 835 14.3. RTCP-IDMS SDP Attribute 837 The SDP attribute "rtcp-idms" defined by this document is registered 838 with the IANA registry of SDP Parameters as follows: 840 SDP Attribute ("att-field"): 842 Attribute name: rtcp-idms 844 Long form: RTCP IDMS Parameters 846 Type of name: att-field 848 Type of attribute: media level 850 Subject to charset: no 851 Purpose: see Section 11 of this document 853 Reference: this document 855 Values: see this document 857 14.4. IDMS XR Block SPST Registry 859 This document defines a new IANA registry subordinate to the IANA 860 RTCP Extended Reports (RTCP XR) Block Type Registry: the IDMS XR 861 Block SPST Registry. 863 Initial values for the Timestamp Reference Clock Source Parameters 864 registry are given below; future assignments are to be made through 865 the Specification Required policy [RFC5226]. 867 Value Name Reference 868 ----- ---- --------- 869 1 Synchronization Client section 7 871 14.5. Contact Information for Registrations 873 The contact information for the registrations is: 875 Ray van Brandenburg (ray.vanbrandenburg@tno.nl) 877 Brassersplein 2 879 2612CT, Delft, The Netherlands 881 15. Contributors 883 The following people have participated as co-authors or provided 884 substantial contributions to this document: Omar Niamut, Fabian 885 Walraven, Ishan Vaishnavi and Rufael Mekuria. In addition, the 886 authors would like to thank Aidan Williams, Colin Perkins, Magnus 887 Westerlund, Roni Even, Peter Musgrave, Ali Begen, Qin Wu and Rob 888 Koenen for their review comments and contributions to the text. 890 16. References 892 16.1. Normative References 894 [I-D.draft-ietf-avtcore-clksrc] 895 Williams, A., Gross, K., van Brandenburg, R., and H. 896 Stokking, "RTP Clock Source Signalling, draft-ietf- 897 avtcore-clksrc-05", May 2013. 899 [RFC2119] Bradner, S., "Key Words for use in RFCs to Indicate 900 Requirement Levels, RFC 2119", March 1997. 902 [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. 903 Jacobson, "RTP: A Transport Protocol for Real-Time 904 Applications, RFC3550", July 2003. 906 [RFC3551] Schulzrinne, H. and S. Casner, "RTP Profile for Audio and 907 Video conferences with Minimal Control, RFC3551", July 908 2003. 910 [RFC3611] Friedman, T., Ed., Caceres, R., Ed., and A. Clark, Ed., 911 "RTP Control Protocol Extended Reports (RTCP XR), 912 RFC3611", November 2003. 914 [RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session 915 Description Protocol, RFC4566", July 2006. 917 [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an 918 IANA Considerations Section in RFCs", May 2008. 920 [RFC5234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax 921 Specifications, RFC5234", January 2008. 923 [RFC5905] Mills, D., Martin, J., Ed., Burbank, J., and W. Kasch, 924 "Network Time Protocol Version 4: Protocol and Algorithms 925 Specifications, RFC5905", February 2010. 927 16.2. Informative References 929 [Boronat2009] 930 Boronat, F., Lloret, J., and M. Garcia, "Multimedia group 931 and inter-stream synchronization techniques: a comparative 932 study, Elsevier Information Systems 34 (2009), pp. 933 108-131", . 935 [I-D.draft-ietf-leap-seconds] 936 Gross, K. and R. Brandenburg, van, "RTP and Leap Seconds, 937 draft-ietf-avtcore-leap-second-02", October 2012. 939 [IEEE-1588] 940 , "1588-2008 - IEEE Standard for a Precision Clock 941 Synchronization Protocol for Networked Measurement and 942 Control Systems", 2008. 944 [Ishibashi2006] 945 Ishibashi, Y., Nagasaka, M., and N. Fujiyoshi, "Subjective 946 Assessment of Fairness among users in multipoint 947 communications, Proceedings of the 2006 ACM SIGCHI 948 internation conference on Advances in computer 949 entertainment technology, 2006", . 951 [RFC5868] Ott, J. and C. Perkins, "Guidelines for Extending the RTP 952 Control Protocol (RTCP), RFC5968", September 2010. 954 [TS183063] 955 , "IMS-based IPTV stage 3 specification, TS 183 063 956 v3.5.2", March 2011. 958 Authors' Addresses 960 Ray van Brandenburg 961 TNO 962 Brassersplein 2 963 Delft 2612CT 964 the Netherlands 966 Phone: +31-88-866-7000 967 Email: ray.vanbrandenburg@tno.nl 969 Hans Stokking 970 TNO 971 Brassersplein 2 972 Delft 2612CT 973 the Netherlands 975 Phone: +31-88-866-7000 976 Email: hans.stokking@tno.nl 978 M. Oskar van Deventer 979 TNO 980 Brassersplein 2 981 Delft 2612CT 982 the Netherlands 984 Phone: +31-88-866-7000 985 Email: oskar.vandeventer@tno.nl 986 Fernando Boronat 987 Universitat Politecnica de Valencia 988 Universitat Politecnica de Valencia (UPV) 989 Valencia 46730 990 Spain 992 Phone: +34 962 849 341 993 Email: fboronat@dcom.upv.es 995 Mario Montagud 996 Universitat Politecnica de Valencia 997 Universitat Politecnica de Valencia (UPV) 998 Valencia 46730 999 Spain 1001 Phone: +34 962 849 341 1002 Email: mamontor@posgrado.upv.es 1004 Kevin Gross 1005 AVA Networks 1007 Phone: +1-303-447-0517 1008 Email: Kevin.Gross@AVAnw.com