idnits 2.17.1 draft-ietf-avtcore-idms-07.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 3 instances of too long lines in the document, the longest one being 71 characters in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (October 11, 2012) is 4209 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Outdated reference: A later version (-01) exists of draft-williams-avtcore-clksrc-00 ** Obsolete normative reference: RFC 4566 (Obsoleted by RFC 8866) -- Possible downref: Non-RFC (?) normative reference: ref. 'TS183063' -- No information found for draft-gross-leap-seconds - is the name correct? Summary: 2 errors (**), 0 flaws (~~), 2 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 AVTCore R. van Brandenburg 3 Internet-Draft H. Stokking 4 Intended status: Standards Track O. van Deventer 5 Expires: April 14, 2013 TNO 6 F. Boronat 7 M. Montagud 8 Universitat Politecnica de 9 Valencia 10 K. Gross 11 AVA Networks 12 October 11, 2012 14 Inter-destination Media Synchronization using the RTP Control Protocol 15 (RTCP) 16 draft-ietf-avtcore-idms-07 18 Abstract 20 This document gives information on an RTP Control Protocol (RTCP) 21 Packet Type and RTCP Extended Report (XR) Block Type including 22 associated Session Description Protocol (SDP) parameters for Inter- 23 Destination Media Synchronization (IDMS). This document mandates the 24 use of the RTCP XR Block Type 12, which is used to collect media 25 playout information from participants in a group playing out 26 (watching, listening, etc.) a specific RTP media stream. This RTCP 27 XR Block Type is specified and registered with IANA by ETSI. The 28 RTCP packet type specified by this document is used to distribute a 29 common target playout point to which all the distributed receivers, 30 sharing a media experience, can synchronize. 32 Typical use cases in which IDMS is usefull are social TV, shared 33 service control (i.e. applications where two or more geographically 34 separated users are watching a media stream together), distance 35 learning, networked video walls, networked loudspeakers, etc. 37 Status of this Memo 39 This Internet-Draft is submitted in full conformance with the 40 provisions of BCP 78 and BCP 79. 42 Internet-Drafts are working documents of the Internet Engineering 43 Task Force (IETF). Note that other groups may also distribute 44 working documents as Internet-Drafts. The list of current Internet- 45 Drafts is at http://datatracker.ietf.org/drafts/current/. 47 Internet-Drafts are draft documents valid for a maximum of six months 48 and may be updated, replaced, or obsoleted by other documents at any 49 time. It is inappropriate to use Internet-Drafts as reference 50 material or to cite them other than as "work in progress." 52 This Internet-Draft will expire on April 14, 2013. 54 Copyright Notice 56 Copyright (c) 2012 IETF Trust and the persons identified as the 57 document authors. All rights reserved. 59 This document is subject to BCP 78 and the IETF Trust's Legal 60 Provisions Relating to IETF Documents 61 (http://trustee.ietf.org/license-info) in effect on the date of 62 publication of this document. Please review these documents 63 carefully, as they describe your rights and restrictions with respect 64 to this document. Code Components extracted from this document must 65 include Simplified BSD License text as described in Section 4.e of 66 the Trust Legal Provisions and are provided without warranty as 67 described in the Simplified BSD License. 69 Table of Contents 71 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 72 2. Rationale . . . . . . . . . . . . . . . . . . . . . . . . . . 4 73 2.1. Applicability of RTCP to IDMS . . . . . . . . . . . . . . 4 74 2.2. Applicability of SDP to IDMS . . . . . . . . . . . . . . . 5 75 2.3. This document and ETSI TISPAN . . . . . . . . . . . . . . 5 76 3. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 77 4. Overview of IDMS operation . . . . . . . . . . . . . . . . . . 5 78 5. Inter-Destination Media Synchronization use cases . . . . . . 7 79 6. Architecture for Inter-Destination Media Synchronization . . . 8 80 6.1. Media Synchronization Application Server (MSAS) . . . . . 9 81 6.2. Synchronization Client (SC) . . . . . . . . . . . . . . . 9 82 6.3. Communication between MSAS and SCs . . . . . . . . . . . . 9 83 7. RTCP XR Block Type for IDMS . . . . . . . . . . . . . . . . . 9 84 8. RTCP Packet Type for IDMS (IDMS Settings) . . . . . . . . . . 12 85 9. Timing and NTP Considerations . . . . . . . . . . . . . . . . 14 86 10. SDP Parameter for RTCP XR IDMS Block Type . . . . . . . . . . 15 87 11. SDP Parameter for RTCP IDMS Packet Type . . . . . . . . . . . 16 88 12. Compatibility with ETSI TISPAN . . . . . . . . . . . . . . . . 17 89 13. SDP rules . . . . . . . . . . . . . . . . . . . . . . . . . . 17 90 13.1. Offer/Answer rules . . . . . . . . . . . . . . . . . . . . 17 91 13.2. Declarative cases . . . . . . . . . . . . . . . . . . . . 19 92 14. On the use of presentation timestamps . . . . . . . . . . . . 19 93 15. Security Considerations . . . . . . . . . . . . . . . . . . . 20 94 16. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20 95 17. Contributors . . . . . . . . . . . . . . . . . . . . . . . . . 21 96 18. References . . . . . . . . . . . . . . . . . . . . . . . . . . 21 97 18.1. Normative References . . . . . . . . . . . . . . . . . . . 21 98 18.2. Informative References . . . . . . . . . . . . . . . . . . 22 99 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 22 101 1. Introduction 103 Inter-Destination Media Synchronization (IDMS) refers to the playout 104 of media streams at two or more geographically distributed locations 105 in a time synchronized manner. It can be applied to both unicast and 106 multicast media streams and can be applied to any type and/or 107 combination of streaming media, such as audio, video and text 108 (subtitles).[Ishibashi2006] and [Boronat2009] provide an overview of 109 technologies and algorithms for IDMS. 111 IDMS requires the exchange of information on media receipt and 112 playout times among participants in an IDMS session. It may also 113 require signaling for the initiation and maintenance of IDMS sessions 114 and groups of receivers. 116 The presented RTCP specification for IDMS is independent of the used 117 synchronization algorithm, which is out-of-scope of this document. 119 2. Rationale 121 2.1. Applicability of RTCP to IDMS 123 Currently, a large share of real-time applications make use of RTP 124 and RTCP [RFC3550]. RTP provides end-to-end network transport 125 functions suitable for applications requiring real-time data 126 transport, such as audio, video or data, over multicast or unicast 127 network services. The timestamps, sequence numbers, and payload 128 (content) type identification mechanisms provided by RTP packets are 129 very useful for reconstructing the original media timing, and for 130 reordering and detecting packet loss at the client side. 132 The data transport is augmented by a control protocol (RTCP) to allow 133 monitoring of the data delivery in a manner that is scalable to large 134 groups, and to provide minimal control and identification 135 functionality. RTP receivers and senders provide reception quality 136 feedback by sending out RTCP Receiver Report (RR) and Sender Report 137 (SR) packets [RFC3550], respectively, which may be augmented by 138 eXtended Reports (XR) [RFC3611]. Both RTP and RTCP are intended to 139 be tailored through modifications in order to include profile- 140 specific information required by particular applications, and the 141 guidelines on doing so are specified in [RFC5868]. 143 IDMS involves the collection, summarizing and distribution of RTP 144 packet arrival and playout times. As information on RTP packet 145 arrival times and playout times can be considered reception quality 146 feedback information, RTCP is well suited for carrying out IDMS, 147 which may facilitate the implementation and deployment in typical 148 multimedia applications. 150 2.2. Applicability of SDP to IDMS 152 RTCP XR [RFC3611] defines the Extended Report (XR) packet type for 153 the RTP Control Protocol (RTCP), and defines how the use of XR 154 packets can be signaled by an application using the Session 155 Description Protocol (SDP) [RFC4566]. 157 SDP signaling is used to set up and maintain a synchronization group 158 between Synchronization Clients (SCs). This document describes two 159 SDP parameters for doing this, one for the RTCP XR block type and one 160 for the new RTCP packet type. 162 2.3. This document and ETSI TISPAN 164 ETSI TISPAN [TS183063] has specified architecture and protocol for 165 IDMS using RTCP XR exchange and SDP signaling. For more information 166 on how this document relates to [TS183063], see Section 12. 168 This document mandates the use of an ETSI specified RTCP XR block for 169 the purpose of collecting RTP packet arrival and playout times. For 170 completeness, that XR block is contained in this document in section 171 7. 173 3. Terminology 175 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 176 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 177 document are to be interpreted as described in RFC 2119 [RFC2119] and 178 indicate requirement levels for compliant implementations. 180 4. Overview of IDMS operation 182 This section provides a brief example of how the RTCP functionality 183 is used for achieving IDMS. The section is tutorial in nature and 184 does not contain any normative statements. 186 Alice's . . . . . . .tv:abc.com . . . . . . . . . Bob's 187 TV (Sync Client) (Sync Server) Laptop (Sync Client) 188 | | | 189 | Media Session | | 190 |<=====================>| | 191 | Invite(URL,Sync-group ID) | 192 |------------------------------------------------->| 193 | | Media Session Set-up | 194 | |<========================>| 195 | | | 196 | Call set-up | 197 |<================================================>| 198 | | | 199 | RTP Packet | RTP Packet | 200 |<----------------------|------------------------->| 201 | RR + IDMS XR | | 202 |---------------------->| RR + IDMS XR | 203 | |<-------------------------| 204 | RTCP IDMS packet | RTCP IDMS packet | 205 |<----------------------|------------------------->| 206 | | | 208 Figure 1: Example of a typical IDMS session 210 Alice is watching TV in her living room. At some point she sees that 211 a football game of Bob's favorite team is on. She sends him an 212 invite to watch the program together. Embedded in the invitation is 213 the link to the media server and a unique sync-group identifier. 215 Bob, who is also at home, receives the invite on his laptop. He 216 accepts Alice's invitation and the RTP client on his laptop sets up a 217 session to the media server. A VoIP connection to Alice's TV is also 218 set up, so that Alice and Bob can talk while watching the game 219 together. 221 As is common with RTP, both the RTP client in Alice's TV as well as 222 the one in Bob's laptop send periodic RTCP Receiver Reports (RR) to 223 the media server. However, in order to make sure Alice and Bob see 224 the events in the football game at (approximately) the same time, 225 their clients also periodically send an IDMS XR block to the sync 226 server function of the media server. Included in the XR blocks are 227 timestamps on when both Alice and Bob received (and, optionally, when 228 they played out) a particular RTP packet. 230 The sync server function in the media server calculates a reference 231 client from the received IDMS XR blocks (e.g. by selecting whichever 232 client received the packet the latest as the reference client). It 233 then sends an RTCP IDMS packet containing the playout information of 234 this reference client to the sync clients of both Alice and Bob. 236 In this case Bob's connection has the longest delay and the reference 237 client therefore includes a delay similar to the one experienced by 238 Bob. Upon reception of this information, Alice's RTP client can 239 choose what to do with this information. In this case it decreases 240 its playout rate temporarily until the playout time matches with the 241 reference client playout (and thus matches Bob's playout). Another 242 option for Alice's TV would be to simply pause playback until it 243 catches up. The exact implementation of the synchronization 244 algorithm is up to the client. 246 Upon reception of the reference client RTCP IDMS packet, Bob's client 247 does not have to do anything since it is already synchronized to the 248 reference client (since it is based on Bob's delay). Note that other 249 synchronization algorithms may introduce even more delay than the one 250 experienced by the most delayed client, e.g. to account for delay 251 variations, for new clients joining an existing synchronization 252 group, etc. 254 For this functionality to work correctly, it is nessecary that the 255 wall clocks of the receivers are synchronized with each other. Alice 256 and Bob both report when they receive, and optionally when they play 257 out, certain RTP packets. In order to correlate their reports to 258 each other, it is necessary that their wallclocks are synchronized. 260 5. Inter-Destination Media Synchronization use cases 262 There are a large number of use cases in which IDMS might be useful. 263 This section will highlight some of them. It should be noted that 264 this section is in no way meant to be exhaustive. 266 A first usage scenario for IDMS is Social TV. Social TV is the 267 combination of media content consumption by two or more users at 268 different devices and locations combined with the real-time 269 communication between those users. An example of Social TV is when 270 two or more users are watching the same television broadcast at 271 different devices and locations, while communicating with each other 272 using text, audio and/or video. A skew in their media playout 273 processes can have adverse effects on their experience. A well-known 274 use case here is one friend experiencing a goal in a football match 275 well before or after other friend(s). 277 Another potential use case for IDMS is a networked video wall. A 278 video wall consists of multiple computer monitors, video projectors, 279 or television sets tiled together contiguously or overlapped in order 280 to form one large screen. Each of the screens reproduces a portion 281 of the larger picture. In some implementations, each screen may be 282 individually connected to the network and receive its portion of the 283 overall image from a network-connected video server or video scaler. 284 Screens are refreshed at 60 hertz (every 16-2/3 milliseconds) or 285 potentially faster. If the refresh is not synchronized, the effect 286 of multiple screens acting as one is broken. 288 A third usage scenario is that of the networked loudspeakers, in 289 which two or more speakers are connected to the network individually. 290 Such situations can for example be found in large conference rooms, 291 legislative chambers, classrooms (especially those supporting 292 distance learning) and other large-scale environments such as 293 stadiums. Since humans are more susceptible to differences in audio 294 delay, this use case needs even more accuracy than the video wall use 295 case. Depending on the exact application, the need for accuracy can 296 then be in the range of microseconds. 298 6. Architecture for Inter-Destination Media Synchronization 300 The architecture for IDMS, which is based on a sync-maestro 301 architecture [Boronat2009], is sketched below. The Synchronization 302 Client (SC) and Media Synchronization Application Server (MSAS) 303 entities are shown as additional functionality for the RTP receiver 304 and sender respectively. 306 It should be noted that a master/slave type of architecture is also 307 supported by having one of the SC devices also act as an MSAS. In 308 this case the MSAS functionality is thus embedded in an RTP receiver 309 instead of an RTP sender. 311 +-----------------------+ +-----------------------+ 312 | | SR + | | 313 | RTP Receiver | RTCP | RTP Sender | 314 | | IDMS | | 315 | +-----------------+ | <----- | +-----------------+ | 316 | | | | | | | | 317 | | Synchronization | | | | Media | | 318 | | Client | | | | Synchronization | | 319 | | (SC) | | | | Application | | 320 | | | | | | Server | | 321 | | | | RR+XR | | (MSAS) | | 322 | | | | -----> | | | | 323 | +-----------------+ | | +-----------------+ | 324 | | | | 325 +-----------------------+ +-----------------------+ 327 IDMS Architecture Diagram 329 6.1. Media Synchronization Application Server (MSAS) 331 An MSAS collects RTP packet arrival times and playout times from one 332 or more SC(s) in a synchronization group. The MSAS summarizes and 333 distributes this information to the SCs in the synchronization group 334 as synchronization settings, e.g. by determining the SC with the most 335 lagged playout and using its reported RTP packet arrival time and 336 playout time as a summary. 338 6.2. Synchronization Client (SC) 340 An SC reports on RTP packet arrival times and playout times of a 341 media stream. It can receive summaries of such information, and use 342 that to adjust its playout buffer. 344 6.3. Communication between MSAS and SCs 346 Two different message types are used for the communication between 347 MSAS and SCs. For the SC->MSAS message containing the playout 348 information of a particular client, an RTCP XR Block Type is used 349 (see Section 6). For the MSAS->SC message containing the 350 synchronization settings instructions, a new RTCP Packet Type is 351 defined (see Section 8). 353 7. RTCP XR Block Type for IDMS 355 For reporting IDMS information, SCs SHALL use the ETSI specified RTCP 356 XR Block Type 12. For completeness, that RTCP XR Block Type is 357 repeated here fully. 359 The definition of the XR block type is based on [RFC3611]. The RTCP 360 XR is used to provide feedback information on receipt times and 361 presentation times of RTP packets to e.g. a Sender [RFC3611], a 362 Feedback Target [RFC5760] or a Third Party Monitor [RFC3611]. 364 In most cases, a single RTP receiver will only be part of single IDMS 365 session, i.e. it will report on receipt and presentation times of RTP 366 packets from a single RTP stream in a certain synchronization group. 367 In some cases however, an RTP receiver may be a member of multiple 368 synchronization groups for the same RTP stream, e.g. watching a 369 single television program simultaneously with different groups. In 370 even further cases, a receiver may wish to synchronize different RTP 371 streams at the same time, either as part of the same synchronization 372 group or as part of multiple synchronization groups. These are all 373 valid scenario's for IDMS, and will require multiple reports by an 374 SC. 376 SCs SHOULD report on a recently received RTP packets. This document 377 does not define new rules on when to sent RTCP reports, but uses the 378 existing rules specified in [RFC3550] for sending RTCP reports. When 379 the RTCP reporting timer allows an SC to send an IDMS report, the SC 380 SHOULD report on an RTP packet received during the period since the 381 last IDMS XR Report Block was sent. For more details on which packet 382 to report on, see below under 'Packet Received RTP timestamp'. 384 0 1 2 3 385 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 386 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 387 |V=2|P| Resrv | PT=XR=207 | length | 388 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 389 | SSRC of packet sender | 390 +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ 391 | BT=12 | SPST |Resrv|P| block length=7 | 392 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 393 | PT | Resrv | 394 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 395 | Media Stream Correlation Identifier | 396 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 397 | SSRC of media source | 398 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 399 | Packet Received NTP timestamp, most significant word | 400 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 401 | Packet Received NTP timestamp, least significant word | 402 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 403 | Packet Received RTP timestamp | 404 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 405 | Packet Presented NTP timestamp | 406 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 408 The first 64 bits form the header of the RTCP XR, as defined in 409 [RFC3611]. The SSRC of packet sender identifies the sender of the 410 specific RTCP packet. 412 The IDMS report block consists of 8 32-bit words, with the following 413 fields: 415 Block Type (BT): 8 bits. It identifies the block format. Its value 416 SHALL be set to 12. 418 Synchronization Packet Sender Type (SPST): 4 bits. This field 419 identifies the role of the packet sender for this specific eXtended 420 Report. It can have the following values: 422 SPST=0 Reserved For future use. 424 SPST=1 The packet sender is an SC. It uses this XR to report 425 synchronization status information. Timestamps relate to the SC 426 input. 428 SPST=2 This setting is reserved in order to preserve compatibility 429 with ETSI TISPAN [TS183063] and is used when the packet sender is 430 an MSAS (see Section 12 for when to use this SPST). 432 SPST=3-4 These settings are reserved in order to preserve 433 compatibility with ETSI TISPAN [TS183063]. 435 SPST=5-15 Reserved for future use. 437 Reserved bits (Resrv): 3 bits. These bits are reserved for future 438 definition. In the absence of such a definition, the bits in this 439 field MUST be set to zero and MUST be ignored by the receiver. 441 Packet Presented NTP timestamp flag (P): 1 bit. Bit set to 1 if the 442 Packet Presented NTP timestamp field contains a value, 0 if it is 443 empty. If this flag is set to zero, then the Packet Presented NTP 444 timestamp SHALL be ignored. 446 Block Length: 16 bits. This field indicates the length of the block 447 in 32 bit words minus one and SHALL be set to 7, as this RTCP Block 448 Type has a fixed length. 450 Payload Type (PT): 7 bits. This field identifies the format of the 451 media payload, according to [RFC3551]. This is the payload type of 452 the RTP packet reported upon. The media payload is associated with 453 an RTP timestamp clock rate. This clock rate provides the time base 454 for the RTP timestamp counter.This clock rate is nessecary for the 455 MSAS to relate reports from different SCs on different RTP timestamp 456 values. 458 Reserved bits (Resrv): 25 bits. These bits are reserved for future 459 use and SHALL be set to 0. 461 Media Stream Correlation Identifier: 32 bits. This identifier is 462 used to correlate synchronized media streams. The value 0 (all bits 463 are set "0") indicates that this field is empty. The value 2^32-1 464 (all bits are set "1") is reserved for future use. If the RTCP 465 Packet Sender is an SC (SPST=1), then the Media Stream Correlation 466 Identifier field contains the Synchronization Group Identifier 467 (SyncGroupId) to which the report applies. 469 SSRC: 32 bits. The SSRC of the media source SHALL be set to the 470 value of the SSRC identifier carried in the RTP header [RFC3550] of 471 the RTP packet to which the XR relates. 473 Packet Received NTP timestamp: 64 bits. This timestamp reflects the 474 wall clock time at the moment of arrival of the first octet of the 475 RTP packet to which the XR relates. It is formatted based on the NTP 476 timestamp format as specified in [RFC5905]. See Section 9 for more 477 information on how this field is used. 479 Packet Received RTP timestamp: 32 bits. This timestamp has the value 480 of the RTP timestamp carried in the RTP header [RFC3550] of the RTP 481 packet to which the XR relates. Several consecutive RTP packets will 482 have equal timestamps if they are (logically) generated at once, 483 e.g., belong to the same video frame. It may well be the case that 484 one receiver reports on the first RTP packet having a certain RTP 485 timestamp and a second receiver reports on the last RTP packet having 486 that same RTP timestamp. This would lead to an error in the 487 synchronization algorithm due to the faulty interpretation of 488 considering both reports to be on the same RTP packet. When 489 reporting on an RTP packet which is one of several consecutive RTP 490 packets having equal timestamps, an SC SHOULD report on the RTP 491 packet it received with the lowest sequence number. Note that with 492 'lowest sequence number' here is meant the first in the sequence of 493 RTP packets just received, not from an earlier time before the last 494 wrap-around of RTP timestamps (unless this wrap-around occurs during 495 the sequence with equal RTP timestamps). 497 Packet Presented NTP timestamp: 32 bits. This timestamp reflects the 498 wall clock time at the moment the rendered frame contained in the 499 first byte of the associated RTP packet is presented to the user. It 500 is based on the time format used by NTP and consists of the least 501 significant 16 bits of the NTP seconds part and the most significant 502 16 bits of the NTP fractional second part. If this field is empty, 503 then it SHALL be set to 0 and the Packet Presented NTP timestamp flag 504 (P) SHALL be set to 0. Presented here means the moment the data is 505 played out to the user of the system, i.e. sound played out through 506 speakers, video images being displayed on some display, etc. The 507 accuracy resulting from the synchronization algorithm will only be as 508 good as the accuracy with which the receivers can determine the delay 509 between receiving packets and presenting them to the end-user. 511 8. RTCP Packet Type for IDMS (IDMS Settings) 513 This section specifies the RTCP Packet Type for indicating 514 synchronization settings instructions to the receivers of the RTP 515 media stream. Its definition is based on [RFC3550]. 517 0 1 2 3 518 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 519 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 520 |V=2|P| Resrv | PT=TBD | length | 521 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 522 | SSRC of packet sender | 523 +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ 524 | SSRC of media source | 525 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 526 | Media Stream Correlation Identifier | 527 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 528 | Packet Received NTP timestamp, most significant word | 529 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 530 | Packet Received NTP timestamp, least significant word | 531 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 532 | Packet Received RTP timestamp | 533 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 534 | Packet Presented NTP timestamp, most significant word | 535 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 536 | Packet Presented NTP timestamp, least significant word | 537 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 539 The first 64 bits form the header of the RTCP Packet Type, as defined 540 in [RFC3550]. The SSRC of packet sender identifies the sender of the 541 specific RTCP packet. 543 The RTCP IDMS packet consists of 7 32-bit words, with the following 544 fields: 546 PT: To be determined upon registration with IANA, inserted by the RFC 547 Editor upon registration with IANA. 549 SSRC: 32 bits. The SSRC of the media source SHALL be set to the 550 value of the SSRC identifier of the media source carried in the RTP 551 header [RFC3550] of the RTP packet to which the RTCP IDMS packet 552 relates. 554 Media Stream Correlation Identifier: 32 bits. This identifier is 555 used to correlate synchronized media streams. The value 0 (all bits 556 are set "0") indicates that this field is empty. The value 2^32-1 557 (all bits are set "1") is reserved for future use. The Media Stream 558 Correlation Identifier contains the SyncGroupId of the group to which 559 this packet is sent. 561 Packet Received NTP timestamp: 64 bits. This timestamp reflects the 562 wall clock time at the reference client at the moment it received the 563 first octet of the RTP packet to which this packet relates. It can 564 be used by the synchronization algorithm on the receiving SC to 565 adjust its playout timing in order to achieve synchronization, e.g. 566 to set the required playout delay. The timestamp is formatted based 567 on the NTP timestamp format as specified in [RFC5905]. See Section 9 568 for more information on how this field is used. Because RTP 569 timestamps do wrap around, the sender of this packet SHOULD use 570 recent values, i.e. choose NTP timestamps that reflect current time 571 and not too far in the future or in the past. 573 Packet Received RTP timestamp: 32 bits. This timestamp has the value 574 of the RTP timestamp carried in the RTP header [RFC3550] of the RTP 575 packet to which the XR relates. This SHOULD relate to the first 576 arriving RTP packet containing this particular RTP timestamp, in case 577 multiple RTP packets contain the same RTP timestamp. 579 Packet Presented NTP timestamp: 64 bits. This timestamp reflects the 580 wall clock time at the reference client at the moment it presented 581 the rendered frame contained in the first octet of the associated RTP 582 packet to the user. The timestamp is formatted based on the NTP 583 timestamp format as specified in [RFC5905]. If this field is empty, 584 then it SHALL be set to 0. This field MAY be left empty if none or 585 only one of the receivers reported on presentation timestamps. 586 Presented here means the moment the data is played out to the user of 587 the system. 589 In some use cases (e.g. phased array transducers), the level of 590 control an MSAS might need to have over the exact moment of playout 591 is so precise that a 32bit Presented Timestamp will not suffice. For 592 this reason, this RTCP Packet Type for IDMS includes a 64bit 593 Presented Timestamp field. Since an MSAS will in practice always add 594 some extra delay to the delay reported by the most lagged receiver 595 (to account for packet jitter), it suffices for the IDMS XR Block 596 Type with which the SCs report on their playout to have a 32bit 597 Presented Timestamp field. 599 9. Timing and NTP Considerations 601 To achieve IDMS, the different receivers involved need synchronized 602 (wall) clocks as a common timeline for synchronization. This 603 synchronized clock is used for reporting the Packet Received NTP 604 Timestamps and the Packet Presented NTP Timestamp, and for 605 interpretation of these fields in received IDMS reports. Depending 606 on the synchronization accuracy required, different clock 607 synchronization methods can be used. For social TV, synchronization 608 accuracy should be achieved on the order of hundreds of milliseconds. 609 In that case, correct use of NTP on receivers will in most situations 610 achieve the required accuracy. As a guideline, to deal with clock 611 drift of receivers, receivers should synchronize their clocks at the 612 beginning of a synchronized session. In case of high required 613 accuracy, the synchronized clocks of different receivers should not 614 drift beyond the accuracy required for the synchronization mechanism. 615 In practice, this can mean that receivers need to synchronize their 616 clocks repeatedly during a synchronization session. 618 Because of the stringent synchronization requirements for achieving 619 good audio in some use cases, a high accuracy will be needed. In 620 this case, use of the global NTP system may not be sufficient. For 621 improved accuracy, a local NTP server could be set up, or some other 622 more accurate clock synchronization mechanism can be used, such as 623 GPS time or the Precision Time Protocol [IEEE-1588]. 625 [I-D.draft-williams-avtcore-clksrc] defines a set of SDP parameters 626 for signaling the clock synchronization source or sources available 627 to and used by the individual receivers. SCs MAY use this draft to 628 indicate their clock synchronization source or sourced in use and 629 available. Using these paramenters, an SC can indicate which 630 synchronization source is being used at the moment, the last time the 631 SC synchronized with this source and the synchronization frequency. 632 An SC can also indicate any other synchronization sources available 633 to it. This allows multiple SCs in an IDMS session to use the same 634 or a similar clock source for their session. 636 Applications performing IDMS may or may not be able to choose a 637 synchronization method for the system clock, because this may be a 638 system-wide setting which the application cannot change. How 639 applications deal with this is up to the implementation. The 640 application might control the system clock, or it might use a 641 separate application clock or even a separate IDMS session clock. It 642 might also report on the system clock and the synchronization method 643 used, without being able to change it. 645 [I-D.draft-gross-leap-second] presents some guidelines on how RTP 646 senders and receivers should deal with leap seconds. When relying on 647 NTP for clock synchronization, IDMS is particularly sensitive to leap 648 second induced timing discrepancies. It is RECOMMENDED to take the 649 guideline specified in [I-D.draft-gross-leap-second] into account 650 when implementing IDMS. 652 10. SDP Parameter for RTCP XR IDMS Block Type 654 The SDP parameter sync-group is used to signal the use of the RTCP XR 655 block for IDMS, as specified by ETSI, included here for completeness. 656 It is also used to carry an identifier of the synchronization group 657 to which clients belong or will belong. This SDP parameter extends 658 rtcp-xr-attrib as follows, using Augmented Backus-Naur Form 660 [RFC5234]. 662 rtcp-xr-attrib = "a=" "rtcp-xr" ":" [xr-format *(SP xr-format)] CRLF 663 ; Original definition from [RFC3611], section 5.1 665 xr-format =/ grp-sync ; Extending xr-format for Inter-Destination 666 Media Synchronization 668 grp-sync = "grp-sync" [",sync-group=" SyncGroupId] 670 SyncGroupId = 1*10DIGIT ; Numerical value from 0 through 4294967294 672 DIGIT = %x30-39 674 SyncGroupId is a 32-bit unsigned integer represented in decimal. 675 SyncGroupId identifies a group of SCs for IDMS. It maps on the Media 676 Stream Correlation Identifier as described in Section 7 and 677 Section 8. The value SyncGroupId=0 represents an empty SyncGroupId. 678 The value 4294967294 (2^32-1) is reserved for future use. 680 The following is an example of the SDP attribute for IDMS 682 a=rtcp-xr:grp-sync,sync-group=42 684 11. SDP Parameter for RTCP IDMS Packet Type 686 The SDP parameter rtcp-idms is used to signal the use of the RTCP 687 IDMS Packet Type for IDMS. It is also used to carry an identifier of 688 the synchronization group to which clients belong or will belong. 689 The SDP parameter is used as a media-level attribute during session 690 setup. This means that in case of multiple related streams, IDMS is 691 performed on one of them. The other streams will be synchronized to 692 this first stream using existing inter-stream synchronization (i.e. 693 lip-sync) solutions, i.e. using Sender Reports based on a common 694 clock source. Basic guidelines for choosing the media stream for 695 IDMS is to choose audio above video, as humans are more sensitive to 696 degradation in audio quality then in video quality. When using 697 mutli-description or multi-view codecs, the IDMS control should be 698 performed on the base layer. 700 This SDP parameter is defined as follows, using Augmented Backus-Naur 701 Form [RFC5234]. 703 rtcp-idms = "a=" "rtcp-idms" ":" [sync-grp] CRLF 705 sync-grp = "sync-group=" SyncGroupId 706 SyncGroupId = 1*10DIGIT ; Numerical value from 0 through 4294967294 708 DIGIT = %x30-39 710 SyncGroupId is a 32-bit unsigned integer and represented in decimal. 711 SyncGroupId identifies a group of SCs for IDMS. The value 712 SyncGroupId=0 represents an empty SyncGroupId. The value 4294967294 713 (2^32-1) is reserved for future use. 715 The following is an example of the SDP attribute for IDMS. 717 a=rtcp-idms:sync-group=42 719 12. Compatibility with ETSI TISPAN 721 As described in Section 2.3, ETSI TISPAN has described its mechanism 722 for IDMS in [TS183063]. One of the main differences between the 723 TISPAN document and this document is the fact that the TISPAN 724 solution uses an RTCP XR block for both the SC->MSAS message and the 725 MSAS->SC message (by selecting SPST-type 2), while this document 726 specifies a new RTCP Packet Type for the MSAS->SC message. The 727 message from MSAS to SC is not in any way a report on how a receiver 728 sees a session, and therefore a separate RTCP packet type is more 729 appropriate than the XR block solution chosen in ETSI TISPAN. To 730 achieve compatibility, MSAS implementations SHOULD implement both the 731 TISPAN RTCP block and the new RTCP IDMS Settings packet for MSAS->SC 732 messages. SCs MAY implement support for both types of messages. For 733 the MSAS->SC signaling, it is recommended to use the RTCP IDMS 734 Settings packet defined in this document. The TISPAN RTCP XR block 735 with SPST=2 MAY be used for purposes of compatibility with the TISPAN 736 solution, but MUST NOT be used if all nodes involved support the new 737 RTCP IDMS Settings packet. 739 13. SDP rules 741 13.1. Offer/Answer rules 743 The SDP usage for IDMS follows the rules defined in RFC3611 in 744 section 5 on SDP signalling, with the exception of what is stated 745 here. The IDMS usage of RTCP is a (loosely coupled) collaborative 746 parameter, in the sense that receivers sent their status information 747 and in response (asynchronously) the MSAS sents synchronization 748 instructions. Both the sync-group parameter (defined by ETSI TISPAN) 749 and the rtcp-idms parameter (defined in this document) thus indicate 750 the ability to sent and the ability to receive indicated RTCP 751 messages. This section defines how these SDP parameters should be 752 used. 754 Most of the times, the IDMS SDP parameters will be used in the offer/ 755 answer context. Receivers will indicate in their SDP which RTCP 756 messages they support. 758 For a unicast situation, three situations are possible in offer/ 759 answer context: 761 - If a receiver indicates at least the rtcp-idms SDP parameter, the 762 MSAS SHOULD reply with only the rtcp-idms parameter and use only 763 the RTCP IDMS Settings packet for MSAS->SC communication 765 - If a receiver indicates only the sync-group SDP parameter, and the 766 MSAS also supports this, it SHOULD reply with only the sync-group 767 parameter and use only the RTCP XR block with SPST=2 for MSAS->SC 768 communication 770 - If a receiver indicates only the sync-group SDP parameter, and the 771 MSAS does not support this, the media sender MUST ignore the 772 parameter. This receiver will not become part of the 773 synchronization session 775 Note that it is possible that for a certain synchronization group, 776 that the MSAS sends RTCP IDMS packets to one receiver and RTCP XR 777 IDMS blocks with SPST=2 to another receiver. 779 In a multicast situation using the offer/answer context, it will work 780 a bit differently. The negotiation is the same as in the unicast 781 situation. But, the MSAS will multicast all RTCP messages to all 782 receivers. So: 784 - If all receivers support the RTCP IDMS Settings packet, the MSAS 785 SHOULD only sent the RTCP IDMS Settings packet for MSAS->SC 786 messages 788 - If not all receivers support the RTCP IDMS Settings packet, but 789 all receivers support the TISPAN solution, the MSAS SHOULD only 790 sent the RTCP XR block with SPST=2 for MSAS->SC messages. 792 - If some receivers support only the RTCP IDMS Settings packet and 793 other receivers support only the TISPAN solution, the MSAS SHOULD 794 sent both the RTCP IDMS Settings packet and the RTCP XR block with 795 SPST=2 for MSAS->SC messages. This is less efficient, since the 796 information sent is duplicated, but this is the only way to 797 include all receivers in a synchronization session in this 798 scenario. 800 In certain multicast situations, there is no offer/answer context, 801 but only a declarative modus. In that case, the MSAS SHOULD use only 802 the RTCP IDMS packet type and thus use only the SDP parameter rtcp- 803 idms. Receivers that do not support the RTCP IDMS packet will just 804 ignore both the SDP parameter and the RTCP IDMS packets, and will 805 thus not join the synchronization session. For compatability with 806 the TISPAN solution, the MSAS MAY choose to use the RTCP XR IDMS 807 block type instead, using the SDP parameter sync-group. The media 808 sender SHOULD NOT use both parameters at the same time in this case 809 of no offer/answer context. 811 13.2. Declarative cases 813 In certain multicast situations, there is no offer/answer context, 814 but only a declarative modus. In that case, the MSAS SHOULD use only 815 the RTCP IDMS packet type and thus use only the SDP parameter rtcp- 816 idms. Receivers that do not support the RTCP IDMS packet will just 817 ignore both the SDP parameter and the RTCP IDMS packets, and will 818 thus not join the synchronization session. For compatability with 819 the TISPAN solution, the MSAS MAY choose to use the RTCP XR IDMS 820 block type instead, using the SDP parameter sync-group. The media 821 sender SHOULD NOT use both parameters at the same time in this case 822 of no offer/answer context. 824 14. On the use of presentation timestamps 826 A receiver can report on different timing events, i.e. on packet 827 arrival times and on playout times. A receiver SHALL report on 828 arrival times and a receiver MAY report on playout times. RTP packet 829 arrival times are relatively easy to report on. Normally, the 830 processing and playout of the same media stream by different 831 receivers will take roughly the same amount of time. Synchronizing 832 on packet arrival times, may lead to some accuracy loss, but it will 833 be adequate for many applications, such as social TV. 835 Also, if the receivers are in some way controlled, e.g. having the 836 same buffer settings and decoding times, high accuracy can be 837 achieved. However, if all receivers in a synchronization session 838 have the ability to report on, and thus synchronize on, actual 839 playout times, or packet presentation times, this may be more 840 accurate. It is up to applications and implementations of this RTCP 841 extension whether to implement and use this. 843 15. Security Considerations 845 The security considerations described in [RFC3611] apply to this 846 document as well. 848 The specified RTCP XR Block Type in this document is used to collect, 849 summarize and distribute information on packet reception- and 850 playout-times of streaming media. The information may be used to 851 orchestrate the media playout at multiple devices. 853 Errors in the information, either accidental or malicious, may lead 854 to undesired behavior. For example, if one device erroneously 855 reports a two-hour delayed playout, then another device in the same 856 synchronization group could decide to delay its playout by two hours 857 as well, in order to keep its playout synchronized. A user would 858 likely interpret this two hour delay as a malfunctioning service. 860 Therefore, the application logic of both Synchronization Clients and 861 Media Synchronization Application Servers should check for 862 inconsistent information. Differences in playout time exceeding 863 configured limits (e.g. more than ten seconds) could be an indication 864 of such inconsistent information. 866 No new mechanisms are introduced in this document to ensure 867 confidentiality. Encryption procedures, such as those being 868 suggested for a Secure RTP (SRTP) at the time that this document was 869 written, can be used when confidentiality is a concern to end hosts. 871 16. IANA Considerations 873 This document defines a new RTCP packet type called IDMS Settings 874 packet in the IANA registry of RTP parameters, part of RTCP Control 875 Packet types (PT), based on the specification in Section 8. 877 Further, this document defines a new SDP parameter "rtcp-idms" within 878 the existing IANA registry of SDP Parameters, part of the "att-field 879 (media level only)". 881 The SDP attribute "rtcp-idms" defined by this document is registered 882 with the IANA registry of SDP Parameters as follows: 884 SDP Attribute ("att-field"): 886 Attribute name: rtcp-idms 888 Long form: RTCP report block for IDMS 890 Type of name: att-field 892 Type of attribute: media level 894 Subject to charset: no 896 Purpose: see sections 7 and 10 of this document 898 Reference: this document 900 Values: see this document 902 17. Contributors 904 The following people have participated as co-authors or provided 905 substantial contributions to this document: Omar Niamut, Fabian 906 Walraven, Ishan Vaishnavi, Rufael Mekuria and Rob Koenen. 908 18. References 910 18.1. Normative References 912 [I-D.draft-williams-avtcore-clksrc] 913 Williams, A., van Brandenburg, R., Stokking, H., and K. 914 Gross, "RTP Clock Source Signalling, 915 draft-williams-avtcore-clksrc-00", March 2012. 917 [RFC2119] Bradner, S., "Key Words for use in RFCs to Indicate 918 Requirement Levels, RFC 2119", March 1997. 920 [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. 921 Jacobson, "RTP: A Transport Protocol for Real-Time 922 Applications, RFC3550", July 2003. 924 [RFC3551] Schulzrinne, H. and S. Casner, "RTP Profile for Audio and 925 Video conferences with Minimal Control, RFC3551", 926 July 2003. 928 [RFC3611] Friedman, T., Ed., Caceres, R., Ed., and A. Clark, Ed., 929 "RTP Control Protocol Extended Reports (RTCP XR), 930 RFC3611", November 2003. 932 [RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session 933 Description Protocol, RFC4566", July 2006. 935 [RFC5234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax 936 Specifications, RFC5234", January 2008. 938 [RFC5760] Ott, J., Chesterfield, J., and E. Schooler, "RTP Control 939 Protocol (RTCP) Extensions for Single-Source Multicast 940 Sessions with Unicast Feedback, RFC5760", February 2010. 942 [RFC5905] Mills, D., Martin, J., Ed., Burbank, J., and W. Kasch, 943 "Network Time Protocol Version 4: Protocol and Algorithms 944 Specifications, RFC5905", February 2010. 946 [TS183063] 947 "IMS-based IPTV stage 3 specification, TS 183 063 v3.4.1", 948 June 2010. 950 18.2. Informative References 952 [Boronat2009] 953 Boronat, F., Lloret, J., and M. Garcia, "Multimedia group 954 and inter-stream synchronization techniques: a comparative 955 study, Elsevier Information Systems 34 (2009), pp. 108- 956 131". 958 [I-D.draft-gross-leap-second] 959 Gross, K. and R. Brandenburg, van, "RTP and Leap Seconds, 960 draft-gross-leap-seconds-01". 962 [IEEE-1588] 963 "1588-2008 - IEEE Standard for a Precision Clock 964 Synchronization Protocol for Networked Measurement and 965 Control Systems", 2008. 967 [Ishibashi2006] 968 Ishibashi, Y., Nagasaka, M., and N. Fujiyoshi, "Subjective 969 Assessment of Fairness among users in multipoint 970 communications, Proceedings of the 2006 ACM SIGCHI 971 internation conference on Advances in computer 972 entertainment technology, 2006". 974 [RFC5868] Ott, J. and C. Perkins, "Guidelines for Extending the RTP 975 Control Protocol (RTCP), RFC5968", September 2010. 977 Authors' Addresses 979 Ray van Brandenburg 980 TNO 981 Brassersplein 2 982 Delft 2612CT 983 the Netherlands 985 Phone: +31-88-866-7000 986 Email: ray.vanbrandenburg@tno.nl 988 Hans Stokking 989 TNO 990 Brassersplein 2 991 Delft 2612CT 992 the Netherlands 994 Phone: +31-88-866-7000 995 Email: hans.stokking@tno.nl 997 M. Oskar van Deventer 998 TNO 999 Brassersplein 2 1000 Delft 2612CT 1001 the Netherlands 1003 Phone: +31-88-866-7000 1004 Email: oskar.vandeventer@tno.nl 1006 Fernando Boronat 1007 Universitat Politecnica de Valencia 1008 IGIC Institute, Universitat Politecnica de Valencia-Campus de Gandia (UPV), C/ Paraninfo, 1, Grao de Gandia, C/ Paraninfo, 1, Grao de Gandia 1009 Valencia 46730 1010 Spain 1012 Phone: +34 962 849 341 1013 Email: fboronat@dcom.upv.es 1014 Mario Montagud 1015 Universitat Politecnica de Valencia 1016 IGIC Institute, Universitat Politecnica de Valencia-Campus de Gandia (UPV), C/ Paraninfo, 1, Grao de Gandia, C/ Paraninfo, 1, Grao de Gandia 1017 Valencia 46730 1018 Spain 1020 Phone: +34 962 849 341 1021 Email: mamontor@posgrado.upv.es 1023 Kevin Gross 1024 AVA Networks 1026 Phone: +1-303-447-0517 1027 Email: Kevin.Gross@AVAnw.com