idnits 2.17.1 draft-ietf-avtcore-idms-08.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 3 instances of too long lines in the document, the longest one being 71 characters in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (January 17, 2013) is 4117 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: 'RFC4566' is defined on line 881, but no explicit reference was found in the text == Unused Reference: 'RFC5760' is defined on line 887, but no explicit reference was found in the text == Outdated reference: A later version (-11) exists of draft-ietf-avtcore-clksrc-01 ** Obsolete normative reference: RFC 4566 (Obsoleted by RFC 8866) -- Possible downref: Non-RFC (?) normative reference: ref. 'TS183063' -- No information found for draft-ietf-leap-seconds - is the name correct? Summary: 2 errors (**), 0 flaws (~~), 4 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 AVTCore R. van Brandenburg 3 Internet-Draft H. Stokking 4 Intended status: Standards Track O. van Deventer 5 Expires: July 21, 2013 TNO 6 F. Boronat 7 M. Montagud 8 Universitat Politecnica de 9 Valencia 10 K. Gross 11 AVA Networks 12 January 17, 2013 14 Inter-destination Media Synchronization using the RTP Control Protocol 15 (RTCP) 16 draft-ietf-avtcore-idms-08 18 Abstract 20 This document defines a new RTP Control Protocol (RTCP) Packet Type 21 and RTCP Extended Report (XR) Block Type to be used for achieving 22 Inter-Destination Media Synchronization (IDMS). IDMS is the process 23 of synchronizing playout across multiple geographically distributed 24 media receivers. Using the RTCP XR IDMS Reporting Block defined in 25 this document, media playout information from participants in a 26 synchronization group can be collected. Based on the collected 27 information, an RTCP IDMS Settings Packet can then be send to 28 distribute a common target playout point to which all the distributed 29 receivers, sharing a media experience, can synchronize. 31 Typical use cases in which IDMS is usefull are social TV, shared 32 service control (i.e. applications where two or more geographically 33 separated users are watching a media stream together), distance 34 learning, networked video walls, networked loudspeakers, etc. 36 Status of this Memo 38 This Internet-Draft is submitted in full conformance with the 39 provisions of BCP 78 and BCP 79. 41 Internet-Drafts are working documents of the Internet Engineering 42 Task Force (IETF). Note that other groups may also distribute 43 working documents as Internet-Drafts. The list of current Internet- 44 Drafts is at http://datatracker.ietf.org/drafts/current/. 46 Internet-Drafts are draft documents valid for a maximum of six months 47 and may be updated, replaced, or obsoleted by other documents at any 48 time. It is inappropriate to use Internet-Drafts as reference 49 material or to cite them other than as "work in progress." 51 This Internet-Draft will expire on July 21, 2013. 53 Copyright Notice 55 Copyright (c) 2013 IETF Trust and the persons identified as the 56 document authors. All rights reserved. 58 This document is subject to BCP 78 and the IETF Trust's Legal 59 Provisions Relating to IETF Documents 60 (http://trustee.ietf.org/license-info) in effect on the date of 61 publication of this document. Please review these documents 62 carefully, as they describe your rights and restrictions with respect 63 to this document. Code Components extracted from this document must 64 include Simplified BSD License text as described in Section 4.e of 65 the Trust Legal Provisions and are provided without warranty as 66 described in the Simplified BSD License. 68 Table of Contents 70 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 71 2. Rationale . . . . . . . . . . . . . . . . . . . . . . . . . . 4 72 2.1. Applicability of RTCP to IDMS . . . . . . . . . . . . . . 4 73 3. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 74 4. Inter-Destination Media Synchronization use cases . . . . . . 5 75 5. Overview of IDMS operation . . . . . . . . . . . . . . . . . . 6 76 6. Architecture for Inter-Destination Media Synchronization . . . 7 77 6.1. Media Synchronization Application Server (MSAS) . . . . . 8 78 6.2. Synchronization Client (SC) . . . . . . . . . . . . . . . 8 79 6.3. Communication between MSAS and SCs . . . . . . . . . . . . 8 80 7. RTCP XR Block for IDMS (IDMS Report Block) . . . . . . . . . . 8 81 8. RTCP Packet Type for IDMS (IDMS Settings) . . . . . . . . . . 12 82 9. Timing and NTP Considerations . . . . . . . . . . . . . . . . 13 83 10. On the use of presentation timestamps . . . . . . . . . . . . 15 84 11. SDP Signalling for RTCP IDMS Packet Type . . . . . . . . . . . 15 85 12. SDP rules . . . . . . . . . . . . . . . . . . . . . . . . . . 16 86 12.1. Offer/Answer rules . . . . . . . . . . . . . . . . . . . . 16 87 12.2. Declarative cases . . . . . . . . . . . . . . . . . . . . 17 88 13. Security Considerations . . . . . . . . . . . . . . . . . . . 17 89 14. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 18 90 14.1. RTCP IDMS Packet Type . . . . . . . . . . . . . . . . . . 18 91 14.2. RTCP XR IDMS Report Block . . . . . . . . . . . . . . . . 18 92 14.3. RTCP-IDMS SDP Attribute . . . . . . . . . . . . . . . . . 19 93 14.4. Contact Information for Registrations . . . . . . . . . . 19 94 15. Contributors . . . . . . . . . . . . . . . . . . . . . . . . . 19 95 16. References . . . . . . . . . . . . . . . . . . . . . . . . . . 20 96 16.1. Normative References . . . . . . . . . . . . . . . . . . . 20 97 16.2. Informative References . . . . . . . . . . . . . . . . . . 21 98 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 21 100 1. Introduction 102 Inter-Destination Media Synchronization (IDMS) refers to the playout 103 of media streams at two or more geographically distributed locations 104 in a time synchronized manner. It can be applied to both unicast and 105 multicast media streams and can be applied to any type and/or 106 combination of streaming media, such as audio, video and text 107 (subtitles).[Ishibashi2006] and [Boronat2009] provide an overview of 108 technologies and algorithms for IDMS. 110 IDMS requires the exchange of information on media receipt and 111 playout times among participants in an IDMS session. It may also 112 require signaling for the initiation and maintenance of IDMS sessions 113 and groups of receivers. 115 The presented RTCP specification for IDMS is independent of the used 116 synchronization algorithm, which is out-of-scope of this document. 118 2. Rationale 120 2.1. Applicability of RTCP to IDMS 122 Currently, a large share of real-time applications make use of RTP 123 and RTCP [RFC3550]. RTP provides end-to-end network transport 124 functions suitable for applications requiring real-time data 125 transport, such as audio, video or data, over multicast or unicast 126 network services. The timestamps, sequence numbers, and payload 127 (content) type identification mechanisms provided by RTP packets are 128 very useful for reconstructing the original media timing, and for 129 reordering and detecting packet loss at the client side. 131 The data transport is augmented by a control protocol (RTCP) to allow 132 monitoring of the data delivery in a manner that is scalable to large 133 groups, and to provide minimal control and identification 134 functionality. RTP receivers and senders provide reception quality 135 feedback by sending out RTCP Receiver Report (RR) and Sender Report 136 (SR) packets [RFC3550], respectively, which may be augmented by 137 eXtended Reports (XR) [RFC3611]. Both RTP and RTCP are intended to 138 be tailored through modifications in order to include profile- 139 specific information required by particular applications, and the 140 guidelines on doing so are specified in [RFC5868]. 142 IDMS involves the collection, summarizing and distribution of RTP 143 packet arrival and playout times. As information on RTP packet 144 arrival times and playout times can be considered reception quality 145 feedback information, RTCP is well suited for carrying out IDMS, 146 which may facilitate the implementation and deployment in typical 147 multimedia applications. 149 3. Terminology 151 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 152 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 153 document are to be interpreted as described in RFC 2119 [RFC2119] and 154 indicate requirement levels for compliant implementations. 156 4. Inter-Destination Media Synchronization use cases 158 There are a large number of use cases in which IDMS might be useful. 159 This section will highlight some of them. It should be noted that 160 this section is in no way meant to be exhaustive. 162 A first usage scenario for IDMS is Social TV. Social TV is the 163 combination of media content consumption by two or more users at 164 different devices and locations combined with the real-time 165 communication between those users. An example of Social TV is when 166 two or more users are watching the same television broadcast at 167 different devices and locations, while communicating with each other 168 using text, audio and/or video. A skew in their media playout 169 processes can have adverse effects on their experience. A well-known 170 use case here is one friend experiencing a goal in a football match 171 well before or after other friend(s). 173 Another potential use case for IDMS is a networked video wall. A 174 video wall consists of multiple computer monitors, video projectors, 175 or television sets tiled together contiguously or overlapped in order 176 to form one large screen. Each of the screens reproduces a portion 177 of the larger picture. In some implementations, each screen may be 178 individually connected to the network and receive its portion of the 179 overall image from a network-connected video server or video scaler. 180 Screens are refreshed at 60 hertz (every 16-2/3 milliseconds) or 181 potentially faster. If the refresh is not synchronized, the effect 182 of multiple screens acting as one is broken. 184 A third usage scenario is that of the networked loudspeakers, in 185 which two or more speakers are connected to the network individually. 186 Such situations can for example be found in large conference rooms, 187 legislative chambers, classrooms (especially those supporting 188 distance learning) and other large-scale environments such as 189 stadiums. Since humans are more susceptible to differences in audio 190 delay, this use case needs even more accuracy than the video wall use 191 case. Depending on the exact application, the need for accuracy can 192 then be in the range of microseconds. 194 5. Overview of IDMS operation 196 This section provides a brief example of how the RTCP functionality 197 is used for achieving IDMS. The section is tutorial in nature and 198 does not contain any normative statements. 200 Alice's . . . . . . .tv:abc.com . . . . . . . . . Bob's 201 TV (Sync Client) (Sync Server) Laptop (Sync Client) 202 | | | 203 | Media Session | | 204 |<=====================>| | 205 | Invite(URL,Sync-group ID) | 206 |------------------------------------------------->| 207 | | Media Session Set-up | 208 | |<========================>| 209 | | | 210 | Call set-up | 211 |<================================================>| 212 | | | 213 | RTP Packet | RTP Packet | 214 |<----------------------|------------------------->| 215 | RR + XR IDMS Report | | 216 |---------------------->| RR + XR IDMS Report | 217 | |<-------------------------| 218 | RTCP IDMS Settings | RTCP IDMS Settings | 219 |<----------------------|------------------------->| 220 | | | 222 Figure 1: Example of a typical IDMS session 224 Alice is watching TV in her living room. At some point she sees that 225 a football game of Bob's favorite team is on. She sends him an 226 invite to watch the program together. Embedded in the invitation is 227 the link to the media server and a unique sync-group identifier. 229 Bob, who is also at home, receives the invite on his laptop. He 230 accepts Alice's invitation and the RTP client on his laptop sets up a 231 session to the media server. A VoIP connection to Alice's TV is also 232 set up, so that Alice and Bob can talk while watching the game 233 together. 235 As is common with RTP, both the RTP client in Alice's TV as well as 236 the one in Bob's laptop send periodic RTCP Receiver Reports (RR) to 237 the media server. However, in order to make sure Alice and Bob see 238 the events in the football game at (approximately) the same time, 239 their clients also periodically send an RTCP XR IDMS Report Block to 240 the sync server function of the media server. Included in the XR 241 blocks are timestamps on when both Alice and Bob received (and, 242 optionally, when they played out) a particular RTP packet. 244 The sync server function in the media server calculates a reference 245 client from the received IDMS Report Blocks (e.g. by selecting 246 whichever client received the packet the latest as the reference 247 client). It then sends an RTCP IDMS Settings packet containing the 248 playout information of this reference client to the sync clients of 249 both Alice and Bob. 251 In this case Bob's connection has the longest delay and the reference 252 client therefore includes a delay similar to the one experienced by 253 Bob. Upon reception of this information, Alice's RTP client can 254 choose what to do with this information. In this case it decreases 255 its playout rate temporarily until the playout time matches with the 256 reference client playout (and thus matches Bob's playout). Another 257 option for Alice's TV would be to simply pause playback until it 258 catches up. The exact implementation of the synchronization 259 algorithm is up to the client. 261 Upon reception of the reference client RTCP IDMS Settings packet, 262 Bob's client does not have to do anything since it is already 263 synchronized to the reference client (since it is based on Bob's 264 delay). Note that other synchronization algorithms may introduce 265 even more delay than the one experienced by the most delayed client, 266 e.g. to account for delay variations, for new clients joining an 267 existing synchronization group, etc. 269 For this functionality to work correctly, it is nessecary that the 270 wall clocks of the receivers are synchronized with each other. Alice 271 and Bob both report when they receive, and optionally when they play 272 out, certain RTP packets. In order to correlate their reports to 273 each other, it is necessary that their wallclocks are synchronized. 275 6. Architecture for Inter-Destination Media Synchronization 277 The architecture for IDMS, which is based on a sync-maestro 278 architecture [Boronat2009], is sketched below. The Synchronization 279 Client (SC) and Media Synchronization Application Server (MSAS) 280 entities are shown as additional functionality for the RTP receiver 281 and sender respectively. 283 It should be noted that a master/slave type of architecture is also 284 supported by having one of the SC devices also act as an MSAS. In 285 this case the MSAS functionality is thus embedded in an RTP receiver 286 instead of an RTP sender. 288 +-----------------------+ +-----------------------+ 289 | | SR + | | 290 | RTP Receiver | RTCP | RTP Sender | 291 | | IDMS | | 292 | +-----------------+ | <----- | +-----------------+ | 293 | | | | | | | | 294 | | Synchronization | | | | Media | | 295 | | Client | | | | Synchronization | | 296 | | (SC) | | | | Application | | 297 | | | | | | Server | | 298 | | | | RR+XR | | (MSAS) | | 299 | | | | -----> | | | | 300 | +-----------------+ | | +-----------------+ | 301 | | | | 302 +-----------------------+ +-----------------------+ 304 IDMS Architecture Diagram 306 6.1. Media Synchronization Application Server (MSAS) 308 An MSAS collects RTP packet arrival times and playout times from one 309 or more SC(s) in a synchronization group. The MSAS summarizes and 310 distributes this information to the SCs in the synchronization group 311 as synchronization settings, e.g. by determining the SC with the most 312 lagged playout and using its reported RTP packet arrival time and 313 playout time as a summary. 315 6.2. Synchronization Client (SC) 317 An SC reports on RTP packet arrival times and playout times of a 318 media stream. It can receive summaries of such information, and use 319 that to adjust its playout buffer. 321 6.3. Communication between MSAS and SCs 323 Two different message types are used for the communication between 324 MSAS and SCs. For the SC->MSAS message containing the playout 325 information of a particular client, an RTCP XR IDMS Report Block used 326 (see Section 7). For the MSAS->SC message containing the 327 synchronization settings instructions, a new RTCP IDMS Settings 328 Packet Type is defined (see Section 8). 330 7. RTCP XR Block for IDMS (IDMS Report Block) 332 This section specifies a new RTCP XR Block Type, the RTCP XR IDMS 333 Report Block, for reporting IDMS information to an MSAS. In 334 particular it is used to provide feedback information on receipt 335 times and presentation times of RTP packets. Its definition is based 336 on [RFC3550] and [RFC3611]. 338 In most cases, a single RTP receiver will only be part of single IDMS 339 session, i.e. it will report on receipt and presentation times of RTP 340 packets from a single RTP stream in a certain synchronization group. 341 In some cases however, an RTP receiver may be a member of multiple 342 synchronization groups for the same RTP stream, e.g. watching a 343 single television program simultaneously with different groups. In 344 even further cases, a receiver may wish to synchronize different RTP 345 streams at the same time, either as part of the same synchronization 346 group or as part of multiple synchronization groups. These are all 347 valid scenario's for IDMS, and will require multiple reports by an 348 SC. 350 SCs SHOULD report on a recently received RTP packets. This document 351 does not define new rules on when to sent RTCP reports, but uses the 352 existing rules specified in [RFC3550] for sending RTCP reports. When 353 the RTCP reporting timer allows an SC to send an IDMS report, the SC 354 SHOULD report on an RTP packet received during the period since the 355 last RTCP XR IDMS Report Block was sent. For more details on which 356 packet to report on, see below under 'Packet Received RTP timestamp'. 358 0 1 2 3 359 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 360 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 361 |V=2|P| Resrv | PT=XR=207 | length | 362 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 363 | SSRC of packet sender | 364 +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ 365 | BT=12 | SPST |Resrv|P| block length=7 | 366 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 367 | PT | Resrv | 368 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 369 | Media Stream Correlation Identifier | 370 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 371 | SSRC of media source | 372 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 373 | Packet Received NTP timestamp, most significant word | 374 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 375 | Packet Received NTP timestamp, least significant word | 376 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 377 | Packet Received RTP timestamp | 378 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 379 | Packet Presented NTP timestamp | 380 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 382 The first 64 bits form the header of the RTCP XR, as defined in 384 [RFC3611]. The SSRC of packet sender identifies the sender of the 385 specific RTCP packet. 387 The IDMS report block consists of 8 32-bit words, with the following 388 fields: 390 Block Type (BT): 8 bits. It identifies the block format. Its value 391 SHALL be set to 12. 393 Synchronization Packet Sender Type (SPST): 4 bits. This field 394 identifies the role of the packet sender for this specific eXtended 395 Report. It can have the following values: 397 SPST=0 Reserved for future use. 399 SPST=1 The packet sender is an SC. It uses this XR to report 400 synchronization status information. Timestamps relate to the SC 401 input. 403 SPST=2-4 Values defined by ETSI TISPAN (see [TS183063]). 405 SPST=5-15 Reserved for future use. 407 Reserved bits (Resrv): 3 bits. These bits are reserved for future 408 definition. In the absence of such a definition, the bits in this 409 field MUST be set to zero and MUST be ignored by the receiver. 411 Packet Presented NTP timestamp flag (P): 1 bit. Bit set to 1 if the 412 Packet Presented NTP timestamp field contains a value, 0 if it is 413 empty. If this flag is set to zero, then the Packet Presented NTP 414 timestamp SHALL be ignored. 416 Block Length: 16 bits. This field indicates the length of the block 417 in 32 bit words minus one and SHALL be set to 7, as this RTCP Block 418 Type has a fixed length. 420 Payload Type (PT): 7 bits. This field identifies the format of the 421 media payload, according to [RFC3551]. This is the payload type of 422 the RTP packet reported upon. The media payload is associated with 423 an RTP timestamp clock rate. This clock rate provides the time base 424 for the RTP timestamp counter.This clock rate is nessecary for the 425 MSAS to relate reports from different SCs on different RTP timestamp 426 values. 428 Reserved bits (Resrv): 25 bits. These bits are reserved for future 429 use and SHALL be set to 0. 431 Media Stream Correlation Identifier: 32 bits. This identifier is 432 used to correlate synchronized media streams. The value 0 (all bits 433 are set "0") indicates that this field is empty. The value 2^32-1 434 (all bits are set "1") is reserved for future use. If the RTCP 435 Packet Sender is an SC (SPST=1), then the Media Stream Correlation 436 Identifier field contains the Synchronization Group Identifier 437 (SyncGroupId) to which the report applies. 439 SSRC: 32 bits. The SSRC of the media source SHALL be set to the 440 value of the SSRC identifier carried in the RTP header [RFC3550] of 441 the RTP packet to which the XR relates. 443 Packet Received NTP timestamp: 64 bits. This timestamp reflects the 444 wall clock time at the moment of arrival of the first octet of the 445 RTP packet to which the XR relates. It is formatted based on the NTP 446 timestamp format as specified in [RFC5905]. See Section 9 for more 447 information on how this field is used. 449 Packet Received RTP timestamp: 32 bits. This timestamp has the value 450 of the RTP timestamp carried in the RTP header [RFC3550] of the RTP 451 packet to which the XR relates. Several consecutive RTP packets will 452 have equal timestamps if they are (logically) generated at once, 453 e.g., belong to the same video frame. It may well be the case that 454 one receiver reports on the first RTP packet having a certain RTP 455 timestamp and a second receiver reports on the last RTP packet having 456 that same RTP timestamp. This would lead to an error in the 457 synchronization algorithm due to the faulty interpretation of 458 considering both reports to be on the same RTP packet. When 459 reporting on an RTP packet which is one of several consecutive RTP 460 packets having equal timestamps, an SC SHOULD report on the RTP 461 packet it received with the lowest sequence number. Note that with 462 'lowest sequence number' here is meant the first in the sequence of 463 RTP packets just received, not from an earlier time before the last 464 wrap-around of RTP timestamps (unless this wrap-around occurs during 465 the sequence with equal RTP timestamps). 467 Packet Presented NTP timestamp: 32 bits. This timestamp reflects the 468 wall clock time at the moment the rendered frame contained in the 469 first byte of the associated RTP packet is presented to the user. It 470 is based on the time format used by NTP and consists of the least 471 significant 16 bits of the NTP seconds part and the most significant 472 16 bits of the NTP fractional second part. If this field is empty, 473 then it SHALL be set to 0 and the Packet Presented NTP timestamp flag 474 (P) SHALL be set to 0. Presented here means the moment the data is 475 played out to the user of the system, i.e. sound played out through 476 speakers, video images being displayed on some display, etc. The 477 accuracy resulting from the synchronization algorithm will only be as 478 good as the accuracy with which the receivers can determine the delay 479 between receiving packets and presenting them to the end-user. 481 8. RTCP Packet Type for IDMS (IDMS Settings) 483 This section specifies the RTCP Packet Type for indicating 484 synchronization settings instructions to the receivers of the RTP 485 media stream. Its definition is based on [RFC3550]. 487 0 1 2 3 488 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 489 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 490 |V=2|P| Resrv | PT=TBD | length | 491 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 492 | SSRC of packet sender | 493 +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ 494 | SSRC of media source | 495 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 496 | Media Stream Correlation Identifier | 497 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 498 | Packet Received NTP timestamp, most significant word | 499 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 500 | Packet Received NTP timestamp, least significant word | 501 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 502 | Packet Received RTP timestamp | 503 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 504 | Packet Presented NTP timestamp, most significant word | 505 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 506 | Packet Presented NTP timestamp, least significant word | 507 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 509 The first 64 bits form the header of the RTCP Packet Type, as defined 510 in [RFC3550]. The SSRC of packet sender identifies the sender of the 511 specific RTCP packet. 513 The RTCP IDMS packet consists of 7 32-bit words, with the following 514 fields: 516 PT: To be determined upon registration with IANA, inserted by the RFC 517 Editor upon registration with IANA. 519 SSRC: 32 bits. The SSRC of the media source SHALL be set to the 520 value of the SSRC identifier of the media source carried in the RTP 521 header [RFC3550] of the RTP packet to which the RTCP IDMS packet 522 relates. 524 Media Stream Correlation Identifier: 32 bits. This identifier is 525 used to correlate synchronized media streams. The value 0 (all bits 526 are set "0") indicates that this field is empty. The value 2^32-1 527 (all bits are set "1") is reserved for future use. The Media Stream 528 Correlation Identifier contains the SyncGroupId of the group to which 529 this packet is sent. 531 Packet Received NTP timestamp: 64 bits. This timestamp reflects the 532 wall clock time at the reference client at the moment it received the 533 first octet of the RTP packet to which this packet relates. It can 534 be used by the synchronization algorithm on the receiving SC to 535 adjust its playout timing in order to achieve synchronization, e.g. 536 to set the required playout delay. The timestamp is formatted based 537 on the NTP timestamp format as specified in [RFC5905]. See Section 9 538 for more information on how this field is used. Because RTP 539 timestamps do wrap around, the sender of this packet SHOULD use 540 recent values, i.e. choose NTP timestamps that reflect current time 541 and not too far in the future or in the past. 543 Packet Received RTP timestamp: 32 bits. This timestamp has the value 544 of the RTP timestamp carried in the RTP header [RFC3550] of the RTP 545 packet to which the XR relates. This SHOULD relate to the first 546 arriving RTP packet containing this particular RTP timestamp, in case 547 multiple RTP packets contain the same RTP timestamp. 549 Packet Presented NTP timestamp: 64 bits. This timestamp reflects the 550 wall clock time at the reference client at the moment it presented 551 the rendered frame contained in the first octet of the associated RTP 552 packet to the user. The timestamp is formatted based on the NTP 553 timestamp format as specified in [RFC5905]. If this field is empty, 554 then it SHALL be set to 0. This field MAY be left empty if none or 555 only one of the receivers reported on presentation timestamps. 556 Presented here means the moment the data is played out to the user of 557 the system. 559 In some use cases (e.g. phased array transducers), the level of 560 control an MSAS might need to have over the exact moment of playout 561 is so precise that a 32bit Presented Timestamp will not suffice. For 562 this reason, this RTCP Packet Type for IDMS includes a 64bit 563 Presented Timestamp field. Since an MSAS will in practice always add 564 some extra delay to the delay reported by the most lagged receiver 565 (to account for packet jitter), it suffices for the IDMS XR Block 566 Type with which the SCs report on their playout to have a 32bit 567 Presented Timestamp field. 569 9. Timing and NTP Considerations 571 To achieve IDMS, the different receivers involved need synchronized 572 (wall) clocks as a common timeline for synchronization. This 573 synchronized clock is used for reporting the Packet Received NTP 574 Timestamps and the Packet Presented NTP Timestamp, and for 575 interpretation of these fields in received IDMS reports. Depending 576 on the synchronization accuracy required, different clock 577 synchronization methods can be used. For social TV, synchronization 578 accuracy should be achieved on the order of hundreds of milliseconds. 579 In that case, correct use of NTP on receivers will in most situations 580 achieve the required accuracy. As a guideline, to deal with clock 581 drift of receivers, receivers should synchronize their clocks at the 582 beginning of a synchronized session. In case of high required 583 accuracy, the synchronized clocks of different receivers should not 584 drift beyond the accuracy required for the synchronization mechanism. 585 In practice, this can mean that receivers need to synchronize their 586 clocks repeatedly during a synchronization session. 588 Because of the stringent synchronization requirements for achieving 589 good audio in some use cases, a high accuracy will be needed. In 590 this case, use of the global NTP system may not be sufficient. For 591 improved accuracy, a local NTP server could be set up, or some other 592 more accurate clock synchronization mechanism can be used, such as 593 GPS time or the Precision Time Protocol [IEEE-1588]. 595 [I-D.draft-ietf-avtcore-clksrc] defines a set of SDP parameters for 596 signaling the clock synchronization source or sources available to 597 and used by the individual receivers. SCs MAY use this draft to 598 indicate their clock synchronization source or sourced in use and 599 available. Using these paramenters, an SC can indicate which 600 synchronization source is being used at the moment, the last time the 601 SC synchronized with this source and the synchronization frequency. 602 An SC can also indicate any other synchronization sources available 603 to it. This allows multiple SCs in an IDMS session to use the same 604 or a similar clock source for their session. 606 Applications performing IDMS may or may not be able to choose a 607 synchronization method for the system clock, because this may be a 608 system-wide setting which the application cannot change. How 609 applications deal with this is up to the implementation. The 610 application might control the system clock, or it might use a 611 separate application clock or even a separate IDMS session clock. It 612 might also report on the system clock and the synchronization method 613 used, without being able to change it. 615 [I-D.draft-ietf-leap-seconds] presents some guidelines on how RTP 616 senders and receivers should deal with leap seconds. When relying on 617 NTP for clock synchronization, IDMS is particularly sensitive to leap 618 second induced timing discrepancies. It is RECOMMENDED to take the 619 guideline specified in [I-D.draft-ietf-leap-seconds] into account 620 when implementing IDMS. 622 10. On the use of presentation timestamps 624 A receiver can report on different timing events, i.e. on packet 625 arrival times and on playout times. A receiver SHALL report on 626 arrival times and a receiver MAY report on playout times. RTP packet 627 arrival times are relatively easy to report on. Normally, the 628 processing and playout of the same media stream by different 629 receivers will take roughly the same amount of time. Synchronizing 630 on packet arrival times, may lead to some accuracy loss, but it will 631 be adequate for many applications, such as social TV. 633 Also, if the receivers are in some way controlled, e.g. having the 634 same buffer settings and decoding times, high accuracy can be 635 achieved. However, if all receivers in a synchronization session 636 have the ability to report on, and thus synchronize on, actual 637 playout times, or packet presentation times, this may be more 638 accurate. It is up to applications and implementations of this RTCP 639 extension whether to implement and use this. 641 11. SDP Signalling for RTCP IDMS Packet Type 643 The SDP attribute rtcp-idms is used to signal the use of the RTCP 644 IDMS Packet Type for IDMS and the associated RTCP XR Block for IDMS. 645 It is also used to carry an identifier of the synchronization group 646 to which clients belong or will belong. The SDP attribute is used as 647 a media-level attribute during session setup. This means that in 648 case of multiple related streams, IDMS is performed on one of them. 649 The other streams will be synchronized to this first stream using 650 existing inter-stream synchronization (i.e. lip-sync) solutions, i.e. 651 using Sender Reports based on a common clock source. Basic 652 guidelines for choosing the media stream for IDMS is to choose audio 653 above video, as humans are more sensitive to degradation in audio 654 quality then in video quality. When using mutli-description or 655 multi-view codecs, the IDMS control should be performed on the base 656 layer. 658 This SDP attribute is defined as follows, using Augmented Backus-Naur 659 Form [RFC5234]. 661 rtcp-idms = "a=" "rtcp-idms" ":" [sync-grp] CRLF 663 sync-grp = "sync-group=" SyncGroupId 665 SyncGroupId = 1*10DIGIT ; Numerical value from 0 through 4294967294 667 DIGIT = %x30-39 668 SyncGroupId is a 32-bit unsigned integer and represented in decimal. 669 SyncGroupId identifies a group of SCs for IDMS. The value 670 SyncGroupId=0 represents an empty SyncGroupId. The value 4294967294 671 (2^32-1) is reserved for future use. For a description on the value 672 of SyncGroupId to include, see Section 12. 674 The following is an example of the SDP attribute for IDMS. 676 a=rtcp-idms:sync-group=42 678 12. SDP rules 680 12.1. Offer/Answer rules 682 The SDP usage for IDMS follows the rules defined in RFC3611 in 683 section 5 on SDP signalling, with the exception of what is stated 684 here. The IDMS usage of RTCP is a (loosely coupled) collaborative 685 attribute, in the sense that receivers sent their status information 686 and in response the MSAS (asynchronously) sends synchronization 687 instructions. The rtcp-idms attribute thus indicates the ability to 688 send and receive indicated RTCP messages. This section defines how 689 this SDP attribute should be used with regards to offer/answer. 691 It is expected that in most cases, the rtcp-idms attribute will be 692 used in an offer/answer context where receivers will have pre- 693 determined, through some means outside the scope of this document, a 694 SyncGroupId before the media session is setup. However, it is also 695 supported that the MSAS assigns such a SyncGroupId, for example if 696 the MSAS contains group management functionality. Thus, both the 697 MSAS and the SC can insert the attribute and the SyncGroupId. 698 Furthermore, it is allowed to insert the attribute for more than one 699 media stream, allowing an SC to become part of multiple 700 synchronization groups simultaneously. This effectively couples two 701 (or more) synchronization groups to each other. If the rtcp-idms 702 attribute is inserted more than once for a particular media session, 703 each SyncGroupId SHALL only be inserted once. 705 In order to join an IDMS session, the receiver (the SC) inserts the 706 rtcp-idms attribute as a media level attribute in the SDP offer. 707 This SDP offer can be an initial offer, if the media session is 708 starting as a synchronized session. The SDP offer can also be an 709 update to an existing media session, converting the session to an 710 IMDS session. If the receiver has a pre-determined SyncGroupId 711 value, it SHOULD use this value for setting the SyncGroupId parameter 712 in the rtcp-idms attribute. If the receiver does not know the 713 SyncGroupId to be used, it MAY leave the SyncGroupId parameter empty 714 by setting its value to 0. 716 The MSAS SHALL include the rtcp-idms attribute in its answer. If the 717 value of the SyncGroupId parameter in the offer was not empty (not 718 equal to 0), the MSAS SHOULD NOT change the SyncGroupId in its 719 answer. If the SyncGroupId was empty, the MSAS SHALL include the 720 proper SyncGroupId in its answer. If the MSAS receives an offer with 721 the value of the SyncGroupId parameter set to 0, and cannot determine 722 the proper SyncGroupId, it SHALL remove the attribute from its 723 answer. 725 An MSAS receiving an SDP offer without the rtcp-idms attribute can 726 also decide that IDMS is applicable to that media session. In such a 727 case, the MSAS MAY insert the rtcp-idms attribute, including a non- 728 empty SyncGroupId, as part of its answer. 730 A receiver receiving an rtcp-idms attribute as part of the SDP answer 731 from an MSAS, SHALL start sending IDMS XR reports (following all the 732 normal RTCP rules for sending RTCP XR blocks) and SHALL be ready to 733 start receiving IDMS Settings. As usual, if a receiver does not 734 support the attribute (e.g. in case of an MSAS-inserted IDMS 735 attribute), it SHALL ignore the attribute. 737 Different updates are applicable to such an IDMS session. Updates 738 can be sent ommitting the rtcp-idms attribute, thereby ending the 739 (involvement in) the synchronization session. Updates can also be 740 sent including the rtcp-idms attribute, but with a different 741 SyncGroupId. This indicates a switch in synchronization group. 742 Updates can also be sent including another rtcp-idms attribute, 743 indicating a membership of another synchronization group, effectively 744 merging the current group(s) with the new one. 746 12.2. Declarative cases 748 In certain situations, there is no offer/answer context, but only a 749 declarative modus. In this case, the MSAS just inserts the rtcp-idms 750 attribute and a valid SyncGroupId. Any receiver receiving the rtcp- 751 idms attribute in such a declarative case, SHALL start sending IDMS 752 XR Report Blocks and SHALL be ready to start receiving RTCP IDMS 753 Settings packets. 755 13. Security Considerations 757 The security considerations described in [RFC3611] apply to this 758 document as well. 760 The RTCP XR IDMS Report Block defined in this document is used to 761 collect, summarize and distribute information on packet reception- 762 and playout-times of streaming media. The information may be used to 763 orchestrate the media playout at multiple devices. 765 Errors in the information, either accidental or malicious, may lead 766 to undesired behavior. For example, if one device erroneously or 767 maliciously reports a two-hour delayed playout, then another device 768 in the same synchronization group could decide to delay its playout 769 by two hours as well, in order to keep its playout synchronized. A 770 user would likely interpret this two hour delay as a malfunctioning 771 service. 773 Therefore, the application logic of both Synchronization Clients and 774 Media Synchronization Application Servers should check for 775 inconsistent information. Differences in playout time exceeding 776 configured limits (e.g. more than ten seconds) could be an indication 777 of such inconsistent information. 779 No new mechanisms are introduced in this document to ensure 780 confidentiality. Encryption procedures, such as those being 781 suggested for a Secure RTP (SRTP) at the time that this document was 782 written, can be used when confidentiality is a concern to end hosts. 784 14. IANA Considerations 786 This document defines a new RTCP packet type, the RTCP IDMS Packet 787 (IDMS Settings), within the existing Internet Assigned Numbers 788 Authority (IANA) registry of RTCP Control Packet Types. This 789 document also defines a new RTCP XR Block Type, the IDMS XR Report 790 Block, within the existing IANA registry of RTCP Extended Reports 791 (RTCP XR) Block Types. 793 Further, this document defines a new SDP attribute "rtcp-idms" within 794 the existing IANA registry of SDP Parameters, part of the "att-field 795 (media level only)" 797 14.1. RTCP IDMS Packet Type 799 This document assigns the packet type value TBD in the IANA 'RTCP 800 Control Packet types (PT) Registry' to the RTCP IDMS Packet Type. 802 [Note to RFC Editor: please replace TBD with the IANA-provided RTCP 803 Packet Type value for this packet type] 805 14.2. RTCP XR IDMS Report Block 807 This document assigns the block type value 12 in the IANA "RTCP XR 808 Block Type Registry" to the RTCP XR IDMS Report Block. 810 [Note to RFC Editor: this block type value is currently assigned to 811 [TS183063]. This document replaces [TS183063] as the normative 812 specification of the RTCP XR IDMS Report Block. Upon publication of 813 this document as RFC, [TS183063] will be changed to reflect this. 815 14.3. RTCP-IDMS SDP Attribute 817 The SDP attribute "rtcp-idms" defined by this document is registered 818 with the IANA registry of SDP Parameters as follows: 820 SDP Attribute ("att-field"): 822 Attribute name: rtcp-idms 824 Long form: RTCP IDMS Parameters 826 Type of name: att-field 828 Type of attribute: media level 830 Subject to charset: no 832 Purpose: see Section 11 of this document 834 Reference: this document 836 Values: see this document 838 14.4. Contact Information for Registrations 840 The contact information for the registrations is: 842 Ray van Brandenburg (ray.vanbrandenburg@tno.nl) 844 Brassersplein 2 846 2612CT, Delft, The Netherlands 848 15. Contributors 850 The following people have participated as co-authors or provided 851 substantial contributions to this document: Omar Niamut, Fabian 852 Walraven, Ishan Vaishnavi and Rufael Mekuria. In addition the 853 authors would like to thank Aidan Williams, Colin Perkins, Magnus 854 Westerlund, Roni Even, Peter Musgrave, Ali Begen, Qin Wu and Rob 855 Koenen for their review comments and contributions to the text. 857 16. References 859 16.1. Normative References 861 [I-D.draft-ietf-avtcore-clksrc] 862 Williams, A., van Brandenburg, R., Stokking, H., and K. 863 Gross, "RTP Clock Source Signalling, 864 draft-ietf-avtcore-clksrc-01", October 2012. 866 [RFC2119] Bradner, S., "Key Words for use in RFCs to Indicate 867 Requirement Levels, RFC 2119", March 1997. 869 [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. 870 Jacobson, "RTP: A Transport Protocol for Real-Time 871 Applications, RFC3550", July 2003. 873 [RFC3551] Schulzrinne, H. and S. Casner, "RTP Profile for Audio and 874 Video conferences with Minimal Control, RFC3551", 875 July 2003. 877 [RFC3611] Friedman, T., Ed., Caceres, R., Ed., and A. Clark, Ed., 878 "RTP Control Protocol Extended Reports (RTCP XR), 879 RFC3611", November 2003. 881 [RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session 882 Description Protocol, RFC4566", July 2006. 884 [RFC5234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax 885 Specifications, RFC5234", January 2008. 887 [RFC5760] Ott, J., Chesterfield, J., and E. Schooler, "RTP Control 888 Protocol (RTCP) Extensions for Single-Source Multicast 889 Sessions with Unicast Feedback, RFC5760", February 2010. 891 [RFC5905] Mills, D., Martin, J., Ed., Burbank, J., and W. Kasch, 892 "Network Time Protocol Version 4: Protocol and Algorithms 893 Specifications, RFC5905", February 2010. 895 [TS183063] 896 "IMS-based IPTV stage 3 specification, TS 183 063 v3.4.1", 897 June 2010. 899 16.2. Informative References 901 [Boronat2009] 902 Boronat, F., Lloret, J., and M. Garcia, "Multimedia group 903 and inter-stream synchronization techniques: a comparative 904 study, Elsevier Information Systems 34 (2009), pp. 108- 905 131". 907 [I-D.draft-ietf-leap-seconds] 908 Gross, K. and R. Brandenburg, van, "RTP and Leap Seconds, 909 draft-ietf-leap-seconds-01", October 2012. 911 [IEEE-1588] 912 "1588-2008 - IEEE Standard for a Precision Clock 913 Synchronization Protocol for Networked Measurement and 914 Control Systems", 2008. 916 [Ishibashi2006] 917 Ishibashi, Y., Nagasaka, M., and N. Fujiyoshi, "Subjective 918 Assessment of Fairness among users in multipoint 919 communications, Proceedings of the 2006 ACM SIGCHI 920 internation conference on Advances in computer 921 entertainment technology, 2006". 923 [RFC5868] Ott, J. and C. Perkins, "Guidelines for Extending the RTP 924 Control Protocol (RTCP), RFC5968", September 2010. 926 Authors' Addresses 928 Ray van Brandenburg 929 TNO 930 Brassersplein 2 931 Delft 2612CT 932 the Netherlands 934 Phone: +31-88-866-7000 935 Email: ray.vanbrandenburg@tno.nl 936 Hans Stokking 937 TNO 938 Brassersplein 2 939 Delft 2612CT 940 the Netherlands 942 Phone: +31-88-866-7000 943 Email: hans.stokking@tno.nl 945 M. Oskar van Deventer 946 TNO 947 Brassersplein 2 948 Delft 2612CT 949 the Netherlands 951 Phone: +31-88-866-7000 952 Email: oskar.vandeventer@tno.nl 954 Fernando Boronat 955 Universitat Politecnica de Valencia 956 IGIC Institute, Universitat Politecnica de Valencia-Campus de Gandia (UPV), C/ Paraninfo, 1, Grao de Gandia, C/ Paraninfo, 1, Grao de Gandia 957 Valencia 46730 958 Spain 960 Phone: +34 962 849 341 961 Email: fboronat@dcom.upv.es 963 Mario Montagud 964 Universitat Politecnica de Valencia 965 IGIC Institute, Universitat Politecnica de Valencia-Campus de Gandia (UPV), C/ Paraninfo, 1, Grao de Gandia, C/ Paraninfo, 1, Grao de Gandia 966 Valencia 46730 967 Spain 969 Phone: +34 962 849 341 970 Email: mamontor@posgrado.upv.es 972 Kevin Gross 973 AVA Networks 975 Phone: +1-303-447-0517 976 Email: Kevin.Gross@AVAnw.com