idnits 2.17.1 draft-romanow-clue-sdp-usage-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 184 has weird spacing: '...idth in pixel...' == Line 618 has weird spacing: '...gotiate defau...' == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (September 11, 2012) is 4238 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'CLUEFRWK' is mentioned on line 81, but not defined == Unused Reference: 'I-D.ietf-clue-framework' is defined on line 922, but no explicit reference was found in the text == Outdated reference: A later version (-25) exists of draft-ietf-clue-framework-06 Summary: 0 errors (**), 0 flaws (~~), 7 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 CLUE A. Romanow 3 Internet-Draft F. Andreasen 4 Intended status: Standards Track A. Krishna 5 Expires: March 15, 2013 Cisco Systems 6 September 11, 2012 8 Investigation of Session Description Protocol (SDP) Usage for 9 ControLling mUltiple streams for tElepresence (CLUE) 10 draft-romanow-clue-sdp-usage-02 12 Abstract 14 This draft investigates how SDP offer/answer can be used to 15 communicate CLUE encoding and encoding group parameters. 17 Status of this Memo 19 This Internet-Draft is submitted in full conformance with the 20 provisions of BCP 78 and BCP 79. 22 Internet-Drafts are working documents of the Internet Engineering 23 Task Force (IETF). Note that other groups may also distribute 24 working documents as Internet-Drafts. The list of current Internet- 25 Drafts is at http://datatracker.ietf.org/drafts/current/. 27 Internet-Drafts are draft documents valid for a maximum of six months 28 and may be updated, replaced, or obsoleted by other documents at any 29 time. It is inappropriate to use Internet-Drafts as reference 30 material or to cite them other than as "work in progress." 32 This Internet-Draft will expire on March 15, 2013. 34 Copyright Notice 36 Copyright (c) 2012 IETF Trust and the persons identified as the 37 document authors. All rights reserved. 39 This document is subject to BCP 78 and the IETF Trust's Legal 40 Provisions Relating to IETF Documents 41 (http://trustee.ietf.org/license-info) in effect on the date of 42 publication of this document. Please review these documents 43 carefully, as they describe your rights and restrictions with respect 44 to this document. Code Components extracted from this document must 45 include Simplified BSD License text as described in Section 4.e of 46 the Trust Legal Provisions and are provided without warranty as 47 described in the Simplified BSD License. 49 Table of Contents 51 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 52 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 53 3. Background . . . . . . . . . . . . . . . . . . . . . . . . . . 3 54 4. Assumptions and Design Principles . . . . . . . . . . . . . . 4 55 5. Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 56 6. Encoding group . . . . . . . . . . . . . . . . . . . . . . . . 5 57 7. Solution overview . . . . . . . . . . . . . . . . . . . . . . 8 58 8. Provider advertisement in the initial offer/answer . . . . . . 9 59 8.1. Bidirectional SDP messages . . . . . . . . . . . . . . . . 13 60 8.2. Unidirectional SDP messages . . . . . . . . . . . . . . . 13 61 9. Multiple offer/answer exchanges . . . . . . . . . . . . . . . 15 62 10. Representation of Encoding and Encoding group in SDP . . . . . 19 63 11. Concerns about conveying CLUE parameters via SDP . . . . . . . 21 64 12. Security Considerations . . . . . . . . . . . . . . . . . . . 22 65 13. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 22 66 14. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 22 67 15. References . . . . . . . . . . . . . . . . . . . . . . . . . . 22 68 15.1. Normative References . . . . . . . . . . . . . . . . . . . 22 69 15.2. Informative References . . . . . . . . . . . . . . . . . . 22 70 Appendix A. Changes From Earlier Versions . . . . . . . . . . . . 23 71 A.1. Changes From Draft -01 . . . . . . . . . . . . . . . . . . 23 72 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 23 74 1. Introduction 76 This draft investigates how SDP could be used to carry CLUE, 77 ControLling mUltiple streams for tElepresence, encoding and encoding 78 group provider advertisements. CLUE specifies how multiple streams 79 can be communicated in a telepresence conference in a standardized 80 manner allowing interoperability. A familiarity is assumed with the 81 CLUE framework document and the concepts defined therein [CLUEFRWK]. 82 The media provider describes to the media consumer both media 83 captures and encodings that it is capable of sending, and the 84 consumer "configures" the provider to send the captures using the 85 potential streams it wants to receive. In order for an interactive 86 telepresence session to be established between A and B, each party 87 needs to configure its role as both a media provider and a media 88 consumer. 90 Encoding parameters are maxBandwidth, maxMbps, maxFps, maxWidth, 91 maxHeight. Encoding group parameters are maxGroupBandwidth and 92 maxGroupVideoMbps. 94 There are 2 reasons we investigated using SDP for CLUE provider 95 advertisements of encoding and encoding groups. It was suggested 96 that: 98 1. By carrying the encoding and encoding group parameters in SDP, 99 the call-control agents, proxies, and other intermediaries can 100 monitor and potentially modify these values thereby allowing 101 administrators to easily configure and enforce either static 102 limitations or more dynamic call-shaping approaches. 103 2. SDP is the usual protocol for communicating the encoding and 104 encoding group related parameters in SIP-based systems 106 Media captures, capture sets, and simultaneous transmission 107 information are all communicated in a new CLUE protocol, whereas 108 encoding and encoding group parameters are communicated in SDP. 110 2. Terminology 112 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 113 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 114 document are to be interpreted as described in RFC 2119 [RFC2119] and 115 indicate requirement levels for compliant implementations. 117 3. Background 119 The following abstractions defined in the CLUE framework document 120 need to be understood in considering this investigation. 122 o Media capture - fundamental representation of the content, audio 123 and video so far defined. Each media capture is associated with 124 one or more encoding groups. 125 o Capture set - A capture set is most usefully thought of as being a 126 set of table rows, with each row consisting of a set of media 127 captures. Media captures within a row can always be used 128 simultaneously, whereas media captures in different sets may or 129 may not be used simultaneously.. By including multiple media 130 captures in a row within a capture set, the provider is signaling 131 that those captures together form a representation of that capture 132 set's scene. 133 o Simultaneous transmission set - there are often physical 134 constraints that determine whether captures can be sent at the 135 same time or not. For example, a camera may be able to create a 136 video capture which is close up and another which is zoomed out. 137 These 2 views cannot be done simultaneously. The simultaneous 138 transmission set expresses the physical constraints on 139 simultaneity. 140 o Encoding - Represents a way to encode a media capture in terms of 141 its associated media characteristics, specifically maxBandwidth, 142 maxFps, maxMbps, maxHeight and maxWidth. Each encoding is 143 uniquely identified within an encoding group. [Practically it's 144 desirable to have a unique ID among all encodings in the 145 conference] 146 o Encoding group - An encoding group includes set(s) of encodings, 147 plus parameters that apply to the group as a whole, specifically 148 maxGroupBandwidth and maxGroupMpbs. Each encoding group has a 149 unique identifier in the SDP. 151 4. Assumptions and Design Principles 153 o For each CLUE type ("purpose") of media capture (audio, video- 154 main, or video-presentation), there will be at most one 155 corresponding SDP media stream ("m=" line). Note: video-main and 156 video-presentation are both using "video" media type. 157 o Multiple media captures of the same CLUE type ("purpose"), or 158 different encodings of a given media capture will all use the same 159 SDP media stream (e.g. via RTP multiplexing with a different SSRC 160 for each). 161 o It is acceptable to use more than one offer/answer exchanges to 162 setup the whole Telepresence session. 163 o The Telepresence session is established by setting up sets of 164 unidirectional streams (not bidirectional streams). 166 5. Encoding 168 The CLUE framework calls an RTP stream that the provider is capable 169 of sending, an Encoding (note that an RTP session may contain 170 multiple RTP streams, with each stream having a unique SSRC). The 171 parameters of an encoding are: encodingID, maxBandwidth, maxMbps, 172 maxWidth, maxHeight, maxFrameRate, as described below in the 173 following Figures: Figure 1 shows video encoding parameters and 174 Figure 2. shows audio encoding parameters. 176 +-----------------+-------------------------------------------------+ 177 | Name | Description 178 +-----------------+-------------------------------------------------+ 179 | encodingID | A unique identifier for the individual encoding | 180 | maxBandwidth | Maximum number of bits per second | 181 | maxMbps | Maximum number of macroblocks per second: | 182 | | ((width + 15) / 16) * ((height + 15) / 16) | 183 | | * framesPerSecond | 184 | maxWidth |Video resolution's max supported width in pixels| 185 | maxHeight |Video resolution's max supported height in pixels| 186 | maxFrameRate | Maximum supported frame rate | 187 +--------------------+----------------------------------------------+ 189 Figure 1: Video encoding parameters 191 +--------------+----------------------------------------+ 192 | Name | Description | 193 +--------------+----------------------------------------+ 194 | encodingID | A unique id for the encoding | 195 +--------------+----------------------------------------+ 196 | maxBandwidth | Max # of bits per second | 197 +--------------+----------------------------------------+ 199 Figure 2: Audio encoding parameters 201 6. Encoding group 203 In addition to encodings, the Framework defines an Encoding Group as 204 a set of individual encodings plus parameters that apply to the whole 205 set, as shown in Figure 3, Encoding group. The two parameters for 206 the group of encodings are maxGroupBandwidth and maxGroupVideoMbps. 207 Max group bandwidth refers to both audio and video, and is the 208 maximum value of the sum of all of the configured bitrates. The max 209 group video macroblocks per second refers only to video and is the 210 maximum value for the sum of all the macroblocks per second in the 211 encoding group. 213 +------------------+-------------------------------------------------+ 214 | Name | Description | 215 +------------------+-------------------------------------------------+ 216 | maxGroupBandwidth| Max # bits per sec for all encodings combined | 217 | maxGroupVideoMbps| Max # macroblks/sec all video encodings combined| 218 | videoEncodings[] | Set of available encodings (list of encodingIDs)| 219 | audioEncodings[] | Set of available encodings (list of encodingIDs)| 220 +------------------+-------------------------------------------------+ 222 Figure 3: Encoding group parameters 224 Additional characteristics of encoding groups include: 226 1. A media provider advertises one or more encoding groups. 227 2. Each encoding group includes one or more individual encodings. 228 3. Each individual encoding can represent a different way of 229 encoding media. For example one individual encoding may be 230 1080p60 video, another could be 720p30, with a third being CIF. 231 4. While a typical 3 codec/display system might have one encoding 232 group per "codec box", there are many possibilities for the 233 number of encoding groups a provider may be able to offer and for 234 the encoding values in each encoding group. The consumer uses 235 the media capture attribute EncGroupID to know which set of 236 encodings to consider using for that capture and what the 237 constraints are. 238 5. In a provider advertisement, the sum of the maxBandwidths of the 239 encodings advertised in an encoding group may total more than the 240 maxGroupBandwidth, as these are potential encodings. Similarly 241 for maxGroupVideoMbps. The group restriction however must be 242 respected by the provider in terms of actual set of encodings 243 used at any given point in time. 244 6. In a consumer request, the sum of the maxBandwidths of the 245 encodings selected in an encoding group may total more than the 246 maxGroupBandwidth. However the provider will send values under 247 the max of the individual encodings and which sum to a total 248 below or equal to that of the group max. 250 The following diagram Figure 4, Association of captures and 251 encodings, depicts the association between the encodings (expressed 252 in SDP) and the media captures, expressed in elsewhere (in a CLUE 253 protocol perhaps). In this diagram the consumer can match Capture 1 254 with any of the encodings in Encoding group 1. 256 The consumer can match capture 2 with any of the encodings in 257 encoding group 2, and match capture 3 wiht any of the encodings in 258 encoding group 1. 260 +--------------------+ 261 | Capture Set 1 | 262 | | Encodings for capture 1 263 | +----------------+--------------- Encoding 1 } 264 | | capture 1 | |------------- Encoding 2 } Encoding group 1 265 | | video | |------------- Encoding 3 } 266 | | ... | | 267 | | encgrpID 1 | | 268 | +----------------+ | 269 | | 270 | +----------------+ | Encodings for capture 2 271 | | capture 2 | | 272 | | audio | |---------------Encoding 4 } 273 | | .... | |---------------Encoding 5 } Encoding group 2 274 | | | | 275 | | encgrpID 2 | | 276 | +----------------+ | 277 | | Encodings for capture 3 278 | +----------------+--------------- Encoding 1 } 279 | | capture 3 | |------------- Encoding 2 } Encoding group 1 280 | | video | |------------- Encoding 3 } 281 | | ... | | 282 | | encgrpID 1 | | 283 | +----------------+ | 284 | | 285 +--------------------+ 287 Figure 4: Association of captures and encodings 289 In this example showing a capture set containing 2 captures, the 290 first captureID 1 is video and is mapped to encoding group 1, 291 encgrpID 1. The consumer can map capture 1 to any or all of the 292 potential RTP streams within encoding group 1. The use case for 293 mapping a single video capture to more than one encoding at a given 294 point in time is simulcast. The information contained in the 295 encoding group parameters helps the consumer choose how to do this 296 mapping by communicating the resource limitations. 298 7. Solution overview 300 Today telepresence endpoints actively negotiate audio and video SDP 301 media streams using the SDP offer/answer model during call 302 establishment and during midcall features, such as hold resume. 303 However, in the CLUE framework, each party in the conference is 304 usually both a provider and a consumer whether point-to-point or 305 multipoint. The parameter values for one provider may well not be 306 the same values for the other provider. Thus each party needs to 307 advertise its provider encoding parameter values to the other side. 308 This is not the typical communication model for SDP offer/answer, 309 which is more often a bidirectional agreement on parameter values. 311 As specified in the CLUE framework, parties in a telepresence 312 conference do not negotiate a single value between them; therefore 313 offer/answer [RFC3264] is not used for the full negotiation of all 314 encoding related values. Rather, we propose, sending CLUE provider 315 advertisements for encodings and encoding group in SDP offer/answer, 316 if it valuable for intermediaries to know these values. Then the 317 consumer uses another protocol, such as a new CLUE protocol, to 318 select which captures it wants at a given point in time and how it 319 wants them encoded, subject to the encoding and encoding group 320 parameters specified in the SDP. Provider advertisements not related 321 to encoding, that is, capture attributes such as spatial relations 322 would also be communicated in the CLUE protocol, rather than in SDP, 323 as they are not provided as part of the SDP media descriptions for 324 the streams. In this approach, the CLUE protocol carries the 325 information related to media captures, as well as implements the 326 control functions described in the CLUE framework including 327 configuring the actual encodings used. 329 For example, if A and B are parties (either endpoints or MCUs) in a 330 telepresence conference, A is a provider for B, and B is a provider 331 for A. We propose that A and B send to each other provider 332 advertisements for encoding and encoding groups through SDP; and A 333 and B send to each other provider advertisements for Captures and 334 consumer requests through a CLUE protocol. 336 The sending of advertisements of capture information and encoding 337 information and the responding consumer request happens not only at 338 call setup but also throughout the conference whenever there is a 339 change in what the provider is capable of sending, or a change in 340 which captures and streams the consumer wants to receive. 342 We will now explore two approaches for sending provider encoding 343 advertisements in SDP. In the first approach, the provider encoding 344 advertisements are accomplished in the initial offer/answer. A sends 345 its provider encoding advertisements in the offer and B sends its 346 provider encoding advertisements in the answer. There are two 347 message options here: bidirectional or unidirectional. 349 In the second approach, a series of offer/answer messages are sent 350 from each participant. Both approaches have pluses and minuses. How 351 each approach handles the situation where the parties advertise 352 different types (SDP media capabilities) is of interest. For example 353 one party may offer video-presentation and the other may not offer 354 it. 356 8. Provider advertisement in the initial offer/answer 358 One possibility for exchange of CLUE parameters between 2 endpoints 359 or MCUs is in the initial SDP offer/answer exchange. One party sends 360 provider encoding advertisements as the SDP offer and the other party 361 answers with its provider encoding advertisements, as shown in 362 Figure 5, Initial offer/answer exchange. A then sends its capture 363 information to B through the CLUE protocol (and vice versa). 365 Provider A 366 +--------------------+ 367 | SDP offer 1 | 368 | | 369 | Offered media | 370 | (actively negotd | 371 | as today) 372 | | 373 | Encoding Group 1 374 | +-------------+ | 375 | | Encoding 1 | | 376 | | Encoding 2 | | 377 | | ... | | 378 | +-------------+ | Offer 379 | |--------> 380 | Encoding Group 2 | 381 | +-------------+ | 382 | | Encoding 3 | | 383 | | Encoding 4 | | 384 | | ... | | 385 | +-------------+ | 386 | ....... | 387 | | 388 +--------------------+ Provider B 389 +--------------------+ 390 | SDP answer 1 | 391 | | 392 | Accepted media | 393 | (actively negotd) | 394 Answer | | 395 <--------| | 396 | Encoding-Group 1 | 397 | +-------------+ | 398 | | Encoding 1 | | 399 | | Encoding 2 | | 400 | | ... | | 401 | +-------------+ | 402 | | 403 | Encoding-Group 2 | 404 | +-------------+ | 405 | | Encoding 3 | | 406 | | Encoding 4 | | 407 | | ... | | 408 | +-------------+ | 409 | ....... | 410 +--------------------+ 412 Figure 5: Initial offer answer exchange 414 There are two possibilities: birdirectional or unidirectional SDP 415 messages. 417 Figure 6, Call message sequence illustrates an end-end call with the 418 message sequence where provider advertisements are in initial offer/ 419 answer. 421 Alice Bob 422 | | 423 ________|__________________________________|_______________________ 424 | |(1) INVITE | | 425 | | Offer (Offered media, | | 426 | | CLUE extensions - | | 427 | | enc-grp 1 | | 428 | | enc 1 | | 429 | | enc 2 | | 430 | | enc 3 | SIP | 431 | | enc-grp 2 | CALL SETUP | 432 | | enc 4 | | 433 | | enc 5 | Alice sends Provider | 434 | | TRANSPORT TBD/CLUE connection | advertisement for | 435 | | params ) | encodings & encoding | 436 | | | groups in SDP | 437 | | | | 438 | |--------------------------------->| | 439 | |(2) 200OK | | 440 | | Answer (Accepted media, | Bob in | 441 | | CLUE extensions | Provider's role | 442 | | enc-grp 1 | sends his | 443 | | enc 1 | advertisement for | 444 | | enc-grp 2 | encodings & encoding | 445 | | enc 2 | groups | 446 | | TRANSPORT TBD/CLUE negotd | | 447 | | connection ) | | 448 | |<---------------------------------| | 449 |________|__________________________________|_______________________| 450 | | 451 | | 452 ________|__________________________________|_______________________ 453 | | (3) CLUE PROTOCOL | | 454 | |--------------------------------->| | 455 | | CLUE Provider MESSAGEs | Alice and Bob in | 456 | | Describe media captures | Provider's role | 457 | | Includes association of MC | | 458 | | to Encoding Group | | 459 | |<---------------------------------| | 460 | | | | 461 |________|__________________________________|_______________________| 462 | | | 463 | . . . . . . . . . . . . . . . . | 464 ________|__________________________________|_______________________| 465 | | (4) CLUE PROTOCOL | | 466 | | | | 467 | | Bob configures the MCs it wants| Bob in consumer role | 468 | | Onto the encodings it wants | sends CLUE consumer | 469 | | Within the encoding group | CONFIGURE message | 470 | | Constraints. | | 471 | |<---------------------------------| | 472 | | | | 473 |________|__________________________________|_______________________| 474 | | 475 ________|__________________________________|_______________________ 476 | | (5) SIP Re-INVITE | | 477 | | Re-negotiate TIAS bw etc | SIP | 478 | | based on consumer choices | Media Re-negotiation| 479 |________|__________________________________|_______________________| 480 | | 481 | . . . . . . . . . . . . . . . . 482 ________|__________________________________|_______________________ 483 | | (6) CLUE PROTOCOL | | 484 | | | | 485 | | Choose the MCs to be sent, | CLUE MESSAGE | 486 | | Request media in various | | 487 | | encoding formats | | 488 | |--------------------------------->| Alice in | 489 | | | Consumer role | 490 |________|__________________________________|_______________________| 491 | | 492 ________|__________________________________|_______________________ 493 | | (7) SIP Re-INVITE | | 494 | | Re-negotiate TIAS bw etc | SIP | 495 | | based on consumer choices| Media Re-negotiation| 496 |________|__________________________________|_______________________| 497 | | 498 . . . . . . . . . . . . . . . . 500 Figure 6: Call message sequence for CLUE advertisements in initial 501 offer/answer 503 8.1. Bidirectional SDP messages 505 The challenge with a bidirectional approach is that the answerer must 506 not respond with a type that was not in the offer. Suppose A offers 507 two SDP media streams video-main and audio. B however wants to offer 508 3 SDP media streams: video-main, audio, and video-presentation. B 509 cannot add a new stream type in the answer. 511 One way around this is for the offerer to always offer all 3 CLUE 512 media types ("purpose") whether it will send them or not, i.e., be a 513 provider. This allows both sides to negotiate use of each "purpose" 514 as a provider and/or a consumer. The SDP direction attributes 515 ("sendrecv", "sendonly", "recvonly", and "inactive") are used to 516 indicate the real intent for a stream; a provider sends whereas a 517 consumer receives, and hence a "sendrecv" stream covers both provider 518 and consumer role. Streams can of course be rejected as per normal 519 offer/answer procedures (e.gg., if neither side wants to use a 520 particular stream such as video-presentation. An adjustment to 521 reflect the real situation can also bemade in a subsequent offer/ 522 answer exchange. 524 Using the example above, A will only send video main and audio, 525 however, following the specification, A offers all 3 audio, video 526 main, video presentation. B is going to send presentation, B accepts 527 all 3 audio, video-main and video-presentation and hence generates an 528 answer with them. The CLUE protocol negotiations take place. Then 529 there is an updated offer in which A does not offer video 530 presentation, thus making the SDP media streams accurate. 532 8.2. Unidirectional SDP messages 534 1) Offer (INVITE) from Alice to Bob 535 (Alice provider=sendonly, Alice consumer=recvonly) 536 ================================================== 537 m=audio 10000 RTP/AVP 96 ;Alice provider audio 538 a=sendonly 539 a=rtpmap:96 PCMU 540 ... 541 m=video 10002 RTP/AVP 96 97;Alice provider video-main 542 a=sendonly 543 a=rtpmap:96 H264 544 a=rtpmap:97 H265 545 ... 546 m=video 10004 RTP/AVP 96 ;Alice provider video-presentation 547 a=sendonly 548 a=rtpmap:96 H264 549 ... 551 m=audio 20000 RTP/AVP 96 ;Alice consumer audio 552 a=recvonly 553 a=rtpmap:96 PCMU 554 ... 555 m=video 20002 RTP/AVP 96 97 ;Alice consumer video-main 556 a=recvonly 557 a=rtpmap:96 H264 558 a=rtpmap:97 H265 559 ... 560 m=video 20004 RTP/AVP 96 ;Alice consumer video-presentation 561 a=recvonly 562 a=rtpmap:96 H264 563 ... 565 2) Answer from Bob to Alice 566 (Bob provider=sendonly, Bob consumer=recvonly) 567 ============================================== 568 m=audio 30000 RTP/AVP 96 ;Bob consumer audio 569 a=recvonly 570 a=rtpmap:96 PCMU 571 ... 572 m=video 30002 RTP/AVP 96 97 ;Bob consumer video-main 573 a=recvonly 574 a=rtpmap:96 H264 575 a=rtpmap:97 H265 576 ... 577 m=video 30004 RTP/AVP 96 ;Bob consumer video-presentation 578 a=recvonly 579 a=rtpmap:96 H264 580 ... 581 m=audio 40000 RTP/AVP 96 ;Bob provider audio 582 a=sendonly 583 a=rtpmap:96 PCMU 584 ... 585 m=video 40002 RTP/AVP 96 97;Bob provider video-main 586 a=sendonly 587 a=rtpmap:96 H264 588 a=rtpmap:97 H265 589 ... 590 m=video 0 RTP/AVP 96 ;Bob does not want to be a provider 591 for video-presentation, so rejected this stream (port=0) 592 a=inactive 593 a=rtpmap:96 H264 594 ... 596 9. Multiple offer/answer exchanges 598 An alternate approach to using the intial offer answer for the 599 provider encoding advertisements is to do multiple offer/answer 600 messages in which both sides make provider advertisements. The 601 initial offer/answer establishes that the parties are CLUE capable 602 and subsequent offer/answer messages exchange provider encoding 603 advertisements. The disadvantage of this approach is that it 604 requires a number of offer/answer messages. 606 In this method, the call is initially established with configurable 607 default media values. The call flow is shown in detail in Figure 7 608 Alice Bob 609 | | 610 ________|__________________________________|_______________________ 611 | |(1) INVITE | | 612 | | Offer {m=audio..., | | 613 | | m=video (main) .... | | 614 | | m=application CLUE | SIP | 615 | | trans params from | CALL SETUP | 616 | | Alice end | | 617 | | } | | 618 | |--------------------------------->| Negotiate default | 619 | |(2) 200OK | media parameters | 620 | | Answer {m=audio..., | | 621 | | m=video (main) .... | | 622 | | m=application CLUE | | 623 | | trans params from | | 624 | | Bob end | | 625 | | } | | 626 | |<---------------------------------| | 627 | | | | 628 | |<================================>| | 629 | | Both Alice & Bob understand | | 630 | | they are talking to CLUE | | 631 | | endpoint | | 632 |________|__________________________________|_______________________| 633 | | 634 | | 635 ____________________________________________ 636 / CLUE CONNECTION ESTABLISHMENT \ 637 \____________________________________________/ 638 | | 639 ________|__________________________________|_______________________ 640 | | Provider Capabilities Ind(Alice)| | 641 | | | | 642 | | | | 643 | |(3)re-INVITE | | 644 | | Offer {m=audio ... | | 645 | | CLUE extensions | | 646 | | enc-grp 1 | | 647 | | enc 1 | | 648 | | enc 2 | | 649 | | enc 3 | ALICE in | 650 | | m=video (main)... | Provider's role | 651 | | CLUE extensions | | 652 | | enc-grp 2 | | 653 | | enc 4 | | 654 | | enc-grp 3 | (keeps previously | 655 | | enc 5 | negotd media | 656 | | m=application CLUE | | 657 | | conn params | params) | 658 | | } | | 659 | |--------------------------------->| Adds Alice's | 660 | |(4) 200OK | provider | 661 | | Answer {m=audio... | encoding | 662 | | m=video (main) ... | capabilities | 663 | | m=application CLUE | for each m-line | 664 | | conn params | | 665 | | } | | 666 | |<---------------------------------| | 667 |________|__________________________________|_______________________| 668 | | 669 | | 670 ________|__________________________________|_______________________ 671 | | Provider Capabilities Ind (Bob) | | 672 | | | | 673 | | | | 674 | |(5)re-INVITE | | 675 | | Offer {m=audio ... | BOB in | 676 | | CLUE extensions | provider role | 677 | | enc-grp 1 | | 678 | | enc 1 | (keeps previously | 679 | | enc 2 | negotd media | 680 | | m=video (main)... | params) | 681 | | CLUE extensions | | 682 | | enc-grp 2 | Is capable of | 683 | | enc 3 | doing presentation| 684 | | m=application CLUE | | 685 | | conn params | Adds new pres | 686 | | m=video (pres)... | m-line | 687 | | CLUE extensions | | 688 | | enc-grp 3 | | 689 | | enc 4 | | 690 | | } | | 691 | |<---------------------------------| | 692 | |(6) 200OK | Adds Bob's | 693 | | Answer {m=audio... | provider | 694 | | m=video (main) ... | encoding capabil | 695 | | m=application CLUE | for each m-line | 696 | | conn params | | 697 | | m=video (pres) ... | | 698 | |--------------------------------->| | 699 |________|__________________________________|_______________________| 700 | | 701 ________|__________________________________|_______________________ 702 | | (7) CLUE PROTOCOL | | 703 | |--------------------------------->| | 704 | | CLUE Provider MESSAGEs | Alice & Bob in | 705 | | Describe media captures | Provider's role | 706 | | Includes association of MC to | 707 | | Encoding Group | | 708 | |<---------------------------------| | 709 | | | | 710 |________|__________________________________|_______________________| 712 | | 713 ________|__________________________________|_______________________ 714 | | (8) CLUE PROTOCOL | | 715 | | | | 716 | | Bob configures the MCs it wants | | 717 | | Onto the streams it wants within | Bob in consumer role | 718 | | the encoding group constraints. | sends CLUE consumer | 719 | | | CONFIGURE message | 720 | | | | 721 | |<---------------------------------| | 722 | | | | 723 |________|__________________________________|_______________________| 724 | | 725 ________|__________________________________|_______________________ 726 | | (9) SIP Re-INVITE | | 727 | | Re-negotiate TIAS bw etc | SIP | 728 | | based on view options | Media Re-negotiation| 729 |________|__________________________________|_______________________| 730 | | 731 | | 732 ________|__________________________________|_______________________ 733 | | (10) CLUE PROTOCOL | | 734 | | | | 735 | | Choose the MC to view, | CLUE MESSAGE | 736 | | Request the video in various | | 737 | | encoding formats | | 738 | |--------------------------------->| Alice in | 739 | | | Consumer role | 740 |________|__________________________________|_______________________| 741 | | 742 ________|__________________________________|_______________________ 743 | | (11) SIP Re-INVITE | | 744 | | Re-negotiate TIAS bw etc | SIP | 745 | | based on view options | Media Re-negotiation| 746 |________|__________________________________|_______________________| 747 | | 748 . . . . . . . . . . . . . . . . 750 Figure 7: Call message sequence for CLUE advertisements multiple 751 offer/answers 753 The offerer establishes a CLUE application with transport connection. 754 The answerer acknowledges, thus indidcating its willingness to 755 participate in CLUE signaling. No encoding advertisements are 756 exchanged at this point. 758 The CLUE connection is established 760 Alice, in the provider's role, indicates ecnoding and encoding group 761 advertisement parameters using re-INVITE (new offer/answer). In this 762 example, Alice does not indicate support for video-presentation. Bob 763 acks back and does not provide any of his provider encoding 764 advertisements at this point 766 Bob now signals his provider encoding advertisements using a fresh 767 re-INVITE (third offer/answer). Bob is capable of video- 768 presentation. It adds a video-presentation m=line and indicates its 769 provider encoding advertisements. Alice may decide to not actively 770 do presentiaton at this time, and may stop the media (port 0) in 771 200OK. But Alice is aware of the CLUE advertisement for the video- 772 presentation to be used in the future. 774 Alice and Bob use CLUE messages exchange provide capture 775 advertisements, including bindings to encoding groups. 777 Bob in consumer role uses the consumer configure CLUE message to 778 select which captures and encodings he wants. 780 Bob sends a re-INVITE (4th offer/answer) to negotiatie TIAS bandwidth 781 based on the consumer requests. This enables call control and 782 intermediaries to manage bandwidth resources. 784 Alice in consumer role uses the consumer configure CLUE message to 785 slect which captures and encodings she wants. 787 Alice sends a re=INVITE to negotiate new TIAS bandwidth based on the 788 consumer requests, updating previous parameter values. 790 This process continues as options change at either end. 792 10. Representation of Encoding and Encoding group in SDP 794 We want a mechanism that will allow the provider to advertise 795 encodings and encoding groups. We can define three new SDP 796 attributes: clue, enc, encgrp. 798 Encgrp is the encoding group. It consists of an encoding group ID, a 799 list of encodings, and a list of encoding group parameters, 800 maxGroupBandwidth, maxGroupVideoMbps. 802 a=encgrp: 804 Enc is the encoding. It consists of an encoding id and a list of 805 clue parameters and their values for the encoding. 807 a=enc: 809 Clue consists of CLUE encoding parameters: max-fps, max-mbps, max- 810 bitrate, imageattr. The parameter list can be extended in the 811 future. 813 a=clue: 815 where clue attributes in our context would be "imageattr[x= ,y= ]" 816 [draft-ietf-mmusic-image-attributes],"max-mbps", "max-br", max-fps 818 Example of 2 video encodings: 820 a=clue:1 imageattr[w=1920, y=1088] 821 a=clue:2 max-fps=60 822 a=clue:3 max-mbps=244800 823 a=clue:4 max-br=4000 824 a=clue:5 imageattr[x=960, y=554] 825 a=clue:6 max-fps=30 826 a=clue:7 max-mbps=61200 828 a=enc:1 clue=1,2,3,4 Encoding 1 and its clue attributes 829 a=enc:2 clue=5,6,7,4 Encoding 2 and its clue attributes 830 Example of 2 video encoding groups 831 a=encgrp: 1 enc=1,2,3 grpparm=3,4,5 832 a=encgrp:2 enc=4,5,6,7 grpparm=2,4,5 834 Figure 8: Example 2 video encodings 836 11. Concerns about conveying CLUE parameters via SDP 838 There are some potential concerns in using SDP for CLUE provider 839 capability announcements. Passing information to two separate 840 protocols and processing it from them complicates the protocol and 841 implementation. Secondly, if the two SIP peers do not communicate 842 directly (e.g. due to SIP Record-Route), then reINVITEs/UPDATEs may 843 be delay-prone compared to an independent and direct end-to-end 844 protocol; this is relatively unimportant so long as any parameters 845 expected to change repeatedly, or that must be sent when a stream 846 switches (i.e., near real-time) are kept out of the SDP. 848 The encoding and encoding group will potentially be large and 849 significantly expand the SDP size. A compact representation is 850 suggested in the above to try and alleviate this. Further work and 851 testing will be required to see if it is a problem in practice, and 852 whether it is inherent to CLUE to begin with. 854 The presence of the CLUE protocol negotiation provides a way to 855 ameliorate this problem for systems operating in a mixed CLUE/ 856 non-CLUE environment that don't wish to send the information to non- 857 CLUE endpoints: the initial offer contains the CLUE protocol m-line, 858 but not the CLUE media descriptions. The answerer sees that the 859 caller is CLUE-capable and includes the media description in their 860 answer, and the offerer can then reINVITE once negotiation is 861 complete with a media description. 863 The value to the intermediaries of the encoding parameters needs to 864 be considered carefully. The intermediaries will have the TIAS value 865 which is updated after CLUE protocol activity. What about the 866 encoding parameters? It is unclear they are of much use to 867 intermediaries by themselves - they are the potential maximum values 868 of potential RTP streams. How are intermediaries expected to use 869 these? 871 It seems the encoding group parameter values, especially the 872 maxGroupBandwidth could be of interest to intermediaries augmenting 873 the TIAS information. An alternative to including provider 874 advertisements in SDP might be to include an attribute in SDP for the 875 sum of all the Encoding group maxGroupBandwidths, which seems more 876 valuable than all the individual encoding group maxGroupBandwidths. 878 Another factor to consider is that suppose an intermediary changes 879 the maxGroupBandwidth value - that change would be seen by the 880 Consumer, not by the provider, and it is the provider who is the one 881 who needs to see a change in encoding group available resources. 882 This could require actual negotiation of these parameters, instead of 883 the currently declarative unidirectional use specified in the draft 884 currently. 886 Note on real time requirements: The provider advertisements and 887 Consumer selection should be as quick as possible, but do not have 888 hard real time requirements. They can be accomplished within the 889 time scale of signaling. 891 There is however one scenario that has timing requirements more 892 stringent than signaling typically provides. Consider a switch=true 893 audio capture, the information on the spatial location of the audio 894 capture needs to happen in real time. The speaker changes frequently 895 and the location of the speaker must be known quickly (in near real- 896 time) in order to assign the incoming stream to a decoder. 898 Editor's Note: add reference to draft-clue-hansen 900 12. Security Considerations 902 TBD 904 13. Acknowledgements 906 Thanks to Rob Hansen for discussions on the advantages and 907 disadvantages of using SDP. 909 14. IANA Considerations 911 TBD 913 15. References 915 15.1. Normative References 917 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 918 Requirement Levels", BCP 14, RFC 2119, March 1997. 920 15.2. Informative References 922 [I-D.ietf-clue-framework] 923 Romanow, A., Duckworth, M., Pepperell, A., and B. Baldino, 924 "Framework for Telepresence Multi-Streams", 925 draft-ietf-clue-framework-06 (work in progress), 926 July 2012. 928 [RFC3264] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model 929 with Session Description Protocol (SDP)", RFC 3264, 930 June 2002. 932 Appendix A. Changes From Earlier Versions 934 Note to the RFC-Editor: please remove this section prior to 935 publication as an RFC. 937 A.1. Changes From Draft -01 939 o Version -02 is being submitted to keep the document available for 940 discussion until decisions are made. 942 Authors' Addresses 944 Allyn Romanow 945 Cisco Systems 946 San Jose, CA 95134 947 USA 949 Email: allyn@cisco.com 951 Flemming Andreasen 952 Cisco Systems 953 Iselin, NJ 954 USA 956 Email: fandreas@cisco.com 958 Arun Krishna 959 Cisco Systems 960 San Jose, CA 961 USA 963 Email: arukrish@cisco.com