idnits 2.17.1 draft-ietf-mmusic-image-attributes-11.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (Feb 18, 2011) is 4814 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Obsolete normative reference: RFC 4566 (Obsoleted by RFC 8866) -- Obsolete informational reference (is this intentional?): RFC 3016 (Obsoleted by RFC 6416) -- Obsolete informational reference (is this intentional?): RFC 3984 (Obsoleted by RFC 6184) Summary: 1 error (**), 0 flaws (~~), 1 warning (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group I. Johansson 3 Internet-Draft Ericsson AB 4 Intended status: Standards Track K. Jung 5 Expires: August 22, 2011 Samsung Electronics Co., Ltd. 6 Feb 18, 2011 8 Negotiation of Generic Image Attributes in SDP 9 draft-ietf-mmusic-image-attributes-11 11 Abstract 13 This document proposes a new generic session set up attribute to make 14 it possible to negotiate different image attributes such as image 15 size. A possible use case is to make it possible for a low-end hand- 16 held terminal to display video without the need to rescale the image, 17 something that may consume large amounts of memory and processing 18 power. The draft also helps to maintain an optimal bitrate for video 19 as only the image size that is desired by the receiver is 20 transmitted. 22 Status of this Memo 24 This Internet-Draft is submitted in full conformance with the 25 provisions of BCP 78 and BCP 79. 27 Internet-Drafts are working documents of the Internet Engineering 28 Task Force (IETF). Note that other groups may also distribute 29 working documents as Internet-Drafts. The list of current Internet- 30 Drafts is at http://datatracker.ietf.org/drafts/current/. 32 Internet-Drafts are draft documents valid for a maximum of six months 33 and may be updated, replaced, or obsoleted by other documents at any 34 time. It is inappropriate to use Internet-Drafts as reference 35 material or to cite them other than as "work in progress." 37 This Internet-Draft will expire on August 22, 2011. 39 Copyright Notice 41 Copyright (c) 2011 IETF Trust and the persons identified as the 42 document authors. All rights reserved. 44 This document is subject to BCP 78 and the IETF Trust's Legal 45 Provisions Relating to IETF Documents 46 (http://trustee.ietf.org/license-info) in effect on the date of 47 publication of this document. Please review these documents 48 carefully, as they describe your rights and restrictions with respect 49 to this document. Code Components extracted from this document must 50 include Simplified BSD License text as described in Section 4.e of 51 the Trust Legal Provisions and are provided without warranty as 52 described in the Simplified BSD License. 54 Table of Contents 56 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 57 1.1. Requirements . . . . . . . . . . . . . . . . . . . . . . . 4 58 2. Conventions, Definitions and Acronyms . . . . . . . . . . . . 5 59 3. Specification of the 'imageattr' SDP Attribute . . . . . . . . 5 60 3.1. Attribute syntax . . . . . . . . . . . . . . . . . . . . . 5 61 3.1.1. Overall view of syntax . . . . . . . . . . . . . . . . 5 62 3.2. Considerations . . . . . . . . . . . . . . . . . . . . . . 11 63 3.2.1. No imageattr in 1st offer . . . . . . . . . . . . . . 11 64 3.2.2. Different payload type numbers in offer and answer . . 11 65 3.2.3. Asymmetry . . . . . . . . . . . . . . . . . . . . . . 12 66 3.2.4. sendonly and recvonly . . . . . . . . . . . . . . . . 12 67 3.2.5. Sample aspect ratio . . . . . . . . . . . . . . . . . 13 68 3.2.6. SDPCapNeg support . . . . . . . . . . . . . . . . . . 13 69 3.2.7. Interaction with codec parameters . . . . . . . . . . 14 70 3.2.8. Change of display in middle of session . . . . . . . . 16 71 3.2.9. Use with layered codecs . . . . . . . . . . . . . . . 16 72 3.2.10. Addition of parameters . . . . . . . . . . . . . . . . 16 73 4. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 74 4.1. A High-Level Example . . . . . . . . . . . . . . . . . . . 16 75 4.2. Detailed Examples . . . . . . . . . . . . . . . . . . . . 17 76 4.2.1. Example 1 . . . . . . . . . . . . . . . . . . . . . . 17 77 4.2.2. Example 2 . . . . . . . . . . . . . . . . . . . . . . 18 78 4.2.3. Example 3 . . . . . . . . . . . . . . . . . . . . . . 19 79 4.2.4. Example 4 . . . . . . . . . . . . . . . . . . . . . . 20 80 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21 81 6. Security Considerations . . . . . . . . . . . . . . . . . . . 21 82 7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 22 83 8. Changes . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 84 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 24 85 9.1. Normative References . . . . . . . . . . . . . . . . . . . 24 86 9.2. Informative References . . . . . . . . . . . . . . . . . . 25 87 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 25 89 1. Introduction 91 This document proposes a new SDP attribute to make it possible to 92 negotiate different image attributes such as image size. The term 93 image size is defined here as it may differ from the physical screen 94 size of for instance a hand-held terminal. As an example it may be 95 beneficial to display a video image on a part of the physical screen 96 and leave space on the screen for other features such as menus and 97 other info. 99 There are a number of benefits with a possibility to negotiate the 100 image size: 102 o Less image distortion: Rescaling of images introduces additional 103 distortion, something that can be avoided (at least on the 104 receiver side) if the image size can be negotiated. 106 o Reduced receiver complexity: Image rescaling can be quite 107 computation intensive. For low end devices this can be a problem. 109 o Optimal quality for the given bitrate: The sender does not need to 110 encode an entire CIF (352x288) image if only an image size of 111 288x256 is displayed on the receiver screen. This alternatively 112 gives a saving in bitrate. 114 o Memory requirement: The receiver device will know the size of the 115 image and can then allocate memory accordingly. 117 o Optimal aspect ratio: The indication of the supported image sizes 118 and aspect ratio allows the receiver to select the most 119 appropriate combination based on its rescaling capabilities and 120 the desired rendering. For example, if a sender proposes three 121 resolutions in its SDP offer, 100x200, 200x100 and 100x100 with 122 sar=1.0 (1:1) etc. then the receiver can select the option that 123 fits the receiver screen best. 125 In cases where rescaling is not implemented (for example, rescaling 126 is not mandatory to implement in H.264 [H.264]), the indication of 127 the image attributes may still provide an optimal use of bandwidth 128 because the attribute will anyway give the encoder a better 129 indication about what image size is preferred and will thus help to 130 avoid wasting bandwidth by encoding with an unnecessarily large 131 resolution. 133 For implementers that are considering rescaling issues, it is worth 134 to notice that there are several benefits to do it on the sender 135 side: 137 o Rescaling on the sender/encoder side is likely to be easier to do 138 as the camera related software/hardware already contains the 139 necessary functionality for zooming/cropping/trimming/sharpening 140 the video signal. Moreover, rescaling is generally done in RGB or 141 YUV domain and should not depend on the codecs used. 143 o The encoder may be able to encode in a number of formats but may 144 not know which format to choose as, without the image attribute, 145 it does not know the receiver's performance or preference. 147 o The quality drop due to digital domain rescaling using 148 interpolation is likely to be lower if it is done before the video 149 encoding rather than after the decoding especially when low 150 bitrate video coding is used. 152 o If low-complexity rescaling operations such as simple cropping 153 must be performed, the benefit with having this functionality on 154 the sender side is that it is then possible to present a miniature 155 "what you send" image on the display to help the user to target 156 the camera correctly. 158 Several of the existing standards ([H.263], [H.264] and [MPEG-4]) 159 have support for different resolutions at different framerates. The 160 purpose of this document is to provide for a generic mechanism and is 161 targeted mainly at the negotiation of the image size but to make it 162 more general the attribute is named 'imageattr'. 164 This document is limited to point-to-point unicast communication 165 scenarios. The attribute may be used in centralized conferencing 166 scenarios as well but due to the abundance of configuration options 167 it may then be difficult to come up with a configuration that fits 168 all parties. 170 1.1. Requirements 172 The design of the image attribute is based on the following 173 requirements which are listed only for informational purposes: 175 REQ-1: Support the indication of one or more set(s) of image 176 attributes that the SDP endpoint wish to receive or send. Each 177 image attribute set must include a specific image size. 179 REQ-2: Support set up/negotiation of image attributes, meaning that 180 each side in the Offer/Answer should be able to negotiate the 181 image attributes it prefers to send and receive. 183 REQ-3: Interoperate with codec specific parameters such as sprop- 184 parameter-sets in [H.264] or config in [MPEG-4]. 186 REQ-4: Make the attribute generic with as few codec specific 187 details/tricks as possible in order to be codec agnostic. 189 Besides the above mentioned requirements, the requirement below may 190 be applicable. 192 OPT-1: The image attribute should support the description of image- 193 related attributes for various types of media, including video, 194 pictures, images, etc. 196 2. Conventions, Definitions and Acronyms 198 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 199 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 200 document are to be interpreted as described in [RFC2119]. 202 3. Specification of the 'imageattr' SDP Attribute 204 This section defines the SDP image attribute 'imageattr', which can 205 be used in an SDP Offer/Answer exchange to indicate various image 206 attribute parameters. In this document, we define the following 207 image attribute parameters: image resolution, sample aspect ratio 208 (sar), allowed range in picture aspect ratio (par) and the preference 209 of a given parameter set over another (q). The attribute is 210 extensible and guidelines for defining additional parameters are 211 provided in Section 3.2.10. 213 3.1. Attribute syntax 215 In this section the syntax of the 'imageattr' attribute is described. 216 The 'imageattr' attribute is a media-level attribute. The section is 217 split up in two parts, the first gives an overall view of the syntax 218 while the second describes how the syntax is used. 220 3.1.1. Overall view of syntax 222 The syntax for the image attribute is in ABNF [RFC5234]: 224 ---- 225 image-attr = "imageattr:" PT 1*2( 1*WSP ( "send" / "recv" ) 226 1*WSP attr-list ) 228 PT = 1*DIGIT / "*" 229 attr-list = ( set *(1*WSP set) ) / "*" 230 ; WSP and DIGIT defined in [RFC5234] 231 ---- 232 ---- 233 set= "[" "x=" xyrange "," "y=" xyrange *( "," key-value ) "]" 234 ; x is the horizontal image size range (pixel count) 235 ; y is the vertical image size range (pixel count) 236 key-value = ( "sar=" srange ) 237 / ( "par=" prange ) 238 / ( "q=" qvalue ) 239 ; Key-value MAY be extended with other keyword 240 ; parameters. 241 ; At most one instance each of sar, par, or q 242 ; allowed in a set. 243 ; 244 ; sar (sample aspect ratio) is the sample aspect ratio 245 ; associated with the set (optional,MAY be ignored) 246 ; par (picture aspect ratio) is the allowed 247 ; ratio between the display's x and y physical 248 ; size (optional) 249 ; q (optional, range [0.0..1.0], default value 0.5) 250 ; is the preference for the given set, 251 ; a higher value means a higher preference 253 ---- 254 onetonine = "1" / "2" / "3" / "4" / "5" / "6" / "7" / "8" / "9" 255 ; Digit between 1 and 9 256 xyvalue = onetonine *5DIGIT 257 ; Digit between 1 and 9 which is 258 ; followed by 0 to 5 other digits 259 step = xyvalue 260 xyrange = ( "[" xyvalue ":" [ step ":" ] xyvalue "]" ) 261 ; Range between a lower and an upper value 262 ; with an optional step, default step = 1 263 ; The rightmost occurrence of xyvalue MUST have a 264 ; higher value than the leftmost occurrence. 265 / ( "[" xyvalue 1*( "," xyvalue ) "]" ) 266 ; Discrete values separated by ',' 267 / ( xyvalue ) 268 ; A single value 269 spvalue = ( "0" "." onetonine *3DIGIT ) 270 ; Values between 0.1000 and 0.9999 271 / ( onetonine "." 1*4DIGIT ) 272 ; Values between 1.0000 and 9.9999 273 srange = ( "[" spvalue 1*( "," spvalue ) "]" ) 274 ; Discrete values separated by ','. 275 ; Each occurrence of spvalue MUST be 276 ; greater than the previous occurrence. 277 / ( "[" spvalue "-" spvalue "]" ) 278 ; Range between a lower and an upper level (inclusive) 279 ; The second occurrence of spvalue MUST have a higher 280 ; value than the first 281 / ( spvalue ) 282 ; A single value 283 prange = ( "[" spvalue "-" spvalue "]" ) 284 ; Range between a lower and an upper level (inclusive) 285 ; The second occurrence of spvalue MUST have a higher 286 ; value than the first 288 qvalue = ( "0" "." 1*2DIGIT ) 289 / ( "1" "." 1*2("0") ) 290 ; Values between 0.00 and 1.00 291 ---- 293 o The attribute typically contains a "send" and a "recv" keyword. 294 These specify the preferences for the media once the session is 295 set up, in the send and receive direction respectively from the 296 point of view of the sender of the session description. One of 297 the keywords ("send" or "recv") MAY be omitted, see Section 3.2.4 298 and Section 3.2.2 for a description of cases when this may be 299 appropriate. 301 o The "send" keyword and corresponding attribute list (attr-list) 302 MUST NOT occur more than once per image attribute. 304 o The "recv" keyword and corresponding attribute list (attr-list) 305 MUST NOT occur more than once per image attribute. 307 o PT is the payload type number, it MAY be set to "*" (wild card) to 308 indicate that the attribute applies to all payload types in the 309 media description. 311 o For sendrecv streams both of the send and recv directions SHOULD 312 be present in the SDP. 314 o For inactive streams it is RECOMMENDED that both of the send and 315 recv directions are present in the SDP. 317 3.1.1.1. Parameter rules 319 For the parameters the following rules apply. 321 Payload type number (PT): The image attribute is bound to a specific 322 codec by means of the payload type number. A wild card (*) can be 323 specified for the payload type number to indicate that it applies 324 to all payload types in the media description. Several image 325 attributes can be defined for instance for different video codec 326 alternatives. This however requires that the payload type numbers 327 differ. Note that the attribute is associated to the codec(s), 328 for instance an SDP offer may specify payload type number 101 329 while the answer may specify 102, this may make it troublesome to 330 specify a payload type number with the 'imageattr' attribute. See 331 Section 3.2.2 for a discussion and recommendation how this is 332 solved. 334 Preference (q): The preference for each set is 0.5 by default, 335 setting the optional q parameter to another value makes it 336 possible to set different preferences for the sets. A higher 337 value gives a higher preference for the given set. 339 sar: The sar (storage aspect ratio) parameter specifies the sample 340 aspect ratio associated to the given range of x and y values. The 341 sar parameter is defined as dx/dy where dx and dy is the physical 342 size of the pixels. Square pixels gives a sar=1.0. The parameter 343 sar MAY be expressed as a range or as a single value. 344 If this parameter is not present a default sar value of 1.0 is 345 assumed. 346 The interpretation of sar differs between the send and the receive 347 directions. 349 * In the send direction it defines a specific sample aspect ratio 350 associated to a given x and y image size (range). 352 * In the recv direction sar expresses that the receiver of the 353 given medium prefers to receive a given x and y resolution with 354 a given sample aspect ratio. 356 See Section 3.2.5 for a more detailed discussion. 357 The sar parameter will likely not solve all the issues that are 358 related to different sample aspect ratios but it can help to solve 359 them and reduce aspect ratio distortion. 360 The response MUST NOT include a sar parameter if there is no 361 acceptable value given. The reason to this is that if the 362 response includes a sar parameter it is interpreted as "sar 363 parameter accepted" while removal of the sar parameter is treated 364 as "sar parameter not accepted", for this reason it is safer to 365 remove an unacceptable sar parameter altogether. 367 par: The par (width/height = x/y ratio) parameter indicates a range 368 of allowed ratios between x and y physical size (picture aspect 369 ratio). This is used to limit the number of x and y image size 370 combinations, par is given as 371 ---- 372 par=[ratio_min-ratio_max] 373 ---- 374 Where ratio_min and ratio_max are the min and max allowed picture 375 aspect ratios. 376 If sar and the sample aspect ratio that the receiver actually uses 377 in the display are the same (or close), the relation between the x 378 and y pixel resolution and the physical size of the image is 379 straightforward. If however sar differs from the sample aspect 380 ratio of the receiver display this must be taken into 381 consideration when the x and y pixel resolution alternatives are 382 sorted out. See Section 4.2.4 for an example of this. 384 3.1.1.2. Offer/answer rules 386 In accordance with [RFC3264], offer answer exchange of the image 387 attribute is as follows. 389 o Offerer sending the offer: 391 * The offerer must be able to support the image attributes that 392 it offers, unless the offerer has expressed a wild card (*) in 393 the attribute list. 395 * It is recommended that a device which sees no reason to use the 396 image attribute, anyway includes the attribute with wild cards 397 (*) in the attribute lists for the send and recv directions. 398 An example of this looks like: 399 ---- 400 a=imageattr:97 send * recv * 401 ---- 402 This gives the answerer the possibility to express its 403 preferences. The use of wild cards introduces a risk that the 404 message size can increase in an uncontrolled way. To reduce 405 this risk these wild cards SHOULD only be replaced by an as 406 small set as possible. 408 o Answerer receiving the offer and sending the answer: 410 * The answerer may choose to keep the image attribute but is not 411 required to do so. 413 * The answerer may, for its receive and send direction, include 414 one or more entries that it can support from the set of entries 415 proposed in the offer. 417 * The answerer may also, for its receive and send direction, 418 replace the entries with a complete new set of entries 419 different from the original proposed by the offerer. The 420 implementor of this feature should however be aware that this 421 may cause extra offer/answer exchanges. 423 * The answerer may also remove its send direction completely if 424 it is deemed that it cannot support any of the proposed 425 entries. 427 * The answerer should not include an image attribute in the 428 answer if it was not present in the offer. 430 o Offerer receiving the answer: 432 * If the image attribute is not included in the SDP answer the 433 offerer SHOULD continue to process the answer as if this 434 mechanism had not been offered. 436 * If the image attribute is included in the SDP answer but none 437 of the entries are usable or acceptable, the offerer MUST 438 resort to other methods to determine the appropriate image 439 size. In this case the offerer must also issue a new offer/ 440 answer without the image attribute to avoid misunderstandings 441 between offerer and answerer. This will avoid the risk on 442 infinite negotiations. 444 3.2. Considerations 446 3.2.1. No imageattr in 1st offer 448 When the initial offer does not contain the 'imageattr' attribute, 449 the rules in Section 3.1.1.2 require the attribute to be absent in 450 the answer. The reasons for this are: 452 o The offerer of the initial SDP is not likely to understand the 453 image attribute if it did not include it in the offer, bearing in 454 mind that Section 3.1.1 recommends that the offerer provide the 455 attribute with wild carded parameters if it has no preference. 457 o Inclusion of the image attribute in the answer may come in 458 conflict with the rules in Section 3.1.1.2 esp. the rules that 459 apply to "offerer receiving the answer". 461 For the above reasons it is RECOMMENDED that a device which sees no 462 reason to use the image attribute, anyway includes the attribute with 463 wild cards (*) in the attribute lists for the send and recv 464 directions. 466 3.2.2. Different payload type numbers in offer and answer 468 In some cases the answer may specify a different media payload type 469 number than the offer. As an example the offer SDP may have the 470 m-line 471 ---- 472 m=video 49154 RTP/AVP 99 473 ---- 474 While the answer SDP may have the m-line 475 ---- 476 m=video 49154 RTP/AVP 100 477 ---- 479 If the image attribute in the offer specifies payload type number 99, 480 this attribute will then have no meaning in the answerers receive 481 direction as the m-line specifies media payload type number 100. 483 There are a few ways to solve this 485 1. Use a wild card "*" as payload type number in the image attribute 486 in the offer SDP. The answer SDP also use the wild card. The 487 drawback with this approach is that this attribute then applies 488 to all payload type numbers in the media description. 490 2. Specify a wild card "*" as payload type number in the image 491 attribute in the answer SDP. The offer SDP may contain a defined 492 payload type number in the image attribute but the answer SDP 493 replaces this with a wild card. The drawback here is similar to 494 what is listed above. 496 3. The image attribute is split in two parts in the SDP answer. For 497 example the offer SDP (only the parts of interest in this 498 discussion) looks like: 499 ---- 500 m=video 49154 RTP/AVP 99 501 a=imageattr:99 send ... recv ... 502 ---- 503 The answer SDP looks like: 504 ---- 505 m=video 49154 RTP/AVP 100 506 a=imageattr:99 send ... 507 a=imageattr:100 recv ... 508 ---- 509 This alternative does not pose any drawbacks. Moreover it allows 510 to specify different image attributes if more than one payload 511 type is specified in the offer SDP. 513 Of the alternatives listed above, the last one MUST be used as it is 514 the most safe. The other alternatives MUST NOT be used. 516 3.2.3. Asymmetry 518 While the image attribute supports asymmetry there are some 519 limitations to this. One important limitation is that the codec 520 being used can only support up to a given maximum resolution for a 521 given profile level. 523 As an example H.264 [H.264] with profile level 1.2 does not support 524 higher resolution than 352x288 (CIF). The offer/answer rules imply 525 that the same profile level must be used in both directions. This 526 means that in an asymmetric scenario where Alice wants an image size 527 of 580x360 and Bob wants 150x120, profile level 2.2 is needed in both 528 directions even though profile level 1 would have been sufficient in 529 one direction. 531 Currently, the only solution to this problem is to specify two 532 unidirectional media descriptions. Note however that the asymmetry 533 issue for the H.264 codec is solved by means of the level-asymmetry- 534 allowed parameter in [RFC3984bis]. 536 3.2.4. sendonly and recvonly 538 If the directional attributes a=sendonly or a=recvonly are given for 539 a medium, there is of course no need to specify the image attribute 540 for both directions. Therefore one of the directions in the 541 attribute may be omitted. However it may be good to do the image 542 attribute negotiation in both directions in case the session is 543 updated for media in both directions at a later stage. 545 3.2.5. Sample aspect ratio 547 The relationship between the sar parameter and the x and y pixel 548 resolution deserves some extra discussion. Consider the offer from 549 Alice to Bob (we set the recv direction aside for the moment): 550 ---- 551 a=imageattr:97 send [x=720,y=576,sar=1.1] 552 ---- 553 If the receiver display has square pixels the 720x576 image would 554 need to be rescaled to for example 792x576 or 720x524 to ensure a 555 correct image aspect ratio. This in practice means that rescaling 556 would need to be performed on the receiver side, something that is 557 contrary to the spirit of this draft. 558 To avoid this problem Alice may specify a range of values for the sar 559 parameter like: 560 ---- 561 a=imageattr:97 send [x=720,y=576,sar=[0.91,1.0,1.09,1.45]] 562 ---- 563 Meaning that Alice can encode with any of the mentioned sample aspect 564 ratios, leaving to Bob to decide which one he prefers. 566 3.2.6. SDPCapNeg support 568 The image attribute can be used within the SDP Capability Negotiation 569 [RFC5939] framework and its use is then specified using the "a=acap" 570 parameter. An example is 571 ---- 572 a=acap:1 imageattr:97 send [x=720,y=576,sar=[0.91,1.0,1.09,1.45]] 573 ---- 575 For use with SDP Media Capability Negotiation extension 576 [SDPMedCapNeg], where it is no longer possible to specify payload 577 type numbers, it is possible to use the parameter substitution rule, 578 an example of this is. 579 ---- 580 ... 581 a=mcap:1 video H264/90000 582 a=acap:1 imageattr:%1% send [x=720,y=576,sar=[0.91,1.0,1.09,1.45]] 583 ... 584 ---- 586 Where %1% maps to media capability number 1. 588 It is also possible to use the a=mscap attribute like in the example 589 below. 590 ---- 591 ... 592 a=mcap:1 video H264/90000 593 a=mscap:1 imageattr send [x=720,y=576,sar=[0.91,1.0,1.09,1.45]] 594 ... 595 ---- 597 3.2.7. Interaction with codec parameters 599 As the SDP for most codecs already specifies some kind of indication 600 of, for example, the image size, at session set up, measures must be 601 taken to avoid conflicts between the image attribute and this already 602 existing information. 604 The following subsections describe the most well known codecs and how 605 they define image-size related information. Section 3.2.7.4 outlines 606 a few possible solutions, but this document does not make any 607 recommendation for any of them. 609 3.2.7.1. H.263 611 The payload format for H.263 [H.263] is described in [RFC4629]. 613 H.263 defines (on the fmtp line) a list of image sizes and their 614 maximum frame rates (profiles) that the offerer can receive. The 615 answerer is not allowed to modify this list and must reject a payload 616 type that contains an unsupported profile. The CUSTOM profile may be 617 used for image size negotiation but support for asymmetry requires 618 the specification of two unidirectional media descriptions using the 619 sendonly/recvonly attributes. 621 3.2.7.2. H.264 623 The payload format for H.264 [H.264] is described in [RFC3984bis]. 625 H.264 defines image size related information in the fmtp line by 626 means of sprop-parameter-sets. According to the specification 627 several sprop-parameter-sets may be defined for one payload type. 628 The sprop-parameter-sets describe the image size (+ more) that the 629 offerer sends in the stream and need not be complete. This means 630 that this does not represent any negotiation. Moreover an answer is 631 not allowed to change the sprop-parameter-sets. 633 This configuration may be changed later inband if for instance image 634 sizes need to be changed or added. 636 3.2.7.3. MPEG-4 638 The payload format for MPEG-4 [MPEG-4] is described in [RFC3016]. 640 MPEG-4 defines a config parameter on the fmtp line which is a 641 hexadecimal representation of the MPEG-4 visual configuration 642 information. This configuration does not represent any negotiation 643 and the answer is not allowed to change the parameter. 645 It is not possible to change the configuration using inband 646 signaling. 648 3.2.7.4. Possible solutions 650 The subsections above clearly indicate that this kind of information 651 must be aligned well with the image attribute to avoid conflicts. 652 There are a number of possible solutions, listed below without any 653 preference: 655 o Ignore payload format parameters: This may not work well in the 656 presence of bad channel conditions especially in the beginning of 657 a session. Moreover this is not a good option for MPEG-4. 659 o 2nd session-wide offer/answer round: In the 2nd offer/answer the 660 codec payload format specific parameters are defined based on the 661 outcome of the 'imageattr' negotiation. The drawback with this is 662 that set up of the entire session (including audio) may be delayed 663 considerably, especially as the 'imageattr' negotiation can 664 already itself cost up to two offer/answer rounds. Also the 665 conflict between the 'imageattr' negotiation and the payload 666 format specific parameters is still present after the first offer/ 667 answer round and a fuzzy/buggy implementation may start media 668 before the second offer/answer is completed with unwanted results. 670 o 2nd session-wide offer/answer round only for video: This is 671 similar to the alternative above with the exception that set up 672 time for audio is not increased, moreover the port number for 673 video is set to 0 during the 1st offer answer round to avoid that 674 media flows. 675 This has the effect that video will blend in some time after the 676 audio is started (up to 2 seconds delay). This alternative is 677 likely the most clean-cut and failsafe alternative. The drawback 678 is, as the port number in the first offer is always zero, the 679 media startup will always be delayed even though it would in fact 680 have been possible to start media already after the first offer/ 681 answer round. 682 Note that according to [RFC3264], a port number of zero means that 683 the whole media line is rejected meaning that a new offer for the 684 same port number should be treated as a completely new stream and 685 not as an update. The most safe way to solve this problem is to 686 use preconditions, this is however outside the scope of this 687 document. 689 3.2.8. Change of display in middle of session 691 A very likely scenario is that a user switches to another phone 692 during a video telephony call or plugs a cellphone into an external 693 monitor. In both cases it is very likely that a renegotiation is 694 initiated using the SIP-REFER or SIP-UPDATE methods. It is 695 RECOMMENDED to negotiate the image size during this renegotiation. 697 3.2.9. Use with layered codecs 699 As the image attribute is a media level attribute, its use with 700 layered codecs causes some concern. If the layers are transported in 701 different RTP streams the layers are specified on different media 702 descriptions and the relation is specified using the grouping 703 framework [RFC5888] and the depend attribute [RFC5583]. As it is not 704 possible to specify only one image attribute for several media 705 descriptions the solution is either to specify the same image 706 attribute for each media description, or to only specify the image 707 attribute for the base layer. 709 3.2.10. Addition of parameters 711 The image attribute allows for the addition of parameters in the 712 future. To make backwards adaptation possible; an entity that 713 processes the attribute MUST ignore any unknown parameters in the 714 offer and MUST NOT include them in the answer it generates. Addition 715 of future parameters that are not understood by the receiving 716 endpoint may lead to ambiguities if mutual dependencies between 717 parameters exist, therefore addition of parameters must be done with 718 great care. 720 4. Examples 722 This section gives some more information on how to use the attribute 723 by means of a high-level example and a few detailed examples. 725 4.1. A High-Level Example 727 Assume that Alice wishes to set up a session with Bob and that Alice 728 takes the first initiative. The syntactical white-space delimiters 729 (1*WSP) and double-quotes are removed to make reading easier. 731 In the offer Alice provides information for both the send and receive 732 (recv) directions. For the send direction Alice provides a list that 733 the answerer can select from. For the receive direction Alice may 734 either specify a desired image size range right away or a * to 735 instruct Bob to reply with a list of image sizes that Bob can support 736 for sending. Using the overall high level syntax the image attribute 737 may then look like 738 ---- 739 a=imageattr:PT send attr-list recv attr-list 740 ---- 741 or 742 ---- 743 a=imageattr:PT send attr-list recv * 744 ---- 745 In the first alternative the recv direction may be a full list of 746 desired image size formats. It may however (and most likely) just be 747 a list with one alternative for the preferred x and y resolution. 749 If Bob supports an x and y resolution in at least one of the X and Y 750 ranges given in the send attr-list and in the recv attr-list of the 751 offer, the answer from Bob will look like: 752 ---- 753 a=imageattr:PT send attr-list recv attr-list 754 ---- 755 And the offer answer negotiation is done. Worth noticing here is 756 that the attr-list will likely be pruned in the answer. While it may 757 contain many different alternatives in the offer it may in the end 758 contain just one or two alternatives. 760 If Bob does not support any x and y resolution in one of the provided 761 send or recv ranges given in the send attr-list or in the recv attr- 762 list, the corresponding part is removed completely. For instance, if 763 Bob doesn't support any of the offered alternatives in the recv attr- 764 list in the offer, the answer from Bob would look like: 765 ---- 766 a=imageattr:PT recv attr-list 767 ---- 769 4.2. Detailed Examples 771 This section gives a few detailed examples, it is assumed where 772 needed that Alice initiates a session with Bob 774 4.2.1. Example 1 776 Two image resolution alternatives are offered with 800x640 with 777 sar=1.1 having the highest preference 778 It is also indicated that Alice wish to display video with a 779 resolution of 330x250 on her display 780 ---- 781 a=imageattr:97 send [x=800,y=640,sar=1.1,q=0.6] [x=480,y=320] \ 782 recv [x=330,y=250] 783 ---- 785 In case Bob accepts the "recv [x=330,y=250]" the answer may look like 786 ---- 787 a=imageattr:97 recv [x=800,y=640,sar=1.1] \ 788 send [x=330,y=250] 789 ---- 791 Indicating that the receiver (Bob) wish the encoder (on Alice's side) 792 to compensate for a sample aspect ratio of 1.1 (11:10) and desires an 793 image size on its screen of 800x640. 795 There is however a possibility that "recv [x=330,y=250]" is not 796 supported. If the case, Bob may completely remove this part or 797 replace it with a list of supported image sizes. 798 ---- 799 a=imageattr:97 recv [x=800,y=640,sar=1.1] \ 800 send [x=[320:16:640],y=[240:16:480],par=[1.2-1.3]] 801 ---- 803 Alice can then select a valid image size which is closest to the one 804 that was originally desired (336x256) and performs a second offer/ 805 answer 806 ---- 807 a=imageattr:97 send [x=800,y=640,sar=1.1] \ 808 recv [x=336,y=256] 809 ---- 811 Bob replies with: 812 ---- 813 a=imageattr:97 recv [x=800,y=640,sar=1.1] \ 814 send [x=336,y=256] 815 ---- 817 4.2.2. Example 2 819 Two image resolution sets are offered with the first having a higher 820 preference (q=0.6). 822 ---- 823 a=imageattr:97 \ 824 send [x=[480:16:800],y=[320:16:640],par=[1.2-1.3],q=0.6] \ 825 [x=[176:8:208],y=[144:8:176],par=[1.2-1.3]] \ 826 recv * 827 ---- 829 The x-axis resolution can take the values 480 to 800 in 16 pixels 830 steps and 176 to 208 in 8 pixels steps. The par parameter limits the 831 set of possible x and y screen resolution combinations such that 832 800x640 (ratio=1.25) is a valid combination while 720x608 833 (ratio=1.18) or 800x608 (ratio=1.31) are invalid combinations. 835 For the recv direction (Bob->Alice) Bob is requested to provide with 836 a list of supported image sizes 838 4.2.3. Example 3 840 In this example more of the SDP offer is shown. A complicating 841 factor is that the answerer changes the media payload type number in 842 the offer/answer exchange. 843 ---- 844 m=video 49154 RTP/AVP 99 845 a=rtpmap:99 H264/90000 846 a=fmtp:99 packetization-mode=0;profile-level-id=42e011; \ 847 sprop-parameter-sets=Z0LgC5ZUCg/I,aM4BrFSAa 848 a=imageattr:99 \ 849 send [x=176,y=144] [x=224,y=176] [x=272,y=224] [x=320,y=240] \ 850 recv [x=176,y=144] [x=224,y=176] [x=272,y=224,q=0.6] [x=320,y=240] 851 ---- 853 In the send direction, sprop-parameter-sets is defined for a 854 resolution of 320x240 which is the largest image size offered in the 855 send direction. This means that if 320x240 is selected, no 856 additional offer/answer is necessary. In the receive direction four 857 alternative image sizes are offered with 272x224 being the preferred 858 choice. 860 The answer may look like: 861 ---- 862 m=video 49154 RTP/AVP 100 863 a=rtpmap:100 H264/90000 864 a=fmtp:100 packetization-mode=0;profile-level-id=42e011; \ 865 sprop-parameter-sets=Z0LgC5ZUCg/I,aM4BrFSAa 866 a=imageattr:99 send [x=320,y=240] 867 a=imageattr:100 recv [x=320,y=240] 868 ---- 870 Indicating (in this example) that the image size is 320x240 in both 871 directions. Although the offerer preferred 272x224 for the receive 872 direction, the answerer might not be able to offer 272x224 or not 873 allow encoding and decoding of video of different image sizes 874 simultaneously. The answerer sets new sprop-parameter-sets, 875 constructed for both send and receive directions at the restricted 876 conditions and image size of 320x240. 878 Note also that, because the payload type number is changed by the 879 answerer, the image attribute is also split in two parts according to 880 the recommendation in Section 3.2.2. 882 4.2.4. Example 4 884 This example illustrates in more detail how compensation for 885 different sample aspect ratios can be negotiated with the image 886 attribute. 888 We set up a session between Alice and Bob, Alice is the offerer of 889 the session. The offer (from Alice) contains the image attribute 890 below: 891 ---- 892 a=imageattr:97 \ 893 send [x=400:16:800],y=[320:16:640],sar=[1.0-1.3],par=[1.2-1.3]] \ 894 recv [x=800,y=600,sar=1.1] 895 ---- 897 First we consider the recv direction: The offerer (Alice) explicitly 898 states that she wishes to receive the screen resolution 800x600, 899 however she also indicates that the screen on her display does not 900 use square pixels, the sar value=1.1 means that Bob must (preferably) 901 compensate for this. 902 So.. If Bob's video camera produces square pixels, and wishes to 903 satisfy Alice's sar requirement, the image processing algorithm must 904 rescale a 880x600 pixel image (880=800*1.1) to 800x600 pixels (could 905 be done other ways). 907 ... and now the send direction: Alice indicates that she can (in the 908 image processing algorithms) rescale the image for sample aspect 909 ratios in the range 1.0 to 1.3. She can also provide with a number 910 of different image sizes (in pixels) ranging from 400x320 to 800x640. 911 Bob inspects the offered sar and image sizes and responds with the 912 modified image attribute 913 ---- 914 a=imageattr:97 \ 915 recv [x=464,y=384,sar=1.15] \ 916 send [x=800,y=600,sar=1.1] 917 ---- 919 Alice will (in order to satisfy Bob's request) need to rescale the 920 image from her video camera from 534x384 (534=464*1.15) to 464x384. 922 5. IANA Considerations 924 Following the guidelines in [RFC4566], the IANA is requested to 925 register one new SDP attribute: 927 Attribute name: imageattr 929 Long form name: Image attribute 931 Type of attribute: Media-level 933 Subject to charset: No 935 Purpose: This attribute defines the ability to negotiate 936 various image attributes such as image sizes. 937 The attribute contains a number of parameters 938 which can be modified in and offer/answer 939 exchange. 941 Appropriate values: See Section 3.1.1 of RFCXXXX 943 Contact name: Authors of RFCXXXX 945 Note to RFC Editor: please replace "RFCXXXX" above with the RFC 946 number of this document, and remove this note. 948 6. Security Considerations 950 The image attribute and especially the parameters that denote the 951 image size can take on values that may cause memory or CPU exhaustion 952 problems. This may happen either as a consequence of a mistake by 953 the sender of the SDP or as a result of an attack issued by a 954 malicious SDP sender. This issue is similar to the case where the 955 a=fmtp line(s) may take on extreme values for the same reasons as 956 outlined above. 958 A receiver of the SDP containing the image attribute MUST ensure that 959 the parameters have values that are reasonable and that the device 960 can handle the implications in terms of memory and CPU usage. 961 Failure to do a sanity check on the parameters may result in memory 962 or CPU exhaustion. 964 In principle, for some SDPs containing the image attribute and for 965 some deployments, it could be the case that simply checking the 966 parameters is not sufficient to detect all potential DoS problems. 967 Implementers ought to consider whether there are any potential DoS 968 attacks that would not be detected by simply checking parameters. 970 7. Acknowledgements 972 The authors would like to thank the people who has contributed with 973 objections and suggestions to this draft and provided with valuable 974 guidance in the amazing video-coding world. Special thanks go to 975 Clinton Priddle, Roni Even, Randell Jesup, and Dan Wing. Thanks also 976 to Robert Sparks and Paul Kyzivat for the help with the last fixes to 977 get the attribute work well with the offer/answer model. 979 8. Changes 981 Note to RFC Editor: please replace remove this section in its 982 entirety before publication. 984 The main changes are: 986 From WG -10 to WG -11 988 * Changes after IESG review 990 From WG -09 to WG -10 992 * Further clarified the issue that offer and answer may use 993 different PT numbers, additional section added. 995 * Additional typos fixed. 997 From WG -08 to WG -09 999 * Clarified the issue that offer and answer may use different PT 1000 numbers 1002 * Clarified that wild cards in send and recv direction should be 1003 used with care and with total message size in mind 1005 * Typos and unclear language fixe 1007 From WG -07 to WG -08 1009 * SHOULD changed to MUST in section "Offer/answer rules" 1011 From WG -05 to WG -06 & -07 1013 * Update based on AD review comments, no changes to fix issue 1014 related to RFC2119 keywords 1016 * Minor editorial changes 1018 * Added extra example to use of attribute with SDPCapNeg 1020 From WG -04 to WG -05 1022 * Review based on WGLC comments 1024 * ABNF improved 1026 * Change use of RFC2119 keywords (MUST, SHOULD, MAY) to lowercase 1027 in some sections 1029 * Clarification on the directions send and recv in sendrecv, 1030 inactive modes 1032 * Clarification around sar parameter added 1034 From WG -03 to WG -04 1036 * Rearrangement of text 1038 * Clarification of offer/answer rules 1040 * Cleaned IANA section 1042 From WG -02 to WG -03 1044 * Partial update based on review comments from Jean-Francois Mule 1046 From WG -01 to WG -02 1048 * Added extra example that highlights the negotiation of sar 1050 From WG -00 to WG -01 1052 * Added info about future addition of parameters and backwards 1053 compatibility 1055 * Added IANA considerations 1057 From individual -02 to WG -00 1059 * Cleanup of syntax, ABNF form 1061 * Additional example 1063 From -01 to -02 1065 * Cleanup of the sar and par parameters to make them match the 1066 established conventions 1068 * Requirement specification added 1070 * New bidirectional syntax 1072 * Interoperability considerations with well known video codecs 1073 discussed 1075 9. References 1077 9.1. Normative References 1079 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1080 Requirement Levels", BCP 14, RFC 2119, March 1997. 1082 [RFC3264] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model 1083 with Session Description Protocol (SDP)", RFC 3264, 1084 June 2002. 1086 [RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session 1087 Description Protocol", RFC 4566, July 2006. 1089 [RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax 1090 Specifications: ABNF", STD 68, RFC 5234, January 2008. 1092 [RFC5583] Schierl, T. and S. Wenger, "Signaling Media Decoding 1093 Dependency in the Session Description Protocol (SDP)", 1094 RFC 5583, July 2009. 1096 [RFC5888] Camarillo, G. and H. Schulzrinne, "The Session Description 1097 Protocol (SDP) Grouping Framework", RFC 5888, June 2010. 1099 9.2. Informative References 1101 [H.263] ITU-T, "ITU-T Recommendation H.263 (2005): "Video coding 1102 for low bit rate communication"". 1104 [H.264] ITU-T, "ITU-T Recommendation H.264, 1105 http://www.itu.int/rec/T-REC-H.264-200711-I/en". 1107 [MPEG-4] ISO/IEC, "ISO/IEC 14496-2:2004: "Information technology - 1108 Coding of audio-visual objects - Part 2: Visual"". 1110 [RFC3016] Kikuchi, Y., Nomura, T., Fukunaga, S., Matsui, Y., and H. 1111 Kimata, "RTP Payload Format for MPEG-4 Audio/Visual 1112 Streams", RFC 3016, November 2000. 1114 [RFC3984bis] 1115 IETF, "RTP Payload Format for H.264 Video, http:// 1116 tools.ietf.org/wg/avt/draft-ietf-avt-rtp-rfc3984bis/". 1118 [RFC4629] Ott, H., Bormann, C., Sullivan, G., Wenger, S., and R. 1119 Even, "RTP Payload Format for ITU-T Rec", RFC 4629, 1120 January 2007. 1122 [RFC5939] Andreasen, F., "Session Description Protocol (SDP) 1123 Capability Negotiation", RFC 5939, September 2010. 1125 [SDPMedCapNeg] 1126 IETF, "SDP media capabilities Negotiation, http:// 1127 tools.ietf.org/wg/mmusic/ 1128 draft-ietf-mmusic-sdp-media-capabilities". 1130 Authors' Addresses 1132 Ingemar Johansson 1133 Ericsson AB 1134 Laboratoriegrand 11 1135 SE-971 28 Luleae 1136 SWEDEN 1138 Phone: +46 73 0783289 1139 Email: ingemar.s.johansson@ericsson.com 1140 Kyunghun Jung 1141 Samsung Electronics Co., Ltd. 1142 Dong Suwon P.O. Box 105 1143 416, Maetan-3Dong, Yeongtong-gu 1144 Suwon-city, Gyeonggi-do 1145 Korea 442-600 1147 Phone: +82 10 9909 4743 1148 Email: kyunghun.jung@samsung.com