idnits 2.17.1 draft-ietf-rtcweb-video-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (October 27, 2014) is 3467 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: 'RFC4175' is defined on line 341, but no explicit reference was found in the text == Unused Reference: 'RFC4421' is defined on line 344, but no explicit reference was found in the text == Unused Reference: 'RFC5104' is defined on line 348, but no explicit reference was found in the text -- Possible downref: Non-RFC (?) normative reference: ref. 'H264' -- Possible downref: Non-RFC (?) normative reference: ref. 'HSUP1' == Outdated reference: A later version (-17) exists of draft-ietf-payload-vp8-11 -- Possible downref: Non-RFC (?) normative reference: ref. 'IEC23001-8' ** Downref: Normative reference to an Informational RFC: RFC 6386 -- Possible downref: Non-RFC (?) normative reference: ref. 'SRGB' == Outdated reference: A later version (-26) exists of draft-ietf-rtcweb-rtp-usage-06 == Outdated reference: A later version (-20) exists of draft-ietf-rtcweb-security-arch-09 == Outdated reference: A later version (-12) exists of draft-ietf-rtcweb-security-06 Summary: 1 error (**), 0 flaws (~~), 8 warnings (==), 5 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group A.B. Roach 3 Internet-Draft Mozilla 4 Intended status: Standards Track October 27, 2014 5 Expires: April 30, 2015 7 WebRTC Video Processing and Codec Requirements 8 draft-ietf-rtcweb-video-02 10 Abstract 12 This specification provides the requirements and considerations for 13 WebRTC applications to send and receive video across a network. It 14 specifies the video processing that is required, as well as video 15 codecs and their parameters. 17 Status of This Memo 19 This Internet-Draft is submitted in full conformance with the 20 provisions of BCP 78 and BCP 79. 22 Internet-Drafts are working documents of the Internet Engineering 23 Task Force (IETF). Note that other groups may also distribute 24 working documents as Internet-Drafts. The list of current Internet- 25 Drafts is at http://datatracker.ietf.org/drafts/current/. 27 Internet-Drafts are draft documents valid for a maximum of six months 28 and may be updated, replaced, or obsoleted by other documents at any 29 time. It is inappropriate to use Internet-Drafts as reference 30 material or to cite them other than as "work in progress." 32 This Internet-Draft will expire on April 30, 2015. 34 Copyright Notice 36 Copyright (c) 2014 IETF Trust and the persons identified as the 37 document authors. All rights reserved. 39 This document is subject to BCP 78 and the IETF Trust's Legal 40 Provisions Relating to IETF Documents 41 (http://trustee.ietf.org/license-info) in effect on the date of 42 publication of this document. Please review these documents 43 carefully, as they describe your rights and restrictions with respect 44 to this document. Code Components extracted from this document must 45 include Simplified BSD License text as described in Section 4.e of 46 the Trust Legal Provisions and are provided without warranty as 47 described in the Simplified BSD License. 49 Table of Contents 51 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 52 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 2 53 3. Pre and Post Processing . . . . . . . . . . . . . . . . . . . 2 54 3.1. Camera Source Video . . . . . . . . . . . . . . . . . . . 3 55 3.2. Screen Source Video . . . . . . . . . . . . . . . . . . . 3 56 4. Stream Orientation . . . . . . . . . . . . . . . . . . . . . 4 57 5. Codec-Specific Considerations . . . . . . . . . . . . . . . . 4 58 5.1. VP8 . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 59 5.2. H.264 . . . . . . . . . . . . . . . . . . . . . . . . . . 5 60 6. Mandatory to Implement Video Codec . . . . . . . . . . . . . 6 61 6.1. Temperature of Working Group . . . . . . . . . . . . . . 6 62 7. Security Considerations . . . . . . . . . . . . . . . . . . . 7 63 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 7 64 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 7 65 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 7 66 10.1. Normative References . . . . . . . . . . . . . . . . . . 7 67 10.2. Informative References . . . . . . . . . . . . . . . . . 9 68 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 9 70 1. Introduction 72 One of the major functions of WebRTC endpoints is the ability to send 73 and receive interactive video. The video might come from a camera, a 74 screen recording, a stored file, or some other source. This 75 specification defines how the video is used and discusses special 76 considerations for processing the video. It also covers the video- 77 related algorithms WebRTC devices need to support. 79 Note that this document only discusses those issues dealing with 80 video codec handling. Issues that are related to transport of media 81 streams across the network are specified in 82 [I-D.ietf-rtcweb-rtp-usage]. 84 2. Terminology 86 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 87 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 88 document are to be interpreted as described in [RFC2119]. 90 3. Pre and Post Processing 92 This section provides guidance on pre- or post-processing of video 93 streams. 95 Unless specified otherwise by the SDP or codec, the color space 96 SHOULD be sRGB [SRGB]. 98 TODO: I'm just throwing this out there to see if a specific proposal, 99 even if wrong, might draw more comment than "TBD". If you don't like 100 sRGB for this purpose, comment on the rtcweb@ietf.org mailing list. 101 It has been suggested that the MPEG "Coding independent media 102 description code points" specification [IEC23001-8] may have 103 applicability here. 105 3.1. Camera Source Video 107 This document imposes no normative requirements on camera capture; 108 however, implementors are encouraged to take advantage of the 109 following features, if feasible for their platform: 111 o Automatic focus, if applicable for the camera in use 113 o Automatic white balance 115 o Automatic light level control 117 3.2. Screen Source Video 119 If the video source is some portion of a computer screen (e.g., 120 desktop or application sharing), then the considerations in this 121 section also apply. 123 Because screen-sourced video can change resolution (due to, e.g., 124 window resizing and similar operations), WebRTC video recipients MUST 125 be prepared to handle mid-stream resolution changes in a way that 126 preserves their utility. Precise handling (e.g., resizing the 127 element a video is rendered in versus scaling down the received 128 stream; decisions around letter/pillarboxing) is left to the 129 discretion of the application. 131 Additionally, attention is drawn to the requirements in 132 [I-D.ietf-rtcweb-security-arch] section 5.2 and the considerations in 133 [I-D.ietf-rtcweb-security] section 4.1.1. 135 TODO: Do we want to define additional metadata to indicate whether a 136 stream is sourced from a camera versus a screen capture? This would 137 allow the receiving party to tune, e.g., output filters. It would 138 appear that H.263 has this kind of indicator built into its 139 bitstream, but I found no analog in H.264 or VP8. 141 4. Stream Orientation 143 In some circumstances - and notably those involving mobile devices - 144 the orientation of the camera may not match the orientation used by 145 the encoder. Of more importance, the orientation may change over the 146 course of a call, requiring the receiver to change the orientation in 147 which it renders the stream. 149 While the sender may elect to simply change the pre-encoding 150 orientation of frames, this may not be practical or efficient (in 151 particular, in cases where the interface to the camera returns pre- 152 compressed video frames). Note that the potential for this behavior 153 adds another set of circumstances under which the resolution of a 154 screen might change in the middle of a video stream, in addition to 155 those mentioned under "Screen Sourced Video," above. 157 To accommodate these circumstances, RTCWEB implementations SHOULD 158 support generating and receiving the R0 and R1 bits of the 159 Coordination of Video Orientation (CVO) mechanism described in 160 section 7.4.5 of [TS26.114]. (TODO: Is "SHOULD support" the right 161 level here?) They MAY support the other bits in the CVO extension, 162 including the higher-resolution rotation bits. 164 Further, some codecs support in-band signaling of orientation (for 165 example, the SEI "Display Orientation" messages in H.264 and H.265). 166 If CVO has been negotiated, then the sender MUST NOT make use of such 167 codec-specific mechanisms. However, when support for CVO is not 168 signaled in the SDP, then such implementations MAY make use of the 169 codec-specific mechanisms instead. 171 5. Codec-Specific Considerations 173 WebRTC endpoints are not required to support the codecs mentioned in 174 this section. 176 However, to foster interoperability between endpoints that have 177 codecs in common, if they do support one of the listed codecs, then 178 they need to meet the requirements specified in the subsection for 179 that codec. 181 SDP allows for codec-independent indication of preferred video 182 resolutions using the mechanism described in [RFC6236]. If a 183 recipient of video indicates a receiving resolution, the sender 184 SHOULD accommodate this resolution, as the receiver may not be 185 capable of handling higher resolutions. 187 Additionally, codecs may include codec-specific means of signaling 188 maximum receiver abilities with regards to resolution, frame rate, 189 and bitrate. 191 Unless otherwise signaled in SDP, recipients of video streams are 192 MUST be able to decode video at a rate of at least 20 fps at a 193 resolution of at least 320x240. These values are selected based on 194 the recommendations in [HSUP1]. 196 Encoders are encouraged to support encoding media with at least the 197 same resolution and frame rates cited above. 199 5.1. VP8 201 If VP8, defined in [RFC6386], is supported, then the endpoint MUST 202 support the payload formats defined in [I-D.ietf-payload-vp8]. In 203 addition it MUST support the 'bilinear' and 'none' reconstruction 204 filters. 206 TODO: There have been claims that VP8 already requires supporting 207 both filters; if true, these do not need to be reiterated here. 209 In addition to the [RFC6236] mechanism, VP8 encoders MUST limit the 210 streams they send to conform to the values indicated by receivers in 211 the corresponding max-fr and max-fs SDP attributes. 213 5.2. H.264 215 If [H264] is supported, then the device MUST support the payload 216 formats defined in [RFC6184]. In addition, they MUST support 217 Constrained Baseline Profile Level 1.2, and they SHOULD support H.264 218 Constrained High Profile Level 1.3. 220 Implementations of the H.264 codec have utilized a wide variety of 221 optional parameters. To improve interoperability the following 222 parameter settings are specified: 224 packetization-mode: Packetization-mode 1 MUST be supported. Other 225 modes MAY be negotiated and used. 227 profile-level-id: Implementations MUST include this parameter within 228 SDP and SHOULD interpret it when receiving it. 230 max-mbps, max-smbps, max-fs, max-cpb, max-dpb, and max-br: These par 231 ameters allow the implementation to specify that they can support 232 certain features of H.264 at higher rates and values than those 233 signalled by their level (set with profile-level-id). 234 Implementations MAY include these parameters in their SDP, but 235 SHOULD interpret them when receiving them, allowing them to send 236 the highest quality of video possible. 238 sprop-parameter-sets: H.264 allows sequence and picture information 239 to be sent both in-band, and out-of-band. WebRTC implementations 240 MUST signal this information in-band; as a result, this parameter 241 will not be present in SDP. 243 TODO: Do we need to require the handling of specific SEI messages? 244 One example that has been raised is freeze-frame messages. 246 6. Mandatory to Implement Video Codec 248 Note: This section is here purely as a placeholder, as there is not 249 yet WG Consensus on Mandatory to Implement video codecs. The issue 250 is more complicated than may be immediately apparent to newcomers, 251 who are strongly encouraged to familiarize themselves with the 252 previous discussions on the topic before engaging on this issue. 254 The currently recorded working group consensus is that all 255 implementations MUST support a single, specified mandatory-to- 256 implement codec. The remaining decision point is a selection of this 257 single codec. 259 6.1. Temperature of Working Group 261 To capture the conversation so far, this section summarizes the 262 result of a straw poll that the working group undertook in December 263 2013 and January 2014. Respondents were asked to answer "Yes," 264 "Acceptable," or "No" for each option. The options were collected 265 from the working group at large prior to the initiation of the straw 266 poll. 268 Yes Acc No 269 --- --- --- 270 1. All entities MUST support H.264 48% 11% 41% 271 2. All entities MUST support VP8 41% 17% 42% 272 3. All entities MUST support both H.264 and VP8 9% 38% 53% 273 4. Browsers MUST support both H.264 and VP8, other 274 entities MUST support at least one of H.264 275 and VP8 11% 34% 55% 276 5. All entities MUST support at least one of 277 H.264 and VP8 10% 16% 74% 278 6. All entities MUST support H.261 5% 23% 72% 279 7. There is no MTI video codec 12% 30% 58% 280 8. All entities MUST support H.261 and all entities 281 MUST support at least one of H.264 and VP8 4% 28% 68% 282 9. All entities MUST support Theora 7% 26% 67% 284 10. All entities MUST implement at least two of 285 {VP8, H.264, H.261} 5% 30% 65% 286 11. All entities MUST implement at least two of 287 {VP8, H.264, H.263} 5% 25% 70% 288 12. All entities MUST support decoding using both 289 H.264 and VP8, and MUST support encoding using 290 at least one of H.264 or VP8 7% 20% 73% 291 13. All entities MUST support H.263 6% 19% 75% 292 14. All entities MUST implement at least two of 293 {VP8, H.264, Theora} 6% 27% 67% 294 15. All entities MUST support decoding using Theora 1% 15% 84% 295 16. All entities MUST support Motion JPEG 1% 25% 74% 297 7. Security Considerations 299 This specification does not introduce any new mechanisms or security 300 concerns beyond what the other documents it references. In WebRTC, 301 video is protected using DTLS/SRTP. A complete discussion of the 302 security can be found in [I-D.ietf-rtcweb-security] and 303 [I-D.ietf-rtcweb-security-arch]. Implementers should consider 304 whether the use of variable bit rate video codecs are appropriate for 305 their application based on [RFC6562]. 307 8. IANA Considerations 309 This document requires no actions from IANA. 311 9. Acknowledgements 313 The authors would like to thank Gaelle Martin-Cocher, Stephan Wenger, 314 and Bernard Aboba for their detailed feedback and assistance with 315 this document. Thanks to Cullen Jennings for providing text and 316 review. This draft includes text from draft-cbran-rtcweb-codec. 318 10. References 320 10.1. Normative References 322 [H264] ITU-T Recommendation H.264, "Advanced video coding for 323 generic audiovisual services", April 2013. 325 [HSUP1] ITU-T Recommendation H.Sup1, "Application profile - Sign 326 language and lip-reading real-time conversation using low 327 bit rate video communication", May 1999. 329 [I-D.ietf-payload-vp8] 330 Westin, P., Lundin, H., Glover, M., Uberti, J., and F. 331 Galligan, "RTP Payload Format for VP8 Video", draft-ietf- 332 payload-vp8-11 (work in progress), February 2014. 334 [IEC23001-8] 335 ISO/IEC 23001-8:2013/DCOR1, "Coding independent media 336 description code points", 2013. 338 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 339 Requirement Levels", BCP 14, RFC 2119, March 1997. 341 [RFC4175] Gharai, L. and C. Perkins, "RTP Payload Format for 342 Uncompressed Video", RFC 4175, September 2005. 344 [RFC4421] Perkins, C., "RTP Payload Format for Uncompressed Video: 345 Additional Colour Sampling Modes", RFC 4421, February 346 2006. 348 [RFC5104] Wenger, S., Chandra, U., Westerlund, M., and B. Burman, 349 "Codec Control Messages in the RTP Audio-Visual Profile 350 with Feedback (AVPF)", RFC 5104, February 2008. 352 [RFC6184] Wang, Y.-K., Even, R., Kristensen, T., and R. Jesup, "RTP 353 Payload Format for H.264 Video", RFC 6184, May 2011. 355 [RFC6236] Johansson, I. and K. Jung, "Negotiation of Generic Image 356 Attributes in the Session Description Protocol (SDP)", RFC 357 6236, May 2011. 359 [RFC6386] Bankoski, J., Koleszar, J., Quillio, L., Salonen, J., 360 Wilkins, P., and Y. Xu, "VP8 Data Format and Decoding 361 Guide", RFC 6386, November 2011. 363 [RFC6562] Perkins, C. and JM. Valin, "Guidelines for the Use of 364 Variable Bit Rate Audio with Secure RTP", RFC 6562, March 365 2012. 367 [SRGB] IEC 61966-2-1, "Multimedia systems and equipment - Colour 368 measurement and management - Part 2-1: Colour management - 369 Default RGB colour space - sRGB.", October 1999. 371 [TS26.114] 372 3GPP TS 26.114 V12.7.0, "3rd Generation Partnership 373 Project; Technical Specification Group Services and System 374 Aspects; IP Multimedia Subsystem (IMS); Multimedia 375 Telephony; Media handling and interaction (Release 12)", 376 September 2014. 378 10.2. Informative References 380 [I-D.ietf-rtcweb-rtp-usage] 381 Perkins, C., Westerlund, M., and J. Ott, "Web Real-Time 382 Communication (WebRTC): Media Transport and Use of RTP", 383 draft-ietf-rtcweb-rtp-usage-06 (work in progress), 384 February 2013. 386 [I-D.ietf-rtcweb-security-arch] 387 Rescorla, E., "WebRTC Security Architecture", draft-ietf- 388 rtcweb-security-arch-09 (work in progress), February 2014. 390 [I-D.ietf-rtcweb-security] 391 Rescorla, E., "Security Considerations for WebRTC", draft- 392 ietf-rtcweb-security-06 (work in progress), January 2014. 394 Author's Address 396 Adam Roach 397 Mozilla 398 \ 399 Dallas 400 US 402 Phone: +1 650 903 0800 x863 403 Email: adam@nostrum.com