idnits 2.17.1 draft-ietf-rtcweb-video-06.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (June 12, 2015) is 3240 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Possible downref: Non-RFC (?) normative reference: ref. 'H264' -- Possible downref: Non-RFC (?) normative reference: ref. 'HSUP1' == Outdated reference: A later version (-17) exists of draft-ietf-payload-vp8-16 == Outdated reference: A later version (-19) exists of draft-ietf-rtcweb-overview-13 -- Possible downref: Non-RFC (?) normative reference: ref. 'IEC23001-8' ** Downref: Normative reference to an Informational RFC: RFC 6386 -- Possible downref: Non-RFC (?) normative reference: ref. 'SRGB' == Outdated reference: A later version (-26) exists of draft-ietf-rtcweb-rtp-usage-24 == Outdated reference: A later version (-20) exists of draft-ietf-rtcweb-security-arch-11 == Outdated reference: A later version (-12) exists of draft-ietf-rtcweb-security-08 Summary: 1 error (**), 0 flaws (~~), 6 warnings (==), 5 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group A.B. Roach 3 Internet-Draft Mozilla 4 Intended status: Standards Track June 12, 2015 5 Expires: December 14, 2015 7 WebRTC Video Processing and Codec Requirements 8 draft-ietf-rtcweb-video-06 10 Abstract 12 This specification provides the requirements and considerations for 13 WebRTC applications to send and receive video across a network. It 14 specifies the video processing that is required, as well as video 15 codecs and their parameters. 17 Status of This Memo 19 This Internet-Draft is submitted in full conformance with the 20 provisions of BCP 78 and BCP 79. 22 Internet-Drafts are working documents of the Internet Engineering 23 Task Force (IETF). Note that other groups may also distribute 24 working documents as Internet-Drafts. The list of current Internet- 25 Drafts is at http://datatracker.ietf.org/drafts/current/. 27 Internet-Drafts are draft documents valid for a maximum of six months 28 and may be updated, replaced, or obsoleted by other documents at any 29 time. It is inappropriate to use Internet-Drafts as reference 30 material or to cite them other than as "work in progress." 32 This Internet-Draft will expire on December 14, 2015. 34 Copyright Notice 36 Copyright (c) 2015 IETF Trust and the persons identified as the 37 document authors. All rights reserved. 39 This document is subject to BCP 78 and the IETF Trust's Legal 40 Provisions Relating to IETF Documents 41 (http://trustee.ietf.org/license-info) in effect on the date of 42 publication of this document. Please review these documents 43 carefully, as they describe your rights and restrictions with respect 44 to this document. Code Components extracted from this document must 45 include Simplified BSD License text as described in Section 4.e of 46 the Trust Legal Provisions and are provided without warranty as 47 described in the Simplified BSD License. 49 Table of Contents 51 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 52 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 2 53 3. Pre and Post Processing . . . . . . . . . . . . . . . . . . . 2 54 3.1. Camera Source Video . . . . . . . . . . . . . . . . . . . 3 55 3.2. Screen Source Video . . . . . . . . . . . . . . . . . . . 3 56 4. Stream Orientation . . . . . . . . . . . . . . . . . . . . . 4 57 5. Mandatory to Implement Video Codec . . . . . . . . . . . . . 4 58 6. Codec-Specific Considerations . . . . . . . . . . . . . . . . 5 59 6.1. VP8 . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 60 6.2. H.264 . . . . . . . . . . . . . . . . . . . . . . . . . . 6 61 7. Security Considerations . . . . . . . . . . . . . . . . . . . 7 62 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 7 63 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 7 64 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 7 65 10.1. Normative References . . . . . . . . . . . . . . . . . . 7 66 10.2. Informative References . . . . . . . . . . . . . . . . . 8 67 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 9 69 1. Introduction 71 One of the major functions of WebRTC endpoints is the ability to send 72 and receive interactive video. The video might come from a camera, a 73 screen recording, a stored file, or some other source. This 74 specification provides the requirements and considerations for WebRTC 75 applications to send and receive video across a network. It 76 specifies the video processing that is required, as well as video 77 codecs and their parameters. 79 Note that this document only discusses those issues dealing with 80 video codec handling. Issues that are related to transport of media 81 streams across the network are specified in 82 [I-D.ietf-rtcweb-rtp-usage]. 84 2. Terminology 86 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 87 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 88 document are to be interpreted as described in [RFC2119]. 90 3. Pre and Post Processing 92 This section provides guidance on pre- and post-processing of video 93 streams. 95 Unless specified otherwise by the SDP or codec, the color space 96 SHOULD be sRGB [SRGB]. For clarity, this is the color space 97 indicated by codepoint 1 from "ColourPrimaries" as defined in 98 [IEC23001-8]. 100 Unless specified otherwise by the SDP or codec, the video scan 101 pattern for video codecs is Y'CbCr 4:2:0. 103 3.1. Camera Source Video 105 This document imposes no normative requirements on camera capture; 106 however, implementors are encouraged to take advantage of the 107 following features, if feasible for their platform: 109 o Automatic focus, if applicable for the camera in use 111 o Automatic white balance 113 o Automatic light level control 115 o Dynamic frame rate for video capture based on actual encoding in 116 use (e.g., if encoding at 15 fps due to bandwidth constraints, low 117 light conditions, or application settings, the camera will ideally 118 capture at 15 fps rather than a higher rate). 120 3.2. Screen Source Video 122 If the video source is some portion of a computer screen (e.g., 123 desktop or application sharing), then the considerations in this 124 section also apply. 126 Because screen-sourced video can change resolution (due to, e.g., 127 window resizing and similar operations), WebRTC video recipients MUST 128 be prepared to handle mid-stream resolution changes in a way that 129 preserves their utility. Precise handling (e.g., resizing the 130 element a video is rendered in versus scaling down the received 131 stream; decisions around letter/pillarboxing) is left to the 132 discretion of the application. 134 Note that the default video scan format (Y'CbCr 4:2:0) is known to be 135 less than optimal for the representation of screen content produced 136 by most systems in use at the time of this document's publication, 137 which generally use RGB with at least 24 bits per sample. In the 138 future, it may be advisable to use video codecs optimized for screen 139 content for the representation of this type of content. 141 Additionally, attention is drawn to the requirements in 142 [I-D.ietf-rtcweb-security-arch] section 5.2 and the considerations in 143 [I-D.ietf-rtcweb-security] section 4.1.1. 145 4. Stream Orientation 147 In some circumstances - and notably those involving mobile devices - 148 the orientation of the camera may not match the orientation used by 149 the encoder. Of more importance, the orientation may change over the 150 course of a call, requiring the receiver to change the orientation in 151 which it renders the stream. 153 While the sender may elect to simply change the pre-encoding 154 orientation of frames, this may not be practical or efficient (in 155 particular, in cases where the interface to the camera returns pre- 156 compressed video frames). Note that the potential for this behavior 157 adds another set of circumstances under which the resolution of a 158 screen might change in the middle of a video stream, in addition to 159 those mentioned under "Screen Sourced Video," above. 161 To accommodate these circumstances, RTCWEB implementations that can 162 generate media in orientations other than the default MUST support 163 generating the R0 and R1 bits of the Coordination of Video 164 Orientation (CVO) mechanism described in section 7.4.5 of [TS26.114], 165 and MUST send them for all orientations when the peer indicates 166 support for the mechanism. They MAY support sending the other bits 167 in the CVO extension, including the higher-resolution rotation bits. 168 All implementations SHOULD support interpretation of the R0 and R1 169 bits, and MAY support the other CVO bits. 171 Further, some codecs support in-band signaling of orientation (for 172 example, the SEI "Display Orientation" messages in H.264 and H.265). 173 If CVO has been negotiated, then the sender MUST NOT make use of such 174 codec-specific mechanisms. However, when support for CVO is not 175 signaled in the SDP, then such implementations MAY make use of the 176 codec-specific mechanisms instead. 178 5. Mandatory to Implement Video Codec 180 For the definitions of "WebRTC Browser," "WebRTC Non-Browser", and 181 "WebRTC-Compatible Endpoint" as they are used in this section, please 182 refer to [I-D.ietf-rtcweb-overview]. 184 WebRTC Browsers MUST implement the VP8 video codec as described in 185 [RFC6386] and H.264 Constrained Baseline as described in [H264]. 187 WebRTC Non-Browsers that support transmitting and/or receiving video 188 MUST implement the VP8 video codec as described in [RFC6386] and 189 H.264 Constrained Baseline as described in [H264]. 191 NOTE: To promote the use of non-royalty bearing video codecs, 192 participants in the RTCWEB working group, and any successor 193 working groups in the IETF, intend to monitor the evolving 194 licensing landscape as it pertains to the two mandatory-to- 195 implement codecs. If compelling evidence arises that one of the 196 codecs is available for use on a royalty-free basis, the working 197 group plans to revisit the question of which codecs are required 198 for Non-Browsers, with the intention being that the royalty-free 199 codec will remain mandatory to implement, and the other will 200 become optional. 202 These provisions apply to WebRTC Non-Browsers only. There is no 203 plan to revisit the codecs required for WebRTC Browsers. 205 "WebRTC-compatible endpoints" are free to implement any video codecs 206 they see fit. This follows logically from the definition of "WebRTC- 207 compatible endpoint." It is, of course, advisable to implement at 208 least one of the video codecs that is mandated for WebRTC Browsers, 209 and implementors are encouraged to do so. 211 6. Codec-Specific Considerations 213 SDP allows for codec-independent indication of preferred video 214 resolutions using the mechanism described in [RFC6236]. WebRTC 215 endpoints MAY send an "a=imageattr" attribute to indicate the maximum 216 resolution they wish to receive. Senders SHOULD interpret and honor 217 this attribute by limiting the encoded resolution to the indicated 218 maximum size, as the receiver may not be capable of handling higher 219 resolutions. 221 Additionally, codecs may include codec-specific means of signaling 222 maximum receiver abilities with regards to resolution, frame rate, 223 and bitrate. 225 Unless otherwise signaled in SDP, recipients of video streams MUST be 226 able to decode video at a rate of at least 20 fps at a resolution of 227 at least 320 pixels by 240 pixels. These values are selected based 228 on the recommendations in [HSUP1]. 230 Encoders are encouraged to support encoding media with at least the 231 same resolution and frame rates cited above. 233 6.1. VP8 235 For the VP8 codec, defined in [RFC6386], endpoints MUST support the 236 payload formats defined in [I-D.ietf-payload-vp8]. 238 In addition to the [RFC6236] mechanism, VP8 encoders MUST limit the 239 streams they send to conform to the values indicated by receivers in 240 the corresponding max-fr and max-fs SDP attributes. 242 Unless otherwise signaled, implementations that use VP8 MUST encode 243 and decode pixels with a implied 1:1 (square) aspect ratio. 245 6.2. H.264 247 For the [H264] codec, endpoints MUST support the payload formats 248 defined in [RFC6184]. In addition, they MUST support Constrained 249 Baseline Profile Level 1.2, and they SHOULD support H.264 Constrained 250 High Profile Level 1.3. 252 Implementations of the H.264 codec have utilized a wide variety of 253 optional parameters. To improve interoperability the following 254 parameter settings are specified: 256 packetization-mode: Packetization-mode 1 MUST be supported. Other 257 modes MAY be negotiated and used. 259 profile-level-id: Implementations MUST include this parameter within 260 SDP and MUST interpret it when receiving it. 262 max-mbps, max-smbps, max-fs, max-cpb, max-dpb, and max-br: These 264 parameters allow the implementation to specify that they can 265 support certain features of H.264 at higher rates and values than 266 those signalled by their level (set with profile-level-id). 267 Implementations MAY include these parameters in their SDP, but 268 SHOULD interpret them when receiving them, allowing them to send 269 the highest quality of video possible. 271 sprop-parameter-sets: H.264 allows sequence and picture information 272 to be sent both in-band, and out-of-band. WebRTC implementations 273 MUST signal this information in-band. This means that WebRTC 274 implementations MUST NOT include this parameter in the SDP they 275 generate. 277 H.264 codecs MAY send and MUST support proper interpretation of SEI 278 "filler payload" and "full frame freeze" messages. "Full frame 279 freeze" messages are used in video switching MCUs, to ensure a stable 280 decoded displayed picture while switching among various input 281 streams. 283 When the use of the video orientation (CVO) RTP header extension is 284 not signaled as part of the SDP, H.264 implementations MAY send and 285 SHOULD support proper interpretation of Display Orientation SEI 286 messages. 288 Implementations MAY send and act upon "User data registered by Rec. 289 ITU-T T.35" and "User data unregistered" messages. Even if they do 290 not act on them, implementations MUST be prepared to receive such 291 messages without any ill effects. 293 Unless otherwise signaled, implementations that use H.264 MUST encode 294 and decode pixels with a implied 1:1 (square) aspect ratio. 296 7. Security Considerations 298 This specification does not introduce any new mechanisms or security 299 concerns beyond what is in the other documents it references. In 300 WebRTC, video is protected using DTLS/SRTP. A complete discussion of 301 the security considerations can be found in 302 [I-D.ietf-rtcweb-security] and [I-D.ietf-rtcweb-security-arch]. 303 Implementors should consider whether the use of variable bit rate 304 video codecs are appropriate for their application, keeping in mind 305 that the degree of inter-frame change (and, by inference, the amount 306 of motion in the frame) may be deduced by an eavesdropper based on 307 the video stream's bit rate. 309 Implementors making use of H.264 are also advised to take careful 310 note of the "Security Considerations" section of [RFC6184], paying 311 special regard to the normative requirement pertaining to SEI 312 messages. 314 8. IANA Considerations 316 This document requires no actions from IANA. 318 9. Acknowledgements 320 The author would like to thank Gaelle Martin-Cocher, Stephan Wenger, 321 and Bernard Aboba for their detailed feedback and assistance with 322 this document. Thanks to Cullen Jennings for providing text and 323 review, and to Russ Housley for a careful final review. This draft 324 includes text from draft-cbran-rtcweb-codec. 326 10. References 328 10.1. Normative References 330 [H264] ITU-T Recommendation H.264, "Advanced video coding for 331 generic audiovisual services (V9)", February 2014, 332 . 334 [HSUP1] ITU-T Recommendation H.Sup1, "Application profile - Sign 335 language and lip-reading real-time conversation using low 336 bit rate video communication", May 1999, 337 . 339 [I-D.ietf-payload-vp8] 340 Westin, P., Lundin, H., Glover, M., Uberti, J., and F. 341 Galligan, "RTP Payload Format for VP8 Video", draft-ietf- 342 payload-vp8-16 (work in progress), June 2015. 344 [I-D.ietf-rtcweb-overview] 345 Alvestrand, H., "Overview: Real Time Protocols for 346 Browser-based Applications", draft-ietf-rtcweb-overview-13 347 (work in progress), November 2014. 349 [IEC23001-8] 350 ISO/IEC 23001-8:2013/DCOR1, "Coding independent media 351 description code points", 2013, . 355 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 356 Requirement Levels", BCP 14, RFC 2119, March 1997. 358 [RFC6184] Wang, Y.-K., Even, R., Kristensen, T., and R. Jesup, "RTP 359 Payload Format for H.264 Video", RFC 6184, May 2011. 361 [RFC6236] Johansson, I. and K. Jung, "Negotiation of Generic Image 362 Attributes in the Session Description Protocol (SDP)", RFC 363 6236, May 2011. 365 [RFC6386] Bankoski, J., Koleszar, J., Quillio, L., Salonen, J., 366 Wilkins, P., and Y. Xu, "VP8 Data Format and Decoding 367 Guide", RFC 6386, November 2011. 369 [SRGB] IEC 61966-2-1, "Multimedia systems and equipment - Colour 370 measurement and management - Part 2-1: Colour management - 371 Default RGB colour space - sRGB.", October 1999, . 374 [TS26.114] 375 3GPP TS 26.114 V12.8.0, "3rd Generation Partnership 376 Project; Technical Specification Group Services and System 377 Aspects; IP Multimedia Subsystem (IMS); Multimedia 378 Telephony; Media handling and interaction (Release 12)", 379 December 2014, . 381 10.2. Informative References 383 [I-D.ietf-rtcweb-rtp-usage] 384 Perkins, C., Westerlund, M., and J. Ott, "Web Real-Time 385 Communication (WebRTC): Media Transport and Use of RTP", 386 draft-ietf-rtcweb-rtp-usage-24 (work in progress), May 387 2015. 389 [I-D.ietf-rtcweb-security-arch] 390 Rescorla, E., "WebRTC Security Architecture", draft-ietf- 391 rtcweb-security-arch-11 (work in progress), March 2015. 393 [I-D.ietf-rtcweb-security] 394 Rescorla, E., "Security Considerations for WebRTC", draft- 395 ietf-rtcweb-security-08 (work in progress), February 2015. 397 Author's Address 399 Adam Roach 400 Mozilla 401 \ 402 Dallas 403 US 405 Phone: +1 650 903 0800 x863 406 Email: adam@nostrum.com