idnits 2.17.1 draft-garcia-simulcast-and-layered-video-webrtc-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (August 05, 2013) is 3915 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'I-D.narten-iana-considerations-rfc2434bis' is defined on line 302, but no explicit reference was found in the text == Unused Reference: 'RFC2629' is defined on line 314, but no explicit reference was found in the text == Unused Reference: 'VP8' is defined on line 317, but no explicit reference was found in the text == Outdated reference: A later version (-16) exists of draft-ietf-rtcweb-use-cases-and-requirements-11 -- Obsolete informational reference (is this intentional?): RFC 2629 (Obsoleted by RFC 7749) Summary: 0 errors (**), 0 flaws (~~), 6 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 RTCWEB Working Group G. Garcia 3 Internet-Draft TokBox 4 Intended status: Informational August 05, 2013 5 Expires: February 06, 2014 7 Simulcast and layered video coding support in WebRTC 8 draft-garcia-simulcast-and-layered-video-webrtc-00 10 Abstract 12 This document describes the use cases and requirements for simulcast 13 and layered video coding support in WebRTC. These techniques 14 simplify the implementation of video stream adaptation to different 15 participants in centralized conferencing solutions. This document 16 also includes a proposal to expose these capabilities in the existing 17 PeerConnection API by defining new media constraint properties. 19 Status of This Memo 21 This Internet-Draft is submitted in full conformance with the 22 provisions of BCP 78 and BCP 79. 24 Internet-Drafts are working documents of the Internet Engineering 25 Task Force (IETF). Note that other groups may also distribute 26 working documents as Internet-Drafts. The list of current Internet- 27 Drafts is at http://datatracker.ietf.org/drafts/current/. 29 Internet-Drafts are draft documents valid for a maximum of six months 30 and may be updated, replaced, or obsoleted by other documents at any 31 time. It is inappropriate to use Internet-Drafts as reference 32 material or to cite them other than as "work in progress." 34 This Internet-Draft will expire on February 06, 2014. 36 Copyright Notice 38 Copyright (c) 2013 IETF Trust and the persons identified as the 39 document authors. All rights reserved. 41 This document is subject to BCP 78 and the IETF Trust's Legal 42 Provisions Relating to IETF Documents 43 (http://trustee.ietf.org/license-info) in effect on the date of 44 publication of this document. Please review these documents 45 carefully, as they describe your rights and restrictions with respect 46 to this document. Code Components extracted from this document must 47 include Simplified BSD License text as described in Section 4.e of 48 the Trust Legal Provisions and are provided without warranty as 49 described in the Simplified BSD License. 51 Table of Contents 53 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 54 1.1. Browser support status . . . . . . . . . . . . . . . . . 3 55 1.2. Requirements Language . . . . . . . . . . . . . . . . . . 3 56 2. Use-cases . . . . . . . . . . . . . . . . . . . . . . . . . . 3 57 2.1. Adaptation to devices with different capabilities . . . . 3 58 2.2. Adaptation to participants with different network 59 conditions . . . . . . . . . . . . . . . . . . . . . . . 3 60 2.3. Recording . . . . . . . . . . . . . . . . . . . . . . . . 4 61 2.4. Increasing video quality for active speaker . . . . . . . 4 62 3. Requirements . . . . . . . . . . . . . . . . . . . . . . . . 4 63 4. Proposed API . . . . . . . . . . . . . . . . . . . . . . . . 5 64 4.1. Simulcasted streams . . . . . . . . . . . . . . . . . . . 5 65 4.2. Layered video coding . . . . . . . . . . . . . . . . . . 5 66 4.3. Example . . . . . . . . . . . . . . . . . . . . . . . . . 6 67 5. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 7 68 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 7 69 7. Security Considerations . . . . . . . . . . . . . . . . . . . 7 70 8. References . . . . . . . . . . . . . . . . . . . . . . . . . 7 71 8.1. Normative References . . . . . . . . . . . . . . . . . . 7 72 8.2. Informative References . . . . . . . . . . . . . . . . . 7 73 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 8 75 1. Introduction 77 Video conferencing using a central server is one of the typical use 78 cases for real-time communication capabilities in browsers 79 [I-D.ietf-rtcweb-use-cases-and-requirements]. 81 Most of today's multiparty videoconference solutions make use of 82 centralized servers to reduce the bandwidth and CPU consumption in 83 the endpoints. Those servers receive streams from each participant 84 and send the streams to rest of the participants, which usually have 85 heterogeneous capabilities (screen size, CPU, bandwidth, etc.). One 86 of the biggest issues is how to perform the adaption to different 87 participants' constraints with the minimum possible impact on video 88 quality and server performance. 90 There are two approaches to adapt the streams to different 91 destinations: one is transcoding (sometimes including mixing), and 92 the other is switching between multiple streams or sub-streams 93 received from the originator. The first solution is computationally 94 expensive and can degrade video quality. The second solution makes a 95 suboptimal use of network resources by sending redundant information, 96 and in addition it is codec-specific. 98 The requirements and proposed API in this document are based on 99 existing JSEP API version and VP8 capabilities. These are the 100 technologies available in existing WebRTC browsers, but this proposal 101 could be extended to other codecs or mapped to other APIs. 103 1.1. Browser support status 105 It is possible to use simulcast with existing WebRTC implementations. 106 However, this requires the use of different PeerConnection objects, 107 and all streams will have the same resolution. 109 Multi-layer encoding is implemented and working in existing WebRTC 110 browsers, and it has been tested in prototypes, but currently there 111 is no way for developers to enable it. In VP8 there is support for 112 temporal scalability, while VP9 will include more advanced control 113 and support for both temporal and spatial scalability. 115 1.2. Requirements Language 117 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 118 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 119 document are to be interpreted as described in RFC 2119 [RFC2119]. 121 2. Use-cases 123 The use cases envisioned for these new WebRTC capabilities are 124 focused on centralized conferencing solutions. 126 2.1. Adaptation to devices with different capabilities 128 Some endpoints connected to a centralized conferencing server have 129 small screens and do not need to receive high-resolution video, or 130 the CPU power and battery consumption make it impossible to receive 131 and decode high-resolution video in real-time. 133 In this situation, it is desirable to send lower-resolution video to 134 those endpoints. 136 2.2. Adaptation to participants with different network conditions 137 Some endpoints connected to a centralized conferencing server do not 138 have enough available bandwidth to receive high-quality video, while 139 other endpoints have enough available bandwidth. 141 In this situation is desirable to send lower-bitrate video to those 142 endpoints. 144 2.3. Recording 146 A conferencing server implements recording and wants to record video 147 in the highest quality possible, while forwarding it in lower quality 148 to endpoints. 150 2.4. Increasing video quality for active speaker 152 A videoconference application shows the video of the active speaker 153 in a larger size than videos of the other participants. 155 It is desirable to increase the resolution and quality of that 156 highlighted video stream, to maintain the perceived video quality. 157 One possible implementation to increase the quality is to have a 158 paused high-quality stream that resumes when voice activity is 159 detected. 161 3. Requirements 163 This section contains the requirements for the API exposed in the 164 browser, derived from the use-cases in Section 2. 166 Requirements on how and when to enable scalable video coding: 168 o REQ-1. It must be possible to enable and configure the scalable 169 video coding before initiating a peer connection. 171 o REQ-2. It must be possible to enable and configure the scalable 172 video coding before answering a peer connection. 174 o REQ-3. It must be possible to enable/disable and re-configure the 175 scalable video coding to update a peer connection. 177 Requirements on the parameters that needs to be configurable: 179 o REQ-5. It must be possible to configure the number of simulcasted 180 streams. 182 o REQ-6. It must be possible to configure the minimum and maximum 183 bitrate of each simulcasted stream. 185 o REQ-7. It must be possible to configure the resolution of each 186 simulcasted stream. 188 o REQ-8. It must be possible to configure the number of temporal 189 layers (1 to 4). This should be the only mandatory parameter when 190 enabling temporal scalability. 192 o REQ-9. It must be possible to configure the bitrate, frame rate 193 decimation factor and membership of frames to layers for each 194 temporal layer of the VP8 stream. 196 Requirements regarding RTP usage: 198 o REQ-10. Congestion control must be supported for all the 199 simulcasted streams between the configured boundaries (min/max 200 bitrate). 202 o REQ-11. Transmission of simulcasted streams must be signaled and 203 negotiated in the SDP and transmitted in RTP sessions, making use 204 of existing standard attributes 205 [I-D.westerlund-avtcore-multistream-and-simulcast]. 207 o REQ-12. Any endpoint should be prepared to receive VP8 multi- 208 layered encoded video not requiring out of band negotiation in 209 SDP. 211 Non functional requirements: 213 o REQ-13. The exposed API must be extensible to new codecs or new 214 codec parameters. 216 4. Proposed API 218 The existing solution in the WebRTC API to modify settings of a 219 PeerConnection is to use media constraints. This section defines 220 some new media constrains to enable and configure the usage of 221 simulcasted and layered video streams. 223 4.1. Simulcasted streams 225 Simulcast capabilities are codec-agnostic and do not require new 226 media constraints. Existing media constrains for resolution, frame 227 rate and bitrate can be reused, but the API needs to support 228 receiving a list of them instead of just one. 230 4.2. Layered video coding 231 Multi-layer capabilities are codec-dependent. For VP8, these are the 232 configuration parameters exposed in the codec, and that needs to be 233 translated to media constraints (the descriptions are taken from VP8 234 source code): 236 o tsNumberLayers: This value specifies the number of coding layers 237 to be used. 239 o tsTargetBitrate: These values specify the target coding bitrate 240 for each coding layer. 242 o tsRateDecimator: These values specify the frame rate decimation 243 factors to apply to each layer. 245 o tsPeriodicity: This value specifies the length of the sequence 246 that defines the membership of frames to layers. For example, if 247 tsPeriodicity=8 then frames are assigned to coding layers with a 248 repeated sequence of length 8. 250 o tsLayerId: This array defines the membership of frames to coding 251 layers. For a 2-layer encoding that assigns even numbered frames 252 to one layer (0) and odd numbered frames to a second layer (1) 253 with tsPeriodicity=8, then tsLayerId = (0,1,0,1,0,1,0,1). 255 4.3. Example 257 Example of media constraints to request two simulcasted streams, the 258 first one with four temporal layers and default bitrate and the 259 second one with a single layer and fixed bitrate. 261 { 262 video: [{ 263 width: 640, 264 height: 480, 265 codecs: { 266 vp8: { tsNumberLayers: 4 } 267 } 268 }, 269 { 270 width: 320, 271 height: 240, 272 bitrate: { min: 100000, max: 100000 } 273 }] 274 } 275 } 277 5. Acknowledgements 279 6. IANA Considerations 281 This memo includes no request to IANA. 283 7. Security Considerations 285 No security implications foreseen. 287 8. References 289 8.1. Normative References 291 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 292 Requirement Levels", BCP 14, RFC 2119, March 1997. 294 8.2. Informative References 296 [I-D.ietf-rtcweb-use-cases-and-requirements] 297 Holmberg, C., Hakansson, S., and G. Eriksson, "Web Real- 298 Time Communication Use-cases and Requirements", draft- 299 ietf-rtcweb-use-cases-and-requirements-11 (work in 300 progress), June 2013. 302 [I-D.narten-iana-considerations-rfc2434bis] 303 Narten, T. and H. Alvestrand, "Guidelines for Writing an 304 IANA Considerations Section in RFCs", draft-narten-iana- 305 considerations-rfc2434bis-09 (work in progress), March 306 2008. 308 [I-D.westerlund-avtcore-multistream-and-simulcast] 309 Westerlund, M. and B. Burman, "RTP Multiple Stream 310 Sessions and Simulcast", draft-westerlund-avtcore- 311 multistream-and-simulcast-00 (work in progress), July 312 2011. 314 [RFC2629] Rose, M., "Writing I-Ds and RFCs using XML", RFC 2629, 315 June 1999. 317 [VP8] The WebM Project, "VP8 source code", 2013, 318 . 320 Author's Address 322 Gustavo Garcia 323 TokBox 324 115 Stillman Street 325 San Francisco, CA 326 US 328 Email: gustavo@tokbox.com