idnits 2.17.1 

draft-ietf-netvc-requirements-07.txt:

  Checking boilerplate required by RFC 5378 and the IETF Trust (see
  https://trustee.ietf.org/license-info):
  ----------------------------------------------------------------------------

     No issues found here.

  Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt:
  ----------------------------------------------------------------------------

     No issues found here.

  Checking nits according to https://www.ietf.org/id-info/checklist :
  ----------------------------------------------------------------------------

     No issues found here.

  Miscellaneous warnings:
  ----------------------------------------------------------------------------

  == The copyright year in the IETF Trust and authors Copyright Line does not
     match the current year

  -- The document date (October 30, 2017) is 2369 days in the past.  Is this
     intentional?


  Checking references for intended status: Informational
  ----------------------------------------------------------------------------

     No issues found here.

     Summary: 0 errors (**), 0 flaws (~~), 1 warning (==), 1 comment (--).

     Run idnits with the --verbose option for more detailed information about
     the items above.
--------------------------------------------------------------------------------

1	Network Working Group                                       A. Filippov
2	Internet Draft                                      Huawei Technologies
3	Intended status: Informational                                A. Norkin
4	                                                                Netflix
5	                                                           J.R. Alvarez
6	                                                    Huawei Technologies
7	Expires: April 30, 2018                                October 30, 2017

9	           <Video Codec Requirements and Evaluation Methodology>
10	                   draft-ietf-netvc-requirements-07.txt

12	Status of this Memo

14	   This Internet-Draft is submitted in full conformance with the
15	   provisions of BCP 78 and BCP 79.

17	   Internet-Drafts are working documents of the Internet Engineering
18	   Task Force (IETF), its areas, and its working groups.  Note that
19	   other groups may also distribute working documents as Internet-
20	   Drafts. The list of current Internet-Drafts can be accessed at
21	   http://datatracker.ietf.org/drafts/current/

23	   Internet-Drafts are draft documents valid for a maximum of six
24	   months and may be updated, replaced, or obsoleted by other documents
25	   at any time. It is inappropriate to use Internet-Drafts as reference
26	   material or to cite them other than as "work in progress."

28	   The list of current Internet-Drafts can be accessed at
29	   http://www.ietf.org/1id-abstracts.html

31	   The list of Internet-Draft Shadow Directories can be accessed at
32	   http://www.ietf.org/shadow.html

34	   This Internet-Draft will expire on April 30, 2018.

36	Copyright Notice

38	   Copyright (c) 2017 IETF Trust and the persons identified as the
39	   document authors. All rights reserved.

41	   This document is subject to BCP 78 and the IETF Trust's Legal
42	   Provisions Relating to IETF Documents
43	   (http://trustee.ietf.org/license-info) in effect on the date of
44	   publication of this document. Please review these documents
45	   carefully, as they describe your rights and restrictions with
46	   respect to this document. Code Components extracted from this
47	   document must include Simplified BSD License text as described in
48	   Section 4.e of the Trust Legal Provisions and are provided without
49	   warranty as described in the Simplified BSD License.

51	Abstract

53	   This document provides requirements for a video codec designed
54	   mainly for use over the Internet. In addition, this document
55	   describes an evaluation methodology needed for measuring the
56	   compression efficiency to ensure whether the stated requirements are
57	   fulfilled or not.

59	   Table of Contents

61	   1. Introduction...................................................3
62	   2. Applications...................................................3
63	      2.1. Internet Video Streaming..................................4
64	      2.2. Internet Protocol Television (IPTV).......................6
65	      2.3. Video conferencing........................................7
66	      2.4. Video sharing.............................................8
67	      2.5. Screencasting.............................................9
68	      2.6. Game streaming...........................................10
69	      2.7. Video monitoring / surveillance..........................11
70	   3. Requirements..................................................12
71	      3.1. General requirements.....................................12
72	      3.2. Basic requirements.......................................14
73	         3.2.1. Input source formats................................14
74	         3.2.2. Coding delay........................................14
75	         3.2.3. Complexity..........................................15
76	         3.2.4. Scalability.........................................15
77	         3.2.5. Error resilience....................................15
78	      3.3. Optional requirements....................................16
79	         3.3.1. Input source formats................................16
80	         3.3.2. Scalability.........................................16
81	         3.3.3. Complexity..........................................16
82	         3.3.4. Coding efficiency...................................16
83	   4. Evaluation methodology........................................17
84	      4.1. Compression performance evaluation.......................17
85	      4.2. Reference software.......................................20
86	   5. Security Considerations.......................................20
87	   6. Conclusions...................................................20
88	   7. IANA Considerations...........................................20
89	   8. References....................................................20
90	      8.1. Normative References.....................................20
91	      8.2. Informative References...................................21
92	   9. Acknowledgments...............................................21
93	   Appendix A. Abbreviations used in the text of this document......23
94	   Appendix B. Used terms...........................................24

96	1. Introduction

98	   In this document, the requirements for a video codec designed mainly
99	   for use over the Internet are presented. The requirements encompass
100	   a wide range of applications that use data transmission over the
101	   Internet including Internet video streaming, IPTV, peer-to-peer
102	   video conferencing, video sharing, screencasting, game streaming and
103	   video monitoring / surveillance. For each application, typical
104	   resolutions, frame-rates and picture access modes are presented.
105	   Specific requirements related to data transmission over packet-loss
106	   networks are considered as well. In this document, when we discuss
107	   data protection techniques we only refer to methods designed and
108	   implemented to protect data inside the video codec since there are
109	   many existing techniques that protect generic data transmitted over
110	   networks with packet losses. From the theoretical point of view,
111	   both packet-loss and bit-error robustness can be beneficial for
112	   video codecs. In practice, packet losses are a more significant
113	   problem than bit corruption in IP networks. It is worth noting that
114	   there is an evident interdependence between possible amount of delay
115	   and the necessity of error robust video streams:

117	   o  If an amount of delay is not crucial for an application, then
118	      reliable transport protocols such as TCP that retransmits
119	      undelivered packets can be used to guarantee correct decoding of
120	      transmitted data.

122	   o  If the amount of delay must be kept low, then either data
123	      transmission should be error free (e.g., by using managed
124	      networks) or compressed video stream should be error resilient.

126	   Thus, error resilience can be useful for delay-critical applications
127	   to provide low delay in packet-loss environment.

129	2. Applications

131	   In this chapter, an overview of video codec applications that are
132	   currently available on the Internet market is presented. It is worth
133	   noting that there are different use cases for each application that
134	   define a target platform, and hence there are different types of
135	   communication channels involved (e.g., wired or wireless channels)
136	   that are characterized by different quality of service as well as
137	   bandwidth; for instance, wired channels are considerably more error-
138	   free than wireless channels and therefore require different QoS
139	   approaches. The target platform, the channel bandwidth and the
140	   channel quality determine resolutions, frame-rates and quality or
141	   bit-rates for video streams to be encoded or decoded. By default,
142	   color format YCbCr 4:2:0 is assumed for the application scenarios
143	   listed below.

145	2.1. Internet Video Streaming

147	   Typical content for this application is movies, TV-series and shows,
148	   and animation. Internet video streaming uses a variety of client
149	   devices and has to operate under changing network conditions. For
150	   this reason, an adaptive streaming model has been widely adopted.
151	   Video material is encoded at different quality levels and different
152	   resolutions, which are then chosen by a client depending on its
153	   capabilities and current network bandwidth. An example combination
154	   of resolutions and bitrates is shown in Table 1 below.

156	   +----------------------+-------------------------+-----------------+
157	   |      Resolution  *   |     Frame-rate, fps     |       PAM       |
158	   +----------------------+-------------------------+-----------------+
159	   +----------------------+-------------------------+-----------------+
160	   |    4K, 3840x2160     |    24/1.001, 24, 25,    |       RA        |
161	   +----------------------+                         +-----------------+
162	   | 2K (1080p), 1920x1080|    30/1.001, 30, 50,    |       RA        |
163	   +----------------------+                         +-----------------+
164	   |   1080i, 1920x1080*  |    60/1.001, 60, 100,   |       RA        |
165	   +----------------------+                         +-----------------+
166	   |    720p, 1280x720    |      120/1.001, 120     |       RA        |
167	   +----------------------+                         +-----------------+
168	   | 576p (EDTV), 720x576 | The set of frame-rates  |       RA        |
169	   +----------------------+                         +-----------------+
170	   | 576i (SDTV), 720x576*| presented in this table |       RA        |
171	   +----------------------+                         +-----------------+
172	   | 480p (EDTV), 720x480 |  is taken from Table 2  |       RA        |
173	   +----------------------+                         +-----------------+
174	   | 480i (SDTV), 720x480*|          in [1]         |       RA        |
175	   +----------------------+                         +-----------------+
176	   |       512x384        |                         |       RA        |
177	   +----------------------+                         +-----------------+
178	   |    QVGA, 320x240     |                         |       RA        |
179	   +----------------------+-------------------------+-----------------+
180	   Table 1. Internet Video Streaming: typical values of resolutions,
181	   frame-rates, and RAPs

183	   NB *: Interlaced content can be handled at the higher system level
184	   and not necessarily by using specialized video coding tools. It is
185	   included in this table only for the sake of completeness as most
186	   video content today is in the progressive format.

188	   A video encoding pipeline in on-demand Internet video streaming
189	   typically operates as follows:

191	   o  Video is encoded in the cloud by software encoders.

193	   o  Source video is split into chunks, each of which is encoded
194	      separately, in parallel.

196	   o  Closed-GOP encoding with 2-5 second intra-picture intervals (or
197	      more) is used.

199	   o  Encoding is perceptually optimized. Perceptual quality is
200	      important and should be considered during the codec development.

202	   Characteristics and requirements of this application scenario are as
203	   follows:

205	   o  High encoder complexity (up to 10x and more) can be tolerated
206	      since encoding happens once and in parallel for different
207	      segments.

209	   o  Decoding complexity should be kept at reasonable levels to enable
210	      efficient decoder implementation.

212	   o  Support and efficient encoding of a wide range of content types
213	      and formats is required:

215	       . High Dynamic Range (HDR), Wide Color Gamut (WCG), high
216	          resolution (currently, up to 4K), high frame-rate content are
217	          important use cases, the codec should be able to encode such
218	          content efficiently.

220	       . Coding efficiency improvement at both lower and higher
221	          resolutions is important since low resolutions are used when
222	          streaming in low bandwidth conditions.

224	       . Improvement on both "easy" and "difficult" content in terms
225	          of compression efficiency at the same quality level
226	          contributes to the overall bitrate/storage savings.

228	       . Film grain (and sometimes other types of noise) is often
229	          present in the streaming movie-type content and is usually a
230	          part of the creative intent.

232	   o  Significant improvements in compression efficiency between
233	      generations of video standards are desirable since this scenario
234	      typically assumes long-term support of legacy video codecs.

236	   o  Random access points are inserted frequently (one per 2-5
237	      seconds) to enable switching between resolutions and fast-forward
238	      playback.

240	   o  Elementary stream should have a model that allows easy parsing
241	      and identification of the sample components.

243	   o  Middle QP values are normally used in streaming, this is also the
244	      range where compression efficiency is important for this
245	      scenario.

247	   o  Scalability or other forms of supporting multiple quality
248	      representations are beneficial if they do not incur significant
249	      bitrate overhead and if mandated in the first version.

251	2.2. Internet Protocol Television (IPTV)

253	   This is a service for delivering television content over IP-based
254	   networks. IPTV may be classified into two main groups based on the
255	   type of delivery, as follows:

257	   o  unicast (e.g., for video on demand), where delay is not crucial;

259	   o  multicast/broadcast (e.g., for transmitting news) where zapping,
260	      i.e. stream changing, delay is important.

262	   In the IPTV scenario, traffic is transmitted over managed (QoS-
263	   based) networks. Typical content used in this application is news,
264	   movies, cartoons, series, TV shows, etc. One important requirement
265	   for both groups is Random access to pictures, i.e. random access
266	   period (RAP) should be kept small enough (approximately, 1-5
267	   seconds). Optional requirements are as follows:

269	   o  Temporal (frame-rate) scalability;

271	   o  Resolution and quality (SNR) scalability.

273	   For this application, typical values of resolutions, frame-rates,
274	   and RAPs are presented in Table 2.

276	   +----------------------+-------------------------+-----------------+
277	   |      Resolution  *   |     Frame-rate, fps     |       PAM       |
278	   +----------------------+-------------------------+-----------------+
279	   +----------------------+-------------------------+-----------------+
280	   | 2160p (4K),3840x2160 |    24/1.001, 24, 25,    |       RA        |
281	   +----------------------+                         +-----------------+
282	   |   1080p, 1920x1080   |    30/1.001, 30, 50,    |       RA        |
283	   +----------------------+                         +-----------------+
284	   |   1080i, 1920x1080*  |    60/1.001, 60, 100,   |       RA        |
285	   +----------------------+                         +-----------------+
286	   |    720p, 1280x720    |      120/1.001, 120     |       RA        |
287	   +----------------------+                         +-----------------+
288	   | 576p (EDTV), 720x576 | The set of frame-rates  |       RA        |
289	   +----------------------+                         +-----------------+
290	   | 576i (SDTV), 720x576*| presented in this table |       RA        |
291	   +----------------------+                         +-----------------+
292	   | 480p (EDTV), 720x480 |  is taken from Table 2  |       RA        |
293	   +----------------------+                         +-----------------+
294	   | 480i (SDTV), 720x480*|          in [1]         |       RA        |
295	   +----------------------+-------------------------+-----------------+
296	   Table 2. IPTV: typical values of resolutions, frame-rates, and RAPs

298	   NB *: Interlaced content can be handled at the higher system level
299	   and not necessarily by using specialized video coding tools. It is
300	   included in this table only for the sake of completeness as most
301	   video content today is in progressive format.

303	2.3. Video conferencing

305	   This is a form of video connection over the Internet. This form
306	   allows users to establish connections to two or more people by two-
307	   way video and audio transmission for communication in real-time. For
308	   this application, both stationary and mobile devices can be used.
309	   The main requirements are as follows:

311	   o  Delay should be kept as low as possible (the preferable and
312	      maximum end-to-end delay values should be less than 100 ms [7]
313	      and 320 ms [2], respectively);

315	   o  Temporal (frame-rate) scalability;

317	   o  Error robustness.

319	   Support of resolution and quality (SNR) scalability is highly
320	   desirable. For this application, typical values of resolutions,
321	   frame-rates, and RAPs are presented in Table 3.

323	   +----------------------+-------------------------+----------------+
324	   |      Resolution      |     Frame-rate, fps     |      PAM       |
325	   +----------------------+-------------------------+----------------+
326	   +----------------------+-------------------------+----------------+
327	   |  1080p,  1920x1080   |          15, 30         |      FIZD      |
328	   +----------------------+-------------------------+----------------+
329	   |  720p,  1280x720     |          30, 60         |      FIZD      |
330	   +----------------------+-------------------------+----------------+
331	   |  4CIF,  704x576      |          30, 60         |      FIZD      |
332	   +----------------------+-------------------------+----------------+
333	   |  4SIF,  704x480      |          30, 60         |      FIZD      |
334	   +----------------------+-------------------------+----------------+
335	   |  VGA,  640x480       |          30, 60         |      FIZD      |
336	   +----------------------+-------------------------+----------------+
337	   |  360p,  640x360      |          30, 60         |      FIZD      |
338	   +----------------------+-------------------------+----------------+

340	   Table 3. Video conferencing: typical values of resolutions, frame-
341	   rates, and RAPs

343	2.4. Video sharing

345	   This is a service that allows people to upload and share video data
346	   (using live streaming or not) and to watch them. It is also known as
347	   video hosting. A typical User-generated Content (UGC) scenario for
348	   this application is to capture video using mobile cameras such as
349	   GoPro or cameras integrated into smartphones (amateur video). The
350	   main requirements are as follows:

352	   o  Random access to pictures for downloaded video data;

354	   o  Temporal (frame-rate) scalability;

356	   o  Error robustness.

358	   Support of resolution and quality (SNR) scalability is highly
359	   desirable. For this application, typical values of resolutions,
360	   frame-rates, and RAPs are presented in Table 4.

362	   +----------------------+-------------------------+----------------+
363	   |      Resolution      |     Frame-rate, fps     |       PAM      |
364	   +----------------------+-------------------------+----------------+
365	   +----------------------+-------------------------+----------------+
366	   | 2160p (4K),3840x2160 |  24, 25, 30, 48, 50, 60 |       RA       |
367	   +----------------------+-------------------------+----------------+
368	   | 1440p (2K),2560x1440 |  24, 25, 30, 48, 50, 60 |       RA       |
369	   +----------------------+-------------------------+----------------+
370	   | 1080p, 1920x1080     |  24, 25, 30, 48, 50, 60 |       RA       |
371	   +----------------------+-------------------------+----------------+
372	   | 720p, 1280x720       |  24, 25, 30, 48, 50, 60 |       RA       |
373	   +----------------------+-------------------------+----------------+
374	   | 480p, 854x480        |  24, 25, 30, 48, 50, 60 |       RA       |
375	   +----------------------+-------------------------+----------------+
376	   | 360p, 640x360        |  24, 25, 30, 48, 50, 60 |       RA       |
377	   +----------------------+-------------------------+----------------+
378	   Table 4. Video sharing: typical values of resolutions, frame-rates
379	   [8], and RAPs

381	2.5. Screencasting

383	   This is a service that allows users to record and distribute
384	   computer desktop screen output. This service requires efficient
385	   compression of computer-generated content with high visual quality
386	   up to visually and mathematically (numerically) lossless [9].
387	   Currently, this application includes business presentations
388	   (powerpoint, word documents, email messages, etc.), animation
389	   (cartoons), gaming content, data visualization, i.e. such type of
390	   content that is characterized by fast motion, rotation, smooth
391	   shade, 3D effect, highly saturated colors with full resolution,
392	   clear textures and sharp edges with distinct colors [9]), virtual
393	   desktop infrastructure (VDI), screen/desktop sharing and
394	   collaboration, supervisory control and data acquisition (SCADA)
395	   display, automotive/navigation display, cloud gaming, factory
396	   automation display, wireless display, display wall, digital
397	   operating room (DiOR), etc. For this application, an important
398	   requirement is the support of low-delay configurations with zero
399	   structural delay, a wide range of video formats (e.g., RGB) in
400	   addition to YCbCr 4:2:0 and YCbCr 4:4:4 [9]. For this application,
401	   typical values of resolutions, frame-rates, and RAPs are presented
402	   in Table 5.

404	   +----------------------+-------------------------+----------------+
405	   |      Resolution      |     Frame-rate, fps     |       PAM      |
406	   +----------------------+-------------------------+----------------+
407	   +----------------------+-------------------------+----------------+
408	   |                     Input color format: RGB 4:4:4               |
409	   +----------------------+-------------------------+----------------+
410	   | 5k, 5120x2880        |       15, 30, 60        |  AI, RA, FIZD  |
411	   +----------------------+-------------------------+----------------+
412	   | 4k, 3840x2160        |       15, 30, 60        |  AI, RA, FIZD  |
413	   +----------------------+-------------------------+----------------+
414	   | WQXGA, 2560x1600     |       15, 30, 60        |  AI, RA, FIZD  |
415	   +----------------------+-------------------------+----------------+
416	   | WUXGA, 1920x1200     |       15, 30, 60        |  AI, RA, FIZD  |
417	   +----------------------+-------------------------+----------------+
418	   | WSXGA+, 1680x1050    |       15, 30, 60        |  AI, RA, FIZD  |
419	   +----------------------+-------------------------+----------------+
420	   | WXGA, 1280x800       |       15, 30, 60        |  AI, RA, FIZD  |
421	   +----------------------+-------------------------+----------------+
422	   | XGA, 1024x768        |       15, 30, 60        |  AI, RA, FIZD  |
423	   +----------------------+-------------------------+----------------+
424	   | SVGA, 800x600        |       15, 30, 60        |  AI, RA, FIZD  |
425	   +----------------------+-------------------------+----------------+
426	   | VGA, 640x480         |       15, 30, 60        |  AI, RA, FIZD  |
427	   +----------------------+-------------------------+----------------+
428	   |                   Input color format: YCbCr 4:4:4               |
429	   +----------------------+-------------------------+----------------+
430	   | 5k, 5120x2880        |       15, 30, 60        |  AI, RA, FIZD  |
431	   +----------------------+-------------------------+----------------+
432	   | 4k, 3840x2160        |       15, 30, 60        |  AI, RA, FIZD  |
433	   +----------------------+-------------------------+----------------+
434	   | 1440p (2K), 2560x1440|       15, 30, 60        |  AI, RA, FIZD  |
435	   +----------------------+-------------------------+----------------+
436	   | 1080p, 1920x1080     |       15, 30, 60        |  AI, RA, FIZD  |
437	   +----------------------+-------------------------+----------------+
438	   | 720p, 1280x720       |       15, 30, 60        |  AI, RA, FIZD  |
439	   +----------------------+-------------------------+----------------+
440	   Table 5. Screencasting for RGB and YCbCr 4:4:4 format: typical
441	   values of resolutions, frame-rates, and RAPs

443	2.6. Game streaming

445	   This is a service that provides game content over the Internet to
446	   different local devices such as notebooks, gaming tablets, etc. In
447	   this category of applications, server renders 3D games in cloud
448	   server, and streams the game to any device with a wired or wireless
449	   broadband connection [10]. There are low latency requirements for
450	   transmitting user interactions and receiving game data in less than
451	   a turn-around delay of 100 ms. This allows anyone to play (or
452	   resume) full featured games from anywhere in the Internet [10]. An
453	   example of this application is Nvidia Grid [10]. Another category
454	   application is broadcast of video games played by people over the
455	   Internet in real time or for later viewing [10]. There are many
456	   companies such as Twitch, YY in China enable game broadcasting [10].
457	   Games typically contain a lot of sharp edges and large motion [10].
458	   The main requirements are as follows:

460	   o  Random access to pictures for game broadcasting;

462	   o  Temporal (frame-rate) scalability;
463	   o  Error robustness.

465	   Support of resolution and quality (SNR) scalability is highly
466	   desirable. For this application, typical values of resolutions,
467	   frame-rates, and RAPs are similar to ones presented in Table 5.

469	2.7. Video monitoring / surveillance

471	   This is a type of live broadcasting over IP-based networks. Video
472	   streams are sent to many receivers at the same time. A new receiver
473	   may connect to the stream at an arbitrary moment, so random access
474	   period should be kept small enough (approximately, ~1-5 seconds).
475	   Data are transmitted publicly in the case of video monitoring and
476	   privately in the case of video surveillance, respectively. For IP-
477	   cameras that have to capture, process and encode video data,
478	   complexity including computational and hardware complexity as well
479	   as memory bandwidth should be kept low to allow real-time
480	   processing. In addition, support of high dynamic range and a
481	   monochrome mode (e.g., for infrared cameras) as well as resolution
482	   and quality (SNR) scalability is an essential requirement for video
483	   surveillance. In some use-cases, high video signal fidelity is
484	   required even after lossy compression. Typical values of
485	   resolutions, frame-rates, and RAPs for video monitoring /
486	   surveillance applications are presented in Table 6.

488	   +----------------------+-------------------------+-----------------+
489	   |      Resolution      |     Frame-rate, fps     |       PAM       |
490	   +----------------------+-------------------------+-----------------+
491	   +----------------------+-------------------------+-----------------+
492	   | 2160p (4K),3840x2160 |       12, 25, 30        |    RA, FIZD     |
493	   +----------------------+-------------------------+-----------------+
494	   | 5Mpixels, 2560x1920  |       12, 25, 30        |    RA, FIZD     |
495	   +----------------------+-------------------------+-----------------+
496	   | 1080p, 1920x1080     |         25, 30          |    RA, FIZD     |
497	   +----------------------+-------------------------+-----------------+
498	   | 1.3Mpixels, 1280x960 |         25, 30          |    RA, FIZD     |
499	   +----------------------+-------------------------+-----------------+
500	   | 720p, 1280x720       |         25, 30          |    RA, FIZD     |
501	   +----------------------+-------------------------+-----------------+
502	   | SVGA, 800x600        |         25, 30          |    RA, FIZD     |
503	   +----------------------+-------------------------+-----------------+
504	   Table 6. Video monitoring / surveillance: typical values of
505	   resolutions, frame-rates, and RAPs

507	3. Requirements

509	   Taking the requirements discussed above for specific video
510	   applications, this chapter proposes requirements for an internet
511	   video codec.

513	3.1. General requirements

515	   3.1.1. The most basic requirement is coding efficiency, i.e.
516	   compression performance on both "easy" and "difficult" content for
517	   applications and use cases in Section 2. The codec should provide
518	   higher coding efficiency over state-of-the-art video codecs such as
519	   HEVC/H.265 and VP9, at least by 25% in accordance with the
520	   methodology described in Section 4.1 of this document. For higher
521	   resolutions, the coding efficiency improvements are expected to be
522	   higher than for lower resolutions.

524	   3.1.2. Good quality specification and well-defined profiles and
525	   levels are required to enable device interoperability and facilitate
526	   decoder implementations. A profile consists of a subset of entire
527	   bitstream syntax elements and consequently it also defines the
528	   necessary tools for decoding a conforming bitstream of that profile.
529	   A level imposes a set of numerical limits to the values of some
530	   syntax elements. An example of codec levels to be supported is
531	   presented in Table 7. An actual level definition should include
532	   constraints on features that impact the decoder complexity. For
533	   example, these features might be as follows: maximum bit-rate, line
534	   buffer size, memory usage, etc.

536	   +------------------------------------------------------------------+
537	   |    Level    |  Example picture resolution at highest frame rate  |
538	   +-------------+----------------------------------------------------+
539	   |             |           128x96(12,288*)@30.0                     |
540	   |      1      |           176x144(25,344*)@15.0                    |
541	   +-------------+----------------------------------------------------+
542	   |      2      |           352x288(101,376*)@30.0                   |
543	   +-------------+----------------------------------------------------+
544	   |             |           352x288(101,376*)@60.0                   |
545	   |      3      |           640x360(230,400*)@30.0                   |
546	   +-------------+----------------------------------------------------+
547	   |             |           640x360(230,400*)@60.0                   |
548	   |      4      |           960x540(518,400*)@30.0                   |
549	   +-------------+----------------------------------------------------+
550	   |             |           720x576(414,720*)@75.0                   |
551	   |      5      |           960x540(518,400*)@60.0                   |
552	   |             |           1280x720(921,600*)@30.0                  |
553	   +-------------+----------------------------------------------------+
554	   |             |           1,280x720(921,600*)@68.0                 |
555	   |      6      |           2,048x1,080(2,211,840*)@30.0             |
556	   +-------------+----------------------------------------------------+
557	   |             |           1,280x720(921,600*)@120.0                |
558	   |      7      |           2,048x1,080(2,211,840*)@60.0             |
559	   +-------------+----------------------------------------------------+
560	   |             |           1,920x1,080(2,073,600*)@120.0            |
561	   |      8      |           3,840x2,160(8,294,400*)@30.0             |
562	   |             |           4,096x2,160(8,847,360*)@30.0             |
563	   +-------------+----------------------------------------------------+
564	   |             |           1,920x1,080(2,073,600*)@250.0            |
565	   |      9      |           4,096x2,160(8,847,360*)@60.0             |
566	   +-------------+----------------------------------------------------+
567	   |             |           1,920x1,080(2,073,600*)@300.0            |
568	   |     10      |           4,096x2,160(8,847,360*)@120.0            |
569	   +-------------+----------------------------------------------------+
570	   |             |           3,840x2,160(8,294,400*)@120.0            |
571	   |     11      |           8,192x4,320(35,389,440*)@30.0            |
572	   +-------------+----------------------------------------------------+
573	   |             |           3,840x2,160(8,294,400*)@250.0            |
574	   |     12      |           8,192x4,320(35,389,440*)@60.0            |
575	   +-------------+----------------------------------------------------+
576	   |             |           3,840x2,160(8,294,400*)@300.0            |
577	   |     13      |           8,192x4,320(35,389,440*)@120.0           |
578	   +-------------+----------------------------------------------------+
579	   Table 7. Codec levels

581	   NB *: The quantities of pixels are presented for such applications
582	   where a picture can have an arbitrary size (e.g., screencasting)

584	   3.1.3. Bitstream syntax should allow extensibility and backward
585	   compatibility. New features can be supported easily by using
586	   metadata (e.g., such as SEI messages, VUI, headers) without
587	   affecting the bitstream compatibility with legacy decoders. A newer
588	   version of the decoder shall be able to play bitstreams of an older
589	   version of the same or lower profile and level.

591	   3.1.4. A bitstream should have a model that allows easy parsing and
592	   identification of the sample components (such as ISO/IEC14496-10,
593	   Annex B or ISO/IEC 14496-15). In particular, information needed for
594	   packet handling (e.g., frame type) should not require parsing
595	   anything below the header level.

597	   3.1.5. Perceptual quality tools (such as adaptive QP and
598	   quantization matrices) should be supported by the codec bit-stream.

600	   3.1.6. The codec specification shall define a buffer model such as
601	   hypothetical reference decoder (HRD).

603	   3.1.7. Specifications providing integration with system and delivery
604	   layers should be developed.

606	3.2. Basic requirements

608	   3.2.1. Input source formats:

610	   o  Bit depth: 8- and 10-bits (up to 12-bits for a high profile) per
611	      color component;

613	   o  Color sampling formats:

615	       . YCbCr 4:2:0;

617	       . YCbCr 4:4:4, YCbCr 4:2:2 and YCbCr 4:0:0 (preferably in
618	          different profile(s)).

620	   o  For profiles with bit depth of 10 bits per sample or higher,
621	      support of high dynamic range and wide color gamut.

623	   o  Support of arbitrary resolution according to the level
624	      constraints for such applications where a picture can have an
625	      arbitrary size (e.g., in screencasting).

627	   3.2.2. Coding delay:

629	   o  Support of configurations with zero structural delay also
630	      referred to as "low-delay" configurations.

632	       . Note 1: end-to-end delay should be up to 320 ms [2] but its
633	          preferable value should be less than 100 ms [7]

635	   o  Support of efficient random access point encoding (such as intra
636	      coding and resetting of context variables) as well as efficient
637	      switching between multiple quality representations.

639	   o  Support of configurations with non-zero structural delay (such as
640	      out-of-order or multi-pass encoding) for applications without
641	      low-delay requirements if such configurations provide additional
642	      compression efficiency improvements.

644	   3.2.3. Complexity:

646	   o  Feasible real-time implementation of both an encoder and a
647	      decoder supporting a chosen subset of tools for hardware and
648	      software implementation on a wide range of state-of-the-art
649	      platforms. The real-time encoder tools subset should provide
650	      meaningful improvement in compression efficiency at reasonable
651	      complexity of hardware and software encoder implementations as
652	      compared to current real-time implementations of state-of-the-art
653	      video compression technologies such as HEVC/H.265 and VP9.

655	   o  High-complexity software encoder implementations used by offline
656	      encoding applications can have 10x or more complexity increase
657	      compared to state-of-the-art video compression technologies such
658	      as HEVC/H.265 and VP9.

660	   3.2.4. Scalability:

662	   o  Temporal (frame-rate) scalability should be supported.

664	   3.2.5. Error resilience:

666	   o  Error resilience tools that are complementary to the error
667	      protection mechanisms implemented on transport level should be
668	      supported.

670	   o  The codec should support mechanisms that facilitate packetization
671	      of a bitstream for common network protocols.

673	   o  Packetization mechanisms should enable frame-level error recovery
674	      by means of retransmission or error concealment.

676	   o  The codec should support effective mechanisms for allowing
677	      decoding and reconstruction of significant parts of pictures in
678	      the event that parts of the picture data are lost in
679	      transmission.

681	   o  The bitstream specification shall support independently decodable
682	      sub-frame units similar to slices or independent tiles. It shall
683	      be possible for the encoder to restrict the bit-stream to allow
684	      parsing of the bit-stream after a packet-loss and to communicate
685	      it to the decoder.

687	3.3. Optional requirements

689	   3.3.1. Input source formats

691	   o  Bit depth: up to 16-bits per color component.

693	   o  Color sampling formats: RGB 4:4:4.

695	   o  Auxiliary channel (e.g., alpha channel) support.

697	   3.3.2. Scalability:

699	   o  Resolution and quality (SNR) scalability that provide low
700	      compression efficiency penalty (up to 5% of BD-rate [12] increase
701	      per layer with reasonable increase of both computational and
702	      hardware complexity) can be supported in the main profile of the
703	      codec being developed by the NETVC WG. Otherwise, a separate
704	      profile is needed to support these types of scalability.

706	   o  Computational complexity scalability(i.e. computational
707	      complexity is decreasing along with degrading picture quality) is
708	      desirable.

710	   3.3.3. Complexity:

712	   Tools that enable parallel processing (e.g., slices, tiles, wave
713	   front propagation processing) at both encoder and decoder sides are
714	   highly desirable for many applications.

716	   o  High-level multi-core parallelism: encoder and decoder operation,
717	      especially entropy encoding and decoding, should allow multiple
718	      frames or sub-frame regions (e.g. 1D slices, 2D tiles, or
719	      partitions) to be processed concurrently, either independently or
720	      with deterministic dependencies that can be efficiently pipelined

722	   o  Low-level instruction set parallelism: favor algorithms that are
723	      SIMD/GPU friendly over inherently serial algorithms

725	   3.3.4. Coding efficiency

727	   Compression efficiency on noisy content, content with film grain,
728	   computer generated content, and low resolution materials is
729	   desirable.

731	4. Evaluation methodology

733	4.1. Compression performance evaluation

735	   As shown in Fig.1, compression performance testing is performed in 3
736	   overlapped ranges that encompass 10 different bitrate values:

738	   o  Low bitrate range (LBR) is the range that contains the 4 lowest
739	      bitrates of the 10 specified bitrates (1 of the 4 bitrate values
740	      is shared with the neighboring range);

742	   o  Medium bitrate range (MBR) is the range that contains the 4
743	      medium bitrates of the 10 specified bitrates (2 of the 4 bitrate
744	      values are shared with the neighboring ranges);

746	   o  High bitrate range (HBR) is the range that contains the 4 highest
747	      bitrates of the 10 specified bitrates (1 of the 4 bitrate values
748	      is shared with the neighboring range).

750	   Initially, for the codec selected as a reference one (e.g., HEVC or
751	   VP9), a set of 10 QP (quantization parameter) values should be
752	   specified (in a separate document on Internet video codec testing)
753	   and corresponding quality values should be calculated. In Fig.1, QP
754	   and quality values are denoted as QP0, QP1, QP2,..., QP8, QP9 and
755	   Q0, Q1, Q2,..., Q8, Q9, respectively. To guarantee the overlaps of
756	   quality levels between the bitrate ranges of the reference and
757	   tested codecs, a quality alignment procedure should be performed for
758	   each range's outermost (left- and rightmost) quality levels Qk of
759	   the reference codec (i.e. for Q0, Q3, Q6, and Q9) and the quality
760	   levels Q'k (i.e. Q'0, Q'3, Q'6, and Q'9) of the tested codec. Thus,
761	   these quality levels Q'k and, hence, the corresponding QP value QP'k
762	   (i.e. QP'0, QP'3, QP'6, and QP'9) of the tested codec should be
763	   selected using the following formulas:

765	   Q'k =   min { abs(Q'i - Qk) },
766	         i in R

768	   QP'k = argmin { abs(Q'i(QP'i) - Qk(QPk)) },
769	          i in R

771	   where R is the range of the QP indexes of the tested codec, i.e. the
772	   candidate Internet video codec. The inner quality levels (i.e. Q'1,
773	   Q'2, Q'4, Q'5, Q'7, and Q'8) as well as their corresponding QP
774	   values of each range (i.e. QP'1, QP'2, QP'4, QP'5, QP'7, and QP'8)
775	   should be as equidistantly spaced as possible between the left- and
776	   rightmost quality levels without explicitly mapping their values
777	   using the above described procedure.

779	   QP'9 QP'8  QP'7 QP'6 QP'5 QP'4 QP'3 QP'2 QP'1 QP'0 <+-----
780	    ^     ^    ^    ^    ^    ^    ^    ^    ^    ^    | Tested
781	    |     |    |    |    |    |    |    |    |    |    | codec
782	   Q'0   Q'1  Q'2  Q'3  Q'4  Q'5  Q'6  Q'7  Q'8  Q'9  <+-----
783	    ^               ^              ^              ^
784	    |               |              |              |
785	   Q0    Q1    Q2   Q3   Q4   Q5   Q6   Q7   Q8   Q9  <+-----
786	    ^    ^     ^    ^    ^    ^    ^    ^    ^    ^    | Reference
787	    |    |     |    |    |    |    |    |    |    |    | codec
788	   QP9  QP8   QP7  QP6  QP5  QP4  QP3  QP2  QP1  QP0  <+-----
789	   +----------------+--------------+--------------+--------->
790	   ^                ^              ^              ^     Bit-rate
791	   |-------LBR------|              |-----HBR------|
792	                    ^              ^
793	                    |------MBR-----|

795	   Figure 1 Quality/QP alignment for compression performance evaluation

797	   Since the QP mapping results may vary for different sequences,
798	   eventually, this quality alignment procedure needs to be separately
799	   performed for each quality assessment index and each sequence used
800	   for codec performance evaluation to fulfill the above described
801	   requirements.

803	   To assess the quality of output (decoded) sequences, two indexes,
804	   PSNR [3] and MS-SSIM [3,11] are separately computed. In the case of
805	   the YCbCr color format, PSNR should be calculated for each color
806	   plane whereas MS-SSIM is calculated for luma channel only. In the
807	   case of the RGB color format, both metrics are computed for R, G and
808	   B channels. Thus, for each sequence, 30 RD-points for PSNR (i.e.
809	   three RD-curves, one for each channel) and 10 RD-points for MS-SSIM
810	   (i.e. one RD-curve, for luma channel only) should be calculated in
811	   the case of YCbCr. If content is encoded as RGB, 60 RD-points (30
812	   for PSNR and 30 for MS-SSIM) should be calculated, i.e. three RD-
813	   curves (one for each channel) are computed for PSNR as well as three
814	   RD-curves (one for each channel) for MS-SSIM.

816	   Finally, to obtain an integral estimation, BD-rate savings [12]
817	   should be computed for each range and each quality index. In
818	   addition, average values over all the 3 ranges should be provided
819	   for both PSNR and MS-SSIM. A list of video sequences that should be
820	   used for testing as well as the 10 QP values for the reference codec
821	   are defined in a separate document. Testing processes should use the
822	   information on the codec applications presented in this document. As
823	   the reference for evaluation, state-of-the-art video codecs such as
824	   HEVC/H.265 [4,5] or VP9 must be used. The reference source code of
825	   the HEVC/H.265 codec can be found at [6]. The HEVC/H.265 codec must
826	   be configured according to [13] and Table 8.

828	   +----------------------+-------------------------------------------+
829	   | Intra-period, second | HEVC/H.265 encoding mode according to [13]|
830	   +----------------------+-------------------------------------------+
831	   |          AI          |        Intra Main or Intra Main10         |
832	   +----------------------+-------------------------------------------+
833	   |          RA          |           Random access Main or           |
834	   |                      |           Random access Main10            |
835	   +----------------------+-------------------------------------------+
836	   |         FIZD         |             Low delay Main or             |
837	   |                      |             Low delay Main10              |
838	   +----------------------+-------------------------------------------+

840	   Table 8. Intra-periods for different HEVC/H.265 encoding modes
841	   according to [13]

843	   According to the coding efficiency requirement described in Section
844	   3.1.1, BD-rate savings calculated for each color plane and averaged
845	   for all the video sequences used to test the NETVC codec should be,
846	   at least,

848	   o  25% if calculated over the whole bitrate range;

850	   o  15% if calculated for each bitrate subrange (LBR, MBR, HBR).

852	   Since values of the two objective metrics (PSNR and MS-SSIM) are
853	   available for some color planes, each value should meet these coding
854	   efficiency requirements, i.e. the final BD-rate saving denoted as S
855	   is calculated for a given color plane as follows:

857	   S = min { S_psnr, S_ms-ssim },

859	   where S_psnr and S_ms-ssim are BD-rate savings calculated for the
860	   given color plane using PSNR and MS-SSIM metrics, respectively.

862	   In addition to the objective quality measures defined above,
863	   subjective evaluation must also be performed for the final NETVC
864	   codec adoption. For subjective tests, the MOS-based evaluation
865	   procedure must be used as described in section 2.1 of [3]. For
866	   perception-oriented tools that primarily impact subjective quality,
867	   additional tests may also be individually assigned even for
868	   intermediate evaluation, subject to a decision of the NETVC WG.

870	4.2. Reference software

872	   Reference software provided to the NETVC WG for candidate codecs
873	   should comprise a fully operational encoder supporting necessary
874	   rate controls, subjective quality optimization features and some
875	   degree of speed optimization and a "real-time" decoder.

877	5. Security Considerations

879	   This document itself does not address any security considerations.
880	   However, it is worth noting that a codec implementation (for both an
881	   encoder and a decoder) should cover the worst case of computational
882	   complexity, memory bandwidth, and physical memory size (e.g., for
883	   decoded pictures used as references). Otherwise, it can be
884	   considered as a security vulnerability and lead to denial-of-service
885	   (DoS) in the case of attacks.

887	6. Conclusions

889	   In this document, an overview of Internet video codec applications
890	   and typical use cases as well as a prioritized list of requirements
891	   for an Internet video codec are presented. An evaluation methodology
892	   for this codec is also proposed.

894	7. IANA Considerations

896	   This document has no IANA actions.

898	8. References

900	8.1. Normative References

902	   [1]   Recommendation ITU-R BT.2020-2: Parameter values for ultra-
903	         high definition television systems for production and
904	         international programme exchange, 2015.

906	   [2]   Recommendation ITU-T G.1091: Quality of Experience
907	         requirements for telepresence services, 2014.

909	   [3]   ISO/IEC PDTR 29170-1: Information technology -- Advanced image
910	         coding and evaluation methodologies -- Part 1: Guidelines for
911	         codec evaluation.

913	   [4]   ISO/IEC 23008-2:2015. Information technology -- High
914	         efficiency coding and media delivery in heterogeneous
915	         environments -- Part 2: High efficiency video coding

917	   [5]   Recommendation ITU-T H.265: High efficiency video coding,
918	         2013.

920	   [6]   https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/

922	8.2. Informative References

924	   [7]   S. Wenger, "The case for scalability support in version 1 of
925	         Future Video Coding," contribution COM 16-C 988 R1-E to ITU-T
926	         SG16/Q6, September 2015."Recommended upload encoding settings
927	         (Advanced)"

929	   [8]   "Recommended upload encoding settings (Advanced)"
930	         https://support.google.com/youtube/answer/1722171?hl=en

932	   [9]   H. Yu, K. McCann, R. Cohen, and P. Amon, "Requirements for
933	         future extensions of HEVC in coding screen content", ISO/IEC
934	         JTC1/SC29/WG11 MPEG2013/N14174, San Jose, USA, Jan. 2014

936	   [10]  Manindra Parhy, "Game streaming requirement for Future Video
937	         Coding," MPEG Contribution m36771, June 2015, Warsaw, Poland.

939	   [11]  Z. Wang, E. P. Simoncelli, and A. C. Bovik, "Multi-scale
940	         structural similarity for image quality assessment," Invited
941	         Paper, IEEE Asilomar Conference on Signals, Systems and
942	         Computers, Nov. 2003, Vol. 2, pp. 1398-1402.

944	   [12]  G. Bjontegaard, "Calculation of average PSNR differences
945	         between RD-curves (VCEG-M33)," in VCEG Meeting (ITU-T SG16
946	         Q.6), Austin, Texas, USA, Apr. 2-4 2001.

948	   [13]  F. Bossen, "Common test conditions and software reference
949	         configurations," JCTVC-L1100, Geneva, Switzerland, Jan. 2013.

951	   [14]  http://www.digitizationguidelines.gov/term.php?term=compressio
952	         nvisuallylossless)

954	9. Acknowledgments

956	10. The authors would like to thank Mr. Jiantong Zhou, Mr. Paul
957	   Coverdale, Mr. Vasily Rufitskiy, and Dr. Haitao Yang for many useful
958	   discussions on this document and their help while preparing it as
959	   well as Mr. Mo Zanaty, Dr. Minhua Zhou, Dr. Ali Begen, Mr. Thomas
960	   Daede, Dr. Thomas Davies, Mr. Jonathan Lennox, Dr. Timothy
961	   Terriberry, Mr. Peter Thatcher, Dr. Jean-Marc Valin, Mr. Jack
962	   Moffitt, Mr. Greg Coppa and Mr. Andrew Krupiczka for their valuable
963	   comments on different revisions of this document.

965	   This document was prepared using 2-Word-v2.0.template.dot.

967	Appendix A.     Abbreviations used in the text of this document

969	   +--------------+---------------------------------------------------+
970	   | Abbreviation |                      Meaning                      |
971	   +--------------+---------------------------------------------------+
972	   |      AI      | All-Intra (each picture is intra-coded)           |
973	   |   BD-Rate    | Bjontegaard Delta Rate                            |
974	   |     FIZD     | just the First picture is Intra-coded, Zero       |
975	   |              | structural Delay                                  |
976	   |     GOP      | Group of Picture                                  |
977	   |     HBR      | High Bitrate Range                                |
978	   |     HDR      | High Dynamic Range                                |
979	   |     HRD      | Hypothetical Reference Decoder                    |
980	   |     IPTV     | Internet Protocol Television                      |
981	   |     LBR      | Low Bitrate Range                                 |
982	   |     MBR      | Medium Bitrate Range                              |
983	   |     MOS      | Mean Opinion Score                                |
984	   |   MS-SSIM    | Multi-Scale Structural Similarity quality index   |
985	   |     PAM      | Picture Access Mode                               |
986	   |     PSNR     | Peak Signal-to-Noise Ratio                        |
987	   |     QoS      | Quality of Service                                |
988	   |     QP       | Quantization Parameter                            |
989	   |     RA       | Random Access                                     |
990	   |     RAP      | Random Access Period                              |
991	   |     RD       | Rate-Distortion                                   |
992	   |     SEI      | Supplemental Enhancement Information              |
993	   |     UGC      | User-Generated Content                            |
994	   |     VDI      | Virtual Desktop Infrastructure                    |
995	   |     VUI      | Video Usability Information                       |
996	   |     WCG      | Wide Color Gamut                                  |
997	   +--------------+---------------------------------------------------+

999	Appendix B.                 Used terms

1001	   +------------------+-----------------------------------------------+
1002	   |      Term        |                   Meaning                     |
1003	   +------------------+-----------------------------------------------+
1004	   | High dynamic     | is a set of techniques that allow a greater   |
1005	   | range imaging    | dynamic range of exposures or values (i.e.,   |
1006	   |                  | a wide range of values between light and dark |
1007	   |                  | areas) than normal digital imaging techniques.|
1008	   |                  | The intention is to accurately represent the  |
1009	   |                  | wide range of intensity levels found in such  |
1010	   |                  | examples as exterior scenes that include      |
1011	   |                  | light-colored items struck by direct sunlight |
1012	   |                  | and areas of deep shadow [14].                |
1013	   |                  |                                               |
1014	   | Random access    | is the period of time between two closest     |
1015	   | period           | independently decodable frames (pictures).    |
1016	   |                  |                                               |
1017	   | RD-point         | A point in a 2 dimensional rate-distortion    |
1018	   |                  | space where the values of bitrate and quality |
1019	   |                  | metric are used as x- and y-coordinates,      |
1020	   |                  | respectively                                  |
1021	   |                  |                                               |
1022	   | Visually         | is a form or manner of lossy compression      |
1023	   | lossless         | where the data that are lost after the file   |
1024	   | compression      | is compressed and decompressed is not         |
1025	   |                  | detectable to the eye; the compressed data    |
1026	   |                  | appearing identical to the uncompressed       |
1027	   |                  | data [14].                                    |
1028	   |                  |                                               |
1029	   | Wide color gamut | is a certain complete color subset (e.g.,     |
1030	   |                  | considered in ITU-R BT.2020) that supports a  |
1031	   |                  | wider range of colors (i.e., an extended range|
1032	   |                  | of colors that can be generated by a specific |
1033	   |                  | input or output device such as a video camera,|
1034	   |                  | monitor or printer and can be interpreted by  |
1035	   |                  | a color model) than conventional color gamuts |
1036	   |                  | (e.g., considered in ITU-R BT.601 or BT.709). |
1037	   +------------------+-----------------------------------------------+

1039	Authors' Addresses

1041	   Alexey Filippov
1042	   Huawei Technologies

1044	   Email: alexey.filippov@huawei.com

1046	   Andrey Norkin
1047	   Netflix

1049	   Email: anorkin@netflix.com

1051	   Jose Roberto Alvarez
1052	   Huawei Technologies

1054	   Email: jose.roberto.alvarez@huawei.com