idnits 2.17.1 draft-samuelsson-netvc-xvc-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 2, 2018) is 2115 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Outdated reference: A later version (-09) exists of draft-ietf-netvc-testing-06 Summary: 0 errors (**), 0 flaws (~~), 2 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group J. Samuelsson 3 Internet-Draft P. Hermansson 4 Intended status: Standards Track Divideon 5 Expires: January 3, 2019 July 2, 2018 7 The xvc video codec 8 draft-samuelsson-netvc-xvc-01 10 Abstract 12 This document presents a high-level technical description of the xvc 13 video codec. The xvc video codec is a novel video codec that was 14 released in its first version in September 2017. This document 15 provides some technical information as well as a discussion section 16 around how the xvc codec meets the objectives of the NETVC Working 17 Group. The document also includes results comparing the second 18 version of xvc with AV1 and HM for three different test cases; All- 19 Intra, single pass Random-Access, and multi-pass Random-Access. 20 Results are also presented for the full xvc codec relative to the 21 royalty-free baseline profile of xvc. 23 Status of This Memo 25 This Internet-Draft is submitted in full conformance with the 26 provisions of BCP 78 and BCP 79. 28 Internet-Drafts are working documents of the Internet Engineering 29 Task Force (IETF). Note that other groups may also distribute 30 working documents as Internet-Drafts. The list of current Internet- 31 Drafts is at https://datatracker.ietf.org/drafts/current/. 33 Internet-Drafts are draft documents valid for a maximum of six months 34 and may be updated, replaced, or obsoleted by other documents at any 35 time. It is inappropriate to use Internet-Drafts as reference 36 material or to cite them other than as "work in progress." 38 This Internet-Draft will expire on January 3, 2019. 40 Copyright Notice 42 Copyright (c) 2018 IETF Trust and the persons identified as the 43 document authors. All rights reserved. 45 This document is subject to BCP 78 and the IETF Trust's Legal 46 Provisions Relating to IETF Documents 47 (https://trustee.ietf.org/license-info) in effect on the date of 48 publication of this document. Please review these documents 49 carefully, as they describe your rights and restrictions with respect 50 to this document. Code Components extracted from this document must 51 include Simplified BSD License text as described in Section 4.e of 52 the Trust Legal Provisions and are provided without warranty as 53 described in the Simplified BSD License. 55 Table of Contents 57 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 58 2. Technical overview of the xvc video codec . . . . . . . . . . 3 59 2.1. Block structure . . . . . . . . . . . . . . . . . . . . . 4 60 2.2. Intra prediction . . . . . . . . . . . . . . . . . . . . 4 61 2.3. Inter prediction . . . . . . . . . . . . . . . . . . . . 5 62 2.4. Transforms . . . . . . . . . . . . . . . . . . . . . . . 5 63 2.5. Quantization . . . . . . . . . . . . . . . . . . . . . . 5 64 2.6. Entropy coding . . . . . . . . . . . . . . . . . . . . . 6 65 2.7. In-loop filtering . . . . . . . . . . . . . . . . . . . . 6 66 2.8. Restriction flags . . . . . . . . . . . . . . . . . . . . 6 67 2.9. Version handling . . . . . . . . . . . . . . . . . . . . 10 68 2.10. Temporal scalability and parallel decoding . . . . . . . 11 69 2.11. Parallel encoding . . . . . . . . . . . . . . . . . . . . 12 70 2.12. Baseline profile . . . . . . . . . . . . . . . . . . . . 12 71 3. The xvc reference software . . . . . . . . . . . . . . . . . 12 72 4. Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 73 4.1. All-Intra . . . . . . . . . . . . . . . . . . . . . . . . 14 74 4.2. Single pass Random-Access . . . . . . . . . . . . . . . . 14 75 4.3. Multi-pass Random-Access . . . . . . . . . . . . . . . . 15 76 4.4. Full xvc relative to baseline xvc for single pass Random- 77 Access . . . . . . . . . . . . . . . . . . . . . . . . . 15 78 4.5. Full xvc relative to baseline xvc for multi-pass Random- 79 Access . . . . . . . . . . . . . . . . . . . . . . . . . 16 80 4.6. Computational complexity . . . . . . . . . . . . . . . . 16 81 5. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 16 82 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 17 83 7. Security Considerations . . . . . . . . . . . . . . . . . . . 18 84 8. Informative References . . . . . . . . . . . . . . . . . . . 18 85 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 19 87 1. Introduction 89 In September 2017, the new video compression format xvc was released, 90 with source code publicly available at xvc.io [xvcio]. The xvc codec 91 is a software-defined video codec with conformance for bitstreams, 92 decoders, and encoders, defined with reference to the publicly 93 available reference software. The first version of xvc has been 94 developed by Divideon but the source code of the reference software 95 has been made publicly available with a desire to invite to broad 96 usage and continuous development. The developers of the xvc codec 97 strongly believe in open, transparent and collaborative processes for 98 technical development, and the purpose of this contribution is to 99 provide information about the xvc project and to present xvc as a 100 candidate for the Internet Video Codec. 102 The xvc codec was developed with a desire to make efficient 103 compression methods available to a global market under as open, 104 transparent and fair conditions as possible. From the start, the 105 ambition has been to construct a codec, which is easy to deploy, 106 taking into account both technical and non-technical considerations. 107 A deeper analysis of such considerations can be found in the xvc 108 introduction white paper [xvcWp], which provides additional 109 background information related to xvc. In summary, it can be said 110 that one of the most central properties of the xvc codec is that it 111 is not a frozen codec and there is no desire to make it a frozen 112 codec. Instead, conformance is defined with respect to the current 113 official version of the xvc reference software. New versions of the 114 software may be (and is intended to be) released, with addition and/ 115 or removal of technologies as deemed feasible. This makes it 116 possible to let the codec evolve over time and ensure that the xvc 117 codec only make use of functionality that is available for licensing 118 under the xvc license [xvcLicense]. 120 Version 2.0 of xvc was released in July, 2018. The new version 121 provides improved compression efficiency compared to xvc 1.0 and 122 includes a royalty-free baseline profile. The software for xvc 2.0 123 is available under dual-licensing; an LGPL license and a commercial 124 license. 126 2. Technical overview of the xvc video codec 128 The xvc codec has been developed completely from scratch and is built 129 primarily using technology that has been investigated in MPEG and 130 VCEG, for example during the HEVC [HEVC] standardization and in 131 exploratory activities after the finalization of HEVC. One of the 132 most fundamental technical properties of the xvc codec is that the 133 different tools (or processing steps) of the codec, have been 134 isolated within separate conditional clauses, controlled by 135 information (restriction flags) included in the bitstream. This 136 makes the xvc codec extremely flexible and it makes it possible to 137 turn off individual processing steps during run-time, if deemed 138 necessary. This design principle is inspired by the proposal in 139 JCTVC-X0034 [Wenger16]. 141 As a summary of the technology in xvc, it can be said that xvc is a 142 block-based hybrid (inter/intra) codec that operates on raw YUV 143 pictures and compresses them to a NAL (network abstraction layer) 144 unit structured bitstream. Each picture in a video sequence is 145 divided into rectangular blocks of samples, which are predicted from 146 samples in the same picture (intra prediction) or samples in 147 previously coded pictures (inter prediction). Residuals are 148 transformed using non-square transforms and the coded symbols are 149 compressed using a context-adaptive binary arithmetic coder. Block 150 boundaries are filtered using a deblocking filter. The xvc codec 151 supports bit-depths of 8, 10 and 12 bits per sample and chroma 152 formats 4:2:0, 4:2:2, 4:4:4 and monochrome. 154 2.1. Block structure 156 A picture that is to be encoded with xvc is divided into blocks of 157 64x64 pixels, called CTUs. A CTU can either be coded in its full 158 size, or it can be split using a binary split, resulting in two 159 coding units or a quad-tree split, resulting in four coding units. 160 These coding units can be further split into smaller blocks of 161 samples all the way down to coding units of size 4x4 (2x2 for 162 chroma). The splits of a CTU gives rise to a coding unit tree where 163 prediction and transform is performed on the leaf of the trees. Two 164 important differences compared to HEVC can be noted here: 166 1. In xvc there is no distinction between coding units, prediction 167 units and transform units. 169 2. In xvc, transform blocks can be non-square. 171 In the default configuration of xvc, the intra pictures use one 172 coding unit tree for luma and a different coding unit tree for 173 chroma. For inter pictures, a single coding unit tree is used for 174 all color components. 176 2.2. Intra prediction 178 There are 67 intra prediction modes in xvc, representing DC 179 prediction, planar prediction and 65 different angular directions. 180 Intra prediction uses previously coded samples to the left and/or 181 above the current block. Reference samples are filtered before being 182 used for prediction. For vertical, horizontal and DC prediction 183 there is also a post filter applied to smooth out the edge to the 184 neighboring block before applying the transform. There is a scheme 185 for estimating which modes are most probable to be used for the next 186 block and through using this scheme the code-words for indicating one 187 of these "most probable modes" can be made shorter than the code- 188 words of other modes. Both chroma components of a block are 189 predicted with the same intra prediction mode, but the chroma 190 prediction mode does not have to be the same as the luma prediction 191 mode. There is also a mode in which the chroma samples are predicted 192 from the luma samples using a linear model. 194 2.3. Inter prediction 196 Inter prediction is performed using either uni-prediction (where a 197 single reference picture is used) or bi-prediction (where two 198 reference pictures are used). The xvc codec supports translational 199 motions and affine motions which is applied with different motion 200 vectors for each 4x4 block of a coding unit. The syntax for 201 signaling inter prediction contains shortcuts for indicating that a 202 candidate motion vector is used (merge mode) with a special case for 203 when there is no residual (skip mode). Motion interpolation filters 204 of length up to 8 are used and applied with quarter-pel resolution 205 for luma. In xvc there is a specific inter prediction mode that 206 accounts for local changes between neighboring samples in the current 207 picture and neighboring samples in the reference picture. This mode 208 gives for example good prediction when the light level of a scene 209 changes over time. 211 2.4. Transforms 213 On the encoder side, residual data can be derived from taking the 214 original samples of a block and subtract the predicted samples value 215 of the block. This residual can then be forward transformed to 216 result in a set of coefficients that are quantized and signaled in 217 the bitstream. On the decoder side a corresponding (but inverse) 218 process is applied so that coefficients are inverse-quantized, then 219 inverse-transformed and then added to the predicted sample values. 220 In the first version of xvc, a separable DCT-like transform was used 221 where width and height do not need to be of equal length. The size 222 of the transform is always the same as the size of the coding unit. 223 For transforms of size 64, high frequency components are zeroed out. 224 The second version of xvc includes additional transforms that are 225 evaluated and selected based on Rate-Distortion optimization. 226 Regardless of the size of the coding unit, transform coefficients are 227 always coded in subblocks of size 4x4 to enable efficient signaling 228 of subblocks without coefficients. 230 2.5. Quantization 232 Transform coefficients in xvc are quantized based on a quantization 233 parameter (QP). Different QP can be used for different CTUs, and the 234 second version of xvc includes support for more efficient signaling 235 of which QP is used for different CTUs. The xvc codec also includes 236 a scheme called sign hiding which makes it possible to derive the 237 sign of one coefficient (instead of signaling it) by rounding the 238 values of the coefficients of a subblock to match the hiding 239 criteria. 241 2.6. Entropy coding 243 A Context Adaptive Binary Arithmetic Coder (CABAC) is used in xvc. 244 It resembles the entropy coding method in AVC and HEVC. When a 245 symbol has been coded, the state of the context is updated in order 246 to adjust the probabilities (i.e. the cost) for signaling the same 247 symbol when the same conditions occur again in the same picture (i.e. 248 when the same context is used). 250 2.7. In-loop filtering 252 The xvc codec includes a single in-loop filter, which is a deblocking 253 filter, applied on a 4x4 grid of the picture, but only when certain 254 conditions are met. For intra pictures, an edge is filtered if and 255 only if the samples on the different sides of the edge belongs to 256 different coding units. The filtering operation modifies up to two 257 samples on each side of the edge, depending on the local 258 characteristics of the edge. 260 2.8. Restriction flags 262 An xvc bitstream starts with a segment header which provides 263 information about the properties of the coded video such as width, 264 height, bitdepth, chroma format etc. The segment header also 265 contains flags that indicate for each individual tool of the codec 266 whether the tool is used in this bitstream or not. These flags 267 correspond to the fundamental design principle in the xvc codec which 268 is to isolate each individual feature (coding tool) within 269 conditional clauses. The conditional clauses are evaluated during 270 run time which means that the decoder is capable of executing both 271 options of each conditional clause. This makes it easy to evaluate 272 the impact of a specific tool in terms of compression and in terms of 273 complexity, without having to compile different versions of the 274 software for each evaluated combination of tools. 276 In the xvc software, the restriction flags are used instead of 277 #define clauses for different tools. This ensures that the whole 278 codebase is exercised during compilation and makes it easy to verify 279 that different combinations of tools turned on and off work well 280 together. The only macro-defined constant in the xvc software is 281 XVC_HIGH_BITDEPTH, which controls if internal bitdepths above 8 are 282 supported. The first version of xvc contained 62 restriction flags 283 that applies to different areas of the codec. The second version of 284 xvc includes another 14 restriction flags corresponding to new coding 285 tools. An additional restriction flag has been added in order to 286 make it possible to disable the use of the DST transform without 287 disabling the use of 4x4 intra blocks altogether. 289 The following 25 restriction flags are set to 0 in the default 290 configuration of xvc. The corresponding coding tools are used both 291 in the royalty-free profile and in the full xvc codec. 293 o disable_intra_ref_padding 295 o disable_intra_planar 297 o disable_intra_mpm_prediction 299 o disable_intra_chroma_predictor 301 o disable_inter_tmvp_merge 303 o disable_inter_merge_candidates 305 o disable_inter_merge_mode 307 o disable_inter_chroma_subpel 309 o disable_inter_bipred 311 o disable_transform_residual_greater_than_flags 313 o disable_transform_last_position 315 o disable_transform_cbf 317 o disable_cabac_ctx_update 319 o disable_cabac_split_flag_ctx 321 o disable_cabac_coeff_sig_ctx 323 o disable_cabac_coeff_greater1_ctx 325 o disable_deblock_weak_filter 327 o disable_deblock_chroma_filter 329 o disable_deblock_initial_sample_decision 331 o disable_deblock_depending_on_qp 333 o disable_high_level_default_checksum_method 335 o disable_ext_transform_size_64 336 o disable_ext2_intra_chroma_from_luma 338 o disable_ext2_transform_select 340 o disable_ext2_cabac_alt_residual_ctx 342 The following 51 restriction flags are set to 0 in the default 343 configuration of the full xvc codec, but are required to be set to 1 344 in the royalty-free profile of xvc (i.e. the corresponding tools are 345 excluded from the royalty-free profile). 347 o disable_intra_ref_sample_filter 349 o disable_intra_dc_post_filter 351 o disable_intra_ver_hor_post_filter 353 o disable_inter_mvp 355 o disable_inter_scaling_mvp 357 o disable_inter_tmvp_mvp 359 o disable_inter_tmvp_ref_list_derivation 361 o disable_inter_merge_bipred 363 o disable_inter_skip_mode 365 o disable_inter_mvd_greater_than_flags 367 o disable_transform_adaptive_scan_order 369 o disable_transform_residual_greater2 371 o disable_transform_root_cbf 373 o disable_transform_subblock_csbf 375 o disable_transform_sign_hiding 377 o disable_transform_adaptive_exp_golomb 379 o disable_cabac_skip_flag_ctx 381 o disable_cabac_inter_dir_ctx 383 o disable_cabac_subblock_csbf_ctx 384 o disable_cabac_coeff_greater2_ctx 386 o disable_cabac_coeff_last_pos_ctx 388 o disable_cabac_init_per_pic_type 390 o disable_cabac_init_per_qp 392 o disable_deblock_strong_filter 394 o disable_deblock_boundary_strength_zero 396 o disable_deblock_boundary_strength_one 398 o disable_deblock_weak_sample_decision 400 o disable_deblock_two_samples_weak_filter 402 o disable_ext_implicit_last_ctu 404 o disable_ext_tmvp_full_resolution 406 o disable_ext_tmvp_exclude_intra_from_ref_list 408 o disable_ext_ref_list_l0_trim 410 o disable_ext_implicit_partition_type 412 o disable_ext_cabac_alt_split_flag_ctx 414 o disable_ext_cabac_alt_inter_dir_ctx 416 o disable_ext_cabac_alt_last_pos_ctx 418 o disable_ext_two_cu_trees 420 o disable_ext_intra_unrestricted_predictor 422 o disable_ext_deblock_subblock_size_4 424 o disable_ext2_intra_67_modes 426 o disable_ext2_intra_6_predictors 428 o disable_ext2_inter_adaptive_fullpel_mv 430 o disable_ext2_inter_affine 431 o disable_ext2_inter_affine_merge 433 o disable_ext2_inter_affine_mvp 435 o disable_ext2_inter_bipred_l1_mvd_zero 437 o disable_ext2_inter_high_precision_mv 439 o disable_ext2_inter_local_illumination_comp 441 o disable_ext2_transform_skip 443 o disable_ext2_transform_high_precision 445 o disable_ext2_transform_dst 447 2.9. Version handling 449 The xvc codec has a simple but powerful scheme for version handling. 450 All xvc bitstreams include, in the beginning of each segment header, 451 one syntax element that represents the major version and one syntax 452 element that represents the minor version. 454 The major version is increased when non-backwards compatible changes 455 are introduced, typically when new technology is added to xvc. When 456 a new major version of xvc is released, all existing decoders need to 457 be updated in order to support bitstreams of the new major version. 458 Existing bitstreams do not have to be updated when the major version 459 of xvc is increased, since new technology is activated only for new 460 bitstreams that uses the new major version. 462 The minor version is increased when backwards compatible changes are 463 introduced. In practice, this occurs when technology is being 464 disabled through setting their restriction flag equal to one. When a 465 new minor version of xvc is released, existing decoders do not need 466 to be updated immediately, since they already have support for 467 decoding bitstreams in which the disabled technology is turned off. 468 However, the xvc license requires that they are updated at some point 469 so that they do not include support for the technology that is no 470 longer available in xvc. Existing bitstreams have to be updated when 471 the minor version of xvc is increased since the new version of the 472 reference decoder will reject bitstreams with too low minor version 473 (since they make use of technology that is no longer available in 474 xvc). In practice, it is expected to be extremely rare that the 475 minor version has to be increased, but the framework for how to 476 handle it is in place to ensure that there is always a deployable and 477 licensable version of xvc available, and that no patent holder (or 478 "patent troll") would be able to block the codec from being used 479 altogether. 481 The separation of major version and minor version makes it possible 482 to always allow for a transition period when changes are introduced 483 to the xvc codec. In the case of a major version increase, new 484 bitstreams will only be created once decoders have been updated to 485 support the new version. In the case of a minor version increase, 486 new decoders will not replace old decoders until existing bitstreams 487 have been updated to use the new minor version. By applying this 488 scheme, the codec can evolve over time without causing any 489 interoperability problems. 491 2.10. Temporal scalability and parallel decoding 493 The default coding structure of the xvc codec is a static 494 hierarchical B-picture structure with 15 B-pictures. The length of 495 the hierarchical structure, called SubGOP, is indicated in the 496 Segment Header. A SubGOP length of 16 corresponds to 15 B-pictures 497 in a dyadic hierarchy, forming four temporal layers above temporal 498 layer zero. 500 Pictures in temporal layers above 0, only predict from temporal layer 501 0 or pictures with lower temporal layers within the same SubGOP. 502 This makes it possible to scale of an arbitrary number of temporal 503 layers in any SubGOP without affecting decodability of any pictures 504 outside the same SubGOP. This is a very useful property in a 505 software decoder since it makes it possible to adjust the decoding 506 complexity in response to sudden changes in available processing 507 resources. Instead of freezing the video and build up a delay, the 508 framerate can momentarily be reduced and full framerate can be 509 resumed as soon as enough processing resources are available. 511 A fixed hierarchical coding structure is also very useful in order to 512 enable efficient parallel decoding. The xvc reference decoder 513 includes support for picture level threaded decoding. When decoding 514 bitstreams with SubGOP length 16 a speed-up factor of between 3x and 515 4x is typically achieved on a CPU with 4 active physical cores. 517 Providing support for temporal scalability and picture level parallel 518 decoding generally comes at a very low cost in terms of compression 519 performance. However, on very specific sequences there might be a 520 compression penalty due to the constraint of not predicting from 521 previously coded pictures in the same or higher temporal layers. 523 The picture level parallelism that is currently implemented in xvc 524 can be combined with other tools for parallelism such as tiles, 525 wavefronts or slices if a higher degree of parallelism is desired, 526 but support for these tools have not yet been added to xvc. 528 2.11. Parallel encoding 530 Support for multi-threaded encoding has been added to the xvc 531 software. The implementation uses picture based parallelism and is 532 very similar to the method used in the decoder, where multiple 533 pictures are processed in parallel as soon as they do not depend on 534 each other. This makes it possible to produce exactly the same 535 result for multi-threaded encoding as for single threaded encoding, 536 i.e. there is no loss in compression performance. The multi-threaded 537 encoding scheme increases memory usage linearly with number of 538 threads and due to the prediction structure of reference pictures, it 539 typically takes some time for the utilization to reach its peak 540 performance. As an example, when using 8 threads, typically 128 541 pictures needs to be encoded before peak utilization is reached. 543 2.12. Baseline profile 545 Version 2.0 of xvc includes a royalty-free baseline profile. The 546 baseline profile is a pure subset of the full xvc codec. The 547 baseline profile uses 25 of the 76 coding tools in xvc. Results for 548 the full xvc codec relative to the baseline profile are included 549 below. 551 3. The xvc reference software 553 The implementation of the xvc reference software has been made from 554 scratch using C++11. Some of the advantages of C++11 compared to 555 earlier versions of C++ is improved support for type management, 556 improved pointer handling, availability of lambda expressions, 557 threading and useful new keywords such as auto. The xvc 558 implementation strictly follows the Google C++ Style Guide [cppStyle] 559 which is a well established formatting recommendation for C++, 560 intended to provide readability and consistency across various C++ 561 project. 563 The low-complex design of the decoding process of xvc makes it 564 possible to run the decoder efficiently on a large variety of 565 devices. The xvc reference decoder has been demonstrated to run 566 FullHD decoding in realtime on mobile devices, and a JavaScript xvc 567 decoder, based on the reference xvc decoder, is able to run xvc 568 decoding directly in browsers, as shown on the Divideon demo page 569 [xvcDemo] 570 The total number of lines of code for the second version of the xvc 571 software is around 25,000. This can be compared to the around 572 200,000 lines of code in the AV1 software. 574 4. Results 576 Experimental results for three different test cases are reported; 577 All-Intra, single pass Random-Access and multi-pass Random-Access. 578 The latter two correspond to "High Latency CQP" and "High Latency 579 Unconstrained" from the Video Codec Testing and Quality Measurement 580 I-D [Daede17]. The results have been generated using the 581 AreWeCompressedYet? framework [AWCY]. The full set of results are 582 available at https://awcy.divideon.com [DAWCY]. 584 For xvc [xvcSource], the commit 585 f0b31546a3251aeb43f7928338b12aa349156139, from 2018-02-28 has been 586 used, with command line parameters: -tune 1 -speed-mode 0 -chroma-qp- 587 offset-u -1 -chroma-qp-offset-v -1. The chroma qp offset is used to 588 better match bit distribution between luma and chroma relative to 589 AV1. The quantizer values that have been used for xvc are: 20, 25, 590 30, 35, 40. 592 For AV1 [av1Source], the commit 593 1a7099443894d07fdf3acbb7e12ed04c2d62b9c5, from 2018-02-02 has been 594 used, with build settings: -DCONFIG_EXPERIMENTAL=1 and --tune- 595 content=screen as extra command line argument. The following 596 quantizer values are used: 20, 32, 43, 55 and 63. For multi-pass 597 coding the run "debargha-adopted-0104@2018-01-05T00:55:50.198Z" was 598 copied from AreWeCompressedYet? [AWCY], since that was the most 599 recent run that was finsihed at the time of this submission. That 600 run was using commit 5a31bc899b308da9b2cacd339d0b19374a74b906 from 601 "2018-01-04" 603 For HM [hmSource], version 16.17 from 2017-10-18 has been used with 604 the default configuration files provided in the cfg folder of the 605 source code repository. These configuration files correspond to the 606 JCT-VC common test conditions described in JCTVC-Z1100 [hmCtc]. The 607 following quantizer values are used: 20, 25, 30, 35, 40. 609 For All-Intra, results are presented for the Subset1 test set, and 610 for the two Random-Access cases results are presented for the 611 Objective-1-fast test set [Testset]. 613 The tables below show the average percentual rate difference measured 614 using the Bjontegaard rate difference, also known as BD- 615 rate [Bjontegaard01]. The BD-rate is measured using the following 616 objective metrics: PSNR, PSNR-HVS [Egiazarian2006], SSIM [Wang04] and 617 MSSIM [Wang03]. 619 4.1. All-Intra 621 In the tables below, results are presented for the Subset1 test set 622 which consist of 50 different still images. 624 The following table present results of xvc relative to HM with 625 numbers indicating the percentual bitrate difference for equivalent 626 quality (negative numbers means that xvc reduces the bitrate). 628 +---------+------+---------+---------+----------+------+---------+ 629 | | PSNR | PSNR Cb | PSNR Cr | PSNR HVS | SSIM | MS SSIM | 630 +---------+------+---------+---------+----------+------+---------+ 631 | Average | -9.3 | -33.3 | -30.3 | -9.8 | -9.9 | -9.5 | 632 +---------+------+---------+---------+----------+------+---------+ 634 All-intra xvc relative to HM 636 The following table present results of xvc relative to AV1 with 637 numbers indicating the percentual bitrate difference for equivalent 638 quality (negative numbers means that xvc reduces the bitrate). 640 +---------+------+---------+---------+----------+------+---------+ 641 | | PSNR | PSNR Cb | PSNR Cr | PSNR HVS | SSIM | MS SSIM | 642 +---------+------+---------+---------+----------+------+---------+ 643 | Average | -3.1 | -7.4 | -7.9 | -2.1 | -2.8 | -3.0 | 644 +---------+------+---------+---------+----------+------+---------+ 646 All-intra xvc relative to AV1 648 4.2. Single pass Random-Access 650 The following table present results of xvc relative to HM with 651 numbers indicating the percentual bitrate difference for equivalent 652 quality (negative numbers means that xvc reduces the bitrate). 654 +---------+-------+---------+---------+----------+-------+---------+ 655 | | PSNR | PSNR Cb | PSNR Cr | PSNR HVS | SSIM | MS SSIM | 656 +---------+-------+---------+---------+----------+-------+---------+ 657 | 1080p | -16.8 | -29.9 | -28.9 | -15.0 | -17.9 | -16.8 | 658 | 1080psc | -13.7 | -44.5 | -40.4 | -15.4 | -16.7 | -17.0 | 659 | 720p | -20.8 | -30.0 | -32.6 | -20.1 | -23.8 | -22.7 | 660 | 360p | -26.1 | -24.7 | -28.8 | -26.4 | -30.2 | -29.6 | 661 | Average | -19.5 | -30.7 | -31.3 | -19.1 | -22.0 | -21.2 | 662 +---------+-------+---------+---------+----------+-------+---------+ 664 Single pass Random-Access xvc relative to HM 666 The following table present results of xvc relative to AV1 with 667 numbers indicating the percentual bitrate difference for equivalent 668 quality (negative numbers means that xvc reduces the bitrate). 669 Results are presented for 360p and 720p only, since we were not able 670 to run encodings for the complete test set for AV1 in time. 672 +---------+-------+---------+---------+----------+-------+---------+ 673 | | PSNR | PSNR Cb | PSNR Cr | PSNR HVS | SSIM | MS SSIM | 674 +---------+-------+---------+---------+----------+-------+---------+ 675 | 720p | -13.3 | -0.8 | -4.4 | -16.4 | -20.2 | -20.5 | 676 | 360p | -19.7 | -9.6 | -3.4 | -22.5 | -23.0 | -26.1 | 677 | Average | -16.5 | -5.2 | -3.9 | -19.4 | -21.6 | -23.3 | 678 +---------+-------+---------+---------+----------+-------+---------+ 680 Single pass Random-Access xvc relative to AV1 682 4.3. Multi-pass Random-Access 684 The following table present results of xvc relative to AV1 with 685 numbers indicating the percentual bitrate difference for equivalent 686 quality (negative numbers means that xvc reduces the bitrate). The 687 results for AV1 are imported from AreWeCompressedYet? [AWCY]. 689 +---------+-------+---------+---------+----------+-------+---------+ 690 | | PSNR | PSNR Cb | PSNR Cr | PSNR HVS | SSIM | MS SSIM | 691 +---------+-------+---------+---------+----------+-------+---------+ 692 | 1080p | -6.0 | -4.5 | -3.2 | -5.6 | -10.9 | -9.7 | 693 | 1080psc | 8.3 | 18.5 | 15.8 | 5.9 | 6.2 | 3.9 | 694 | 720p | -0.5 | 0.4 | 4.8 | -1.4 | -6.0 | -5.5 | 695 | 360p | -15.9 | -6.2 | 11.2 | -19.9 | -19.0 | -21.2 | 696 | Average | -5.1 | -0.7 | 4.6 | -6.4 | -9.4 | -9.6 | 697 +---------+-------+---------+---------+----------+-------+---------+ 699 Multi-pass Random-Access xvc relative to AV1 701 4.4. Full xvc relative to baseline xvc for single pass Random-Access 703 The following table present results of the full xvc codec relative to 704 the baseline profile of xvc with numbers indicating the percentual 705 bitrate difference for equivalent quality. 707 +---------+-------+---------+---------+----------+-------+---------+ 708 | | PSNR | PSNR Cb | PSNR Cr | PSNR HVS | SSIM | MS SSIM | 709 +---------+-------+---------+---------+----------+-------+---------+ 710 | 1080p | -11.6 | -14.6 | -14.4 | -10.9 | -11.9 | -11.5 | 711 | 1080psc | -15.1 | -23.8 | -24.4 | -11.5 | -12.7 | -12.0 | 712 | 720p | -10.2 | -14.7 | -15.8 | -9.9 | -11.4 | -10.6 | 713 | 360p | -14.9 | -18.6 | -21.4 | -15.0 | -16.4 | -15.6 | 714 | Average | -12.5 | -16.8 | -17.7 | -11.7 | -12.9 | -12.3 | 715 +---------+-------+---------+---------+----------+-------+---------+ 717 Multi-pass Random-Access full xvc relative to baseline xvc 719 4.5. Full xvc relative to baseline xvc for multi-pass Random-Access 721 The following table present results of the full xvc codec relative to 722 the baseline profile of xvc with numbers indicating the percentual 723 bitrate difference for equivalent quality. 725 +---------+-------+---------+---------+----------+-------+---------+ 726 | | PSNR | PSNR Cb | PSNR Cr | PSNR HVS | SSIM | MS SSIM | 727 +---------+-------+---------+---------+----------+-------+---------+ 728 | 1080p | -11.5 | -14.8 | -14.1 | -10.6 | -11.5 | -11.1 | 729 | 1080psc | -15.8 | -23.7 | -24.7 | -12.6 | -14.1 | -13.1 | 730 | 720p | -10.3 | -14.0 | -15.1 | -10.3 | -12.0 | -11.1 | 731 | 360p | -14.3 | -17.7 | -18.5 | -14.1 | -15.7 | -14.7 | 732 | Average | -12.5 | -16.5 | -16.8 | -11.6 | -12.9 | -12.2 | 733 +---------+-------+---------+---------+----------+-------+---------+ 735 Single pass Random-Access full xvc relative to baseline xvc 737 4.6. Computational complexity 739 The single pass Random-Access configurations have been run on the 740 same platform (Intel Xeon E5-2660). The encoding times and decoding 741 times can be used to estimate the computational complexity when 742 running that particular setting on that particular hardware. On 743 average the encoding time is around 340% higher for AV1 than for xvc. 744 The decoding time is around 70% higher for AV1 than for xvc. 746 5. Discussion 748 The xvc codec has been developed with the aim of creating the most 749 efficient video codec that can be made available under a single 750 reasonable license. Different coding tools have been included based 751 solely on their technical merits. Technology is replaced or removed 752 only if the patent holder of that particular technology has requested 753 for it to be removed, or if the patent holder by other means have 754 made it clear that they intend to seek royalties for the technology 755 outside of the xvc licensing program or initiate patent litigation or 756 a patent lawsuit related to the technology. This approach has made 757 it possible to make use of the best available technology rather than 758 having to compromise, exclude and/or design around technology to 759 create a lesser efficient alternative. 761 Divideon is in continuous dialog with potential patent holders and 762 has previously sent out a Call for Patents in xvc [xvcCall], to make 763 sure that the license actually covers all the necessary technology to 764 use, implement and distribute conforming xvc implementations. As 765 with all modern video codecs it is not possible to give a definite 766 answer to the patent situation, but what is unique with xvc is the 767 well-defined framework for handling third party patent assertions. 769 In the Call for Patents in xvc that was sent out by Divideon there 770 was also a request for feedback regarding the idea of defining a 771 royalty-free profile of xvc. The second version of xvc includes such 772 a royalty-free baseline profile as described above. 774 The xvc codec is brought to the NETVC Working Group as a candidate 775 for the Internet Video Codec. The objectives of the Working Group 776 states that "This WG is chartered to produce a high-quality video 777 codec that meets the following conditions: 779 1. Is competitive (in the sense of having comparable or better 780 performance) with current video codecs in widespread use. 782 2. Is optimized for use in interactive web applications. 784 3. Is viewed as having IPR licensing terms that allow it to be 785 widely implemented and deployed." 787 The authors of this document believe that xvc is well positioned to 788 fulfill all of these conditions. A framework with a well-defined 789 single license and a royalty-free baseline profile ensures that 790 condition 1 and 3 can both be fulfilled, and the reference software 791 of xvc constitutes a good example of that the codec (and especially 792 the decoder) can be run in realtime on common hardware including 793 mobile devices, and thereby fulfill the second condition. 795 6. IANA Considerations 797 This document has no IANA considerations. 799 7. Security Considerations 801 This document has no security considerations. 803 8. Informative References 805 [av1Source] 806 Alliance for Open Media, "The av1 source code", n.d., 807 . 809 [AWCY] Xiph.Org Foundation, "Are We Compressed Yet?", n.d., 810 . 812 [Bjontegaard01] 813 Bjontegaard, G., "Calculation of average PSNR differences 814 between RD-curves", 2001, . 817 [cppStyle] 818 Google, "Google C++ Style Guide", n.d., 819 . 821 [Daede17] Daede, T., Norkin, A., and I. Brailovsky, "Video Codec 822 Testing and Quality Measurement", IETF NETVC Internet- 823 Draft draft-ietf-netvc-testing-06, October 2017. 825 [DAWCY] Xiph.Org Foundation, "Are We Compressed Yet? Divideon 826 clone", n.d., . 828 [Egiazarian2006] 829 Egiazarian, K., Astola, J., Ponomarenko, N., Lukin, V., 830 Battisti, F., and M. Carli, "Two new full-reference 831 quality metrics based on HVS", Proceedings of the Second 832 International Workshop on Video Processing and Quality 833 Metrics for Consumer Electronics VPQM, January 2006. 835 [HEVC] ISO/IEC/ITU-T, "High efficiency video coding", n.d., 836 . 838 [hmCtc] Sharman, K. and K. Suehring, "Common test conditions", 839 2017, 840 . 842 [hmSource] 843 ISO/IEC/ITU-T, "The HM source code", n.d., 844 . 846 [Testset] Daede, T., "Test Sets", n.d., 847 . 849 [Wang03] Wang, Z., Simoncelli, E., and A. Bovik, "Multiscale 850 structural similarity for image quality assessment", The 851 37th Asilomar Conference on Signals, Systems 852 Computers Volume 2, November 2003. 854 [Wang04] Wang, Z., Bovik, A., Sheikh, H., and E. Simoncelli, "Image 855 Quality Assessment: From Error Visibility to Structural 856 Similarity", issn 1057-7149, IEEE transactions on image 857 processing Volume 13, number 4, April 2004. 859 [Wenger16] 860 Wenger, S., "On profiles and per-feature Flags", 2016, 861 . 864 [xvcCall] Divideon, "Call for Patents in xvc", 2018, 865 . 868 [xvcDemo] Divideon, "Mobile video streaming with xvc", 2018, 869 . 872 [xvcio] Divideon, "The xvc web page", n.d., . 874 [xvcLicense] 875 Divideon, "The xvc license", n.d., 876 . 878 [xvcSource] 879 Divideon, "The xvc source code", n.d., 880 . 882 [xvcWp] Samuelsson, J. and P. Hermansson, "Introducing xvc - a 883 Divideon white paper", 2017, . 887 Authors' Addresses 888 Jonatan Samuelsson 889 Divideon 890 Kulstoetarvaegen 14 891 Stockholm 12240 892 Sweden 894 Email: jonatan.samuelsson@divideon.com 896 Per Hermansson 897 Divideon 898 Kulstoetarvaegen 14 899 Stockholm 12240 900 Sweden 902 Email: per.hermansson@divideon.com