idnits 2.17.1 draft-ietf-cellar-ffv1-v4-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 27, 2018) is 2099 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: '1' on line 602 -- Looks like a reference, but probably isn't: '2' on line 602 == Outdated reference: A later version (-20) exists of draft-ietf-cellar-ffv1-03 ** Downref: Normative reference to an Informational draft: draft-ietf-cellar-ffv1 (ref. 'I-D.ietf-cellar-ffv1') ** Obsolete normative reference: RFC 4288 (Obsoleted by RFC 6838) ** Downref: Normative reference to an Informational RFC: RFC 4732 Summary: 3 errors (**), 0 flaws (~~), 2 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 cellar M. Niedermayer 3 Internet-Draft 4 Intended status: Standards Track D. Rice 5 Expires: January 28, 2019 6 J. Martinez 7 July 27, 2018 9 FFV1 Video Coding Format Version 4 10 draft-ietf-cellar-ffv1-v4-01 12 Abstract 14 This document defines FFV1, a lossless intra-frame video encoding 15 format. FFV1 is designed to efficiently compress video data in a 16 variety of pixel formats. Compared to uncompressed video, FFV1 17 offers storage compression, frame fixity, and self-description, which 18 makes FFV1 useful as a preservation or intermediate video format. 20 Status of This Memo 22 This Internet-Draft is submitted in full conformance with the 23 provisions of BCP 78 and BCP 79. 25 Internet-Drafts are working documents of the Internet Engineering 26 Task Force (IETF). Note that other groups may also distribute 27 working documents as Internet-Drafts. The list of current Internet- 28 Drafts is at https://datatracker.ietf.org/drafts/current/. 30 Internet-Drafts are draft documents valid for a maximum of six months 31 and may be updated, replaced, or obsoleted by other documents at any 32 time. It is inappropriate to use Internet-Drafts as reference 33 material or to cite them other than as "work in progress." 35 This Internet-Draft will expire on January 28, 2019. 37 Copyright Notice 39 Copyright (c) 2018 IETF Trust and the persons identified as the 40 document authors. All rights reserved. 42 This document is subject to BCP 78 and the IETF Trust's Legal 43 Provisions Relating to IETF Documents 44 (https://trustee.ietf.org/license-info) in effect on the date of 45 publication of this document. Please review these documents 46 carefully, as they describe your rights and restrictions with respect 47 to this document. Code Components extracted from this document must 48 include Simplified BSD License text as described in Section 4.e of 49 the Trust Legal Provisions and are provided without warranty as 50 described in the Simplified BSD License. 52 Table of Contents 54 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4 55 2. Notation and Conventions . . . . . . . . . . . . . . . . . . 4 56 2.1. Definitions . . . . . . . . . . . . . . . . . . . . . . . 4 57 2.2. Conventions . . . . . . . . . . . . . . . . . . . . . . . 5 58 2.2.1. Pseudo-code . . . . . . . . . . . . . . . . . . . . . 5 59 2.2.2. Arithmetic Operators . . . . . . . . . . . . . . . . 5 60 2.2.3. Assignment Operators . . . . . . . . . . . . . . . . 6 61 2.2.4. Comparison Operators . . . . . . . . . . . . . . . . 6 62 2.2.5. Mathematical Functions . . . . . . . . . . . . . . . 7 63 2.2.6. Order of Operation Precedence . . . . . . . . . . . . 7 64 2.2.7. Range . . . . . . . . . . . . . . . . . . . . . . . . 8 65 2.2.8. NumBytes . . . . . . . . . . . . . . . . . . . . . . 8 66 2.2.9. Bitstream Functions . . . . . . . . . . . . . . . . . 8 67 3. General Description . . . . . . . . . . . . . . . . . . . . . 8 68 3.1. Border . . . . . . . . . . . . . . . . . . . . . . . . . 8 69 3.2. Samples . . . . . . . . . . . . . . . . . . . . . . . . . 9 70 3.3. Median Predictor . . . . . . . . . . . . . . . . . . . . 9 71 3.4. Context . . . . . . . . . . . . . . . . . . . . . . . . . 10 72 3.5. Quantization Table Sets . . . . . . . . . . . . . . . . . 11 73 3.6. Quantization Table Set Indexes . . . . . . . . . . . . . 11 74 3.7. Color spaces . . . . . . . . . . . . . . . . . . . . . . 11 75 3.7.1. YCbCr . . . . . . . . . . . . . . . . . . . . . . . . 11 76 3.7.2. RGB . . . . . . . . . . . . . . . . . . . . . . . . . 12 77 3.8. Coding of the Sample Difference . . . . . . . . . . . . . 13 78 3.8.1. Range Coding Mode . . . . . . . . . . . . . . . . . . 13 79 3.8.2. Golomb Rice Mode . . . . . . . . . . . . . . . . . . 17 80 4. Bitstream . . . . . . . . . . . . . . . . . . . . . . . . . . 19 81 4.1. Parameters . . . . . . . . . . . . . . . . . . . . . . . 20 82 4.1.1. version . . . . . . . . . . . . . . . . . . . . . . . 21 83 4.1.2. micro_version . . . . . . . . . . . . . . . . . . . . 22 84 4.1.3. coder_type . . . . . . . . . . . . . . . . . . . . . 23 85 4.1.4. state_transition_delta . . . . . . . . . . . . . . . 23 86 4.1.5. colorspace_type . . . . . . . . . . . . . . . . . . . 23 87 4.1.6. chroma_planes . . . . . . . . . . . . . . . . . . . . 24 88 4.1.7. bits_per_raw_sample . . . . . . . . . . . . . . . . . 24 89 4.1.8. log2_h_chroma_subsample . . . . . . . . . . . . . . . 24 90 4.1.9. log2_v_chroma_subsample . . . . . . . . . . . . . . . 24 91 4.1.10. alpha_plane . . . . . . . . . . . . . . . . . . . . . 24 92 4.1.11. num_h_slices . . . . . . . . . . . . . . . . . . . . 25 93 4.1.12. num_v_slices . . . . . . . . . . . . . . . . . . . . 25 94 4.1.13. quant_table_set_count . . . . . . . . . . . . . . . . 25 95 4.1.14. states_coded . . . . . . . . . . . . . . . . . . . . 25 96 4.1.15. initial_state_delta . . . . . . . . . . . . . . . . . 25 97 4.1.16. ec . . . . . . . . . . . . . . . . . . . . . . . . . 25 98 4.1.17. intra . . . . . . . . . . . . . . . . . . . . . . . . 26 99 4.2. Configuration Record . . . . . . . . . . . . . . . . . . 26 100 4.2.1. reserved_for_future_use . . . . . . . . . . . . . . . 26 101 4.2.2. configuration_record_crc_parity . . . . . . . . . . . 27 102 4.2.3. Mapping FFV1 into Containers . . . . . . . . . . . . 27 103 4.3. Frame . . . . . . . . . . . . . . . . . . . . . . . . . . 28 104 4.4. Slice . . . . . . . . . . . . . . . . . . . . . . . . . . 29 105 4.5. Slice Header . . . . . . . . . . . . . . . . . . . . . . 29 106 4.5.1. slice_x . . . . . . . . . . . . . . . . . . . . . . . 30 107 4.5.2. slice_y . . . . . . . . . . . . . . . . . . . . . . . 30 108 4.5.3. slice_width . . . . . . . . . . . . . . . . . . . . . 30 109 4.5.4. slice_height . . . . . . . . . . . . . . . . . . . . 30 110 4.5.5. quant_table_set_index_count . . . . . . . . . . . . . 30 111 4.5.6. quant_table_set_index . . . . . . . . . . . . . . . . 31 112 4.5.7. picture_structure . . . . . . . . . . . . . . . . . . 31 113 4.5.8. sar_num . . . . . . . . . . . . . . . . . . . . . . . 31 114 4.5.9. sar_den . . . . . . . . . . . . . . . . . . . . . . . 31 115 4.5.10. reset_contexts . . . . . . . . . . . . . . . . . . . 31 116 4.5.11. slice_coding_mode . . . . . . . . . . . . . . . . . . 31 117 4.6. Slice Content . . . . . . . . . . . . . . . . . . . . . . 32 118 4.6.1. primary_color_count . . . . . . . . . . . . . . . . . 32 119 4.6.2. plane_pixel_height . . . . . . . . . . . . . . . . . 32 120 4.6.3. slice_pixel_height . . . . . . . . . . . . . . . . . 32 121 4.6.4. slice_pixel_y . . . . . . . . . . . . . . . . . . . . 33 122 4.7. Line . . . . . . . . . . . . . . . . . . . . . . . . . . 33 123 4.7.1. plane_pixel_width . . . . . . . . . . . . . . . . . . 33 124 4.7.2. slice_pixel_width . . . . . . . . . . . . . . . . . . 33 125 4.7.3. slice_pixel_x . . . . . . . . . . . . . . . . . . . . 33 126 4.7.4. sample_difference . . . . . . . . . . . . . . . . . . 33 127 4.8. Slice Footer . . . . . . . . . . . . . . . . . . . . . . 34 128 4.8.1. slice_size . . . . . . . . . . . . . . . . . . . . . 34 129 4.8.2. error_status . . . . . . . . . . . . . . . . . . . . 34 130 4.8.3. slice_crc_parity . . . . . . . . . . . . . . . . . . 34 131 4.9. Quantization Table Set . . . . . . . . . . . . . . . . . 34 132 4.9.1. quant_tables . . . . . . . . . . . . . . . . . . . . 36 133 4.9.2. context_count . . . . . . . . . . . . . . . . . . . . 36 134 5. Restrictions . . . . . . . . . . . . . . . . . . . . . . . . 36 135 6. Security Considerations . . . . . . . . . . . . . . . . . . . 36 136 7. Media Type Definition . . . . . . . . . . . . . . . . . . . . 37 137 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 39 138 9. Appendixes . . . . . . . . . . . . . . . . . . . . . . . . . 39 139 9.1. Decoder implementation suggestions . . . . . . . . . . . 39 140 9.1.1. Multi-threading Support and Independence of Slices . 39 141 10. Changelog . . . . . . . . . . . . . . . . . . . . . . . . . . 39 142 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 39 143 11.1. Normative References . . . . . . . . . . . . . . . . . . 39 144 11.2. Informative References . . . . . . . . . . . . . . . . . 40 146 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 41 148 1. Introduction 150 This document describes FFV1, a lossless video encoding format. The 151 design of FFV1 considers the storage of image characteristics, data 152 fixity, and the optimized use of encoding time and storage 153 requirements. FFV1 is designed to support a wide range of lossless 154 video applications such as long-term audiovisual preservation, 155 scientific imaging, screen recording, and other video encoding 156 scenarios that seek to avoid the generational loss of lossy video 157 encodings. 159 This document defines a version 4 of FFV1. Prior versions of FFV1 160 are defined within [I-D.ietf-cellar-ffv1]. 162 The latest version of this document is available at 163 165 This document assumes familiarity with mathematical and coding 166 concepts such as Range coding [range-coding] and YCbCr color spaces 167 [YCbCr]. 169 2. Notation and Conventions 171 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 172 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 173 document are to be interpreted as described in [RFC2119]. 175 2.1. Definitions 177 "Frame": An encoded representation of a complete static image. 179 "Slice": A spatial sub-section of a "Frame" that is encoded 180 separately from an other region of the same frame. 182 "Container": Format that encapsulates "Frames" and (when required) a 183 "Configuration Record" into a bitstream. 185 "Sample": The smallest addressable representation of a color 186 component or a luma component in a "Frame". Examples of sample are 187 Luma, Blue Chrominance, Red Chrominance, Alpha, Red, Green, and Blue. 189 "Pixel": The smallest addressable representation of a color in a 190 "Frame". It is composed of 1 or more samples. 192 "ESC": An ESCape symbol to indicate that the symbol to be stored is 193 too large for normal storage and that an alternate storage method. 195 "MSB": Most Significant Bit, the bit that can cause the largest 196 change in magnitude of the symbol. 198 "RCT": Reversible Color Transform, a near linear, exactly reversible 199 integer transform that converts between RGB and YCbCr representations 200 of a Pixel. 202 "VLC": Variable Length Code, a code that maps source symbols to a 203 variable number of bits. 205 "RGB": A reference to the method of storing the value of a Pixel by 206 using three numeric values that represent Red, Green, and Blue. 208 "YCbCr": A reference to the method of storing the value of a Pixel by 209 using three numeric values that represent the luma of the Pixel (Y) 210 and the chrominance of the Pixel (Cb and Cr). YCbCr word is used for 211 historical reasons and currently references any color space relying 212 on 1 luma sample and 2 chrominance samples e.g. YCbCr, YCgCo or 213 ICtCp. Exact meaning of the three numeric values is unspecified. 215 "TBA": To Be Announced. Used in reference to the development of 216 future iterations of the FFV1 specification. 218 2.2. Conventions 220 2.2.1. Pseudo-code 222 The FFV1 bitstream is described in this document using pseudo-code. 223 Note that the pseudo-code is used for clarity in order to illustrate 224 the structure of FFV1 and not intended to specify any particular 225 implementation. The pseudo-code used is based upon the C programming 226 language [ISO.9899.1990] and uses its "if/else", "while" and "for" 227 functions as well as functions defined within this document. 229 2.2.2. Arithmetic Operators 231 Note: the operators and the order of precedence are the same as used 232 in the C programming language [ISO.9899.1990]. 234 "a + b" means a plus b. 236 "a - b" means a minus b. 238 "-a" means negation of a. 240 "a * b" means a multiplied by b. 242 "a / b" means a divided by b. 244 "a & b" means bit-wise "and" of a and b. 246 "a | b" means bit-wise "or" of a and b. 248 "a >> b" means arithmetic right shift of two's complement integer 249 representation of a by b binary digits. 251 "a << b" means arithmetic left shift of two's complement integer 252 representation of a by b binary digits. 254 2.2.3. Assignment Operators 256 "a = b" means a is assigned b. 258 "a++" is equivalent to a is assigned a + 1. 260 "a--" is equivalent to a is assigned a - 1. 262 "a += b" is equivalent to a is assigned a + b. 264 "a -= b" is equivalent to a is assigned a - b. 266 "a *= b" is equivalent to a is assigned a * b. 268 2.2.4. Comparison Operators 270 "a > b" means a is greater than b. 272 "a >= b" means a is greater than or equal to b. 274 "a < b" means a is less than b. 276 "a <= b" means a is less than or equal b. 278 "a == b" means a is equal to b. 280 "a != b" means a is not equal to b. 282 "a && b" means Boolean logical "and" of a and b. 284 "a || b" means Boolean logical "or" of a and b. 286 "!a" means Boolean logical "not" of a. 288 "a ? b : c" if a is true, then b, otherwise c. 290 2.2.5. Mathematical Functions 292 floor(a) the largest integer less than or equal to a 294 ceil(a) the smallest integer greater than or equal to a 296 sign(a) extracts the sign of a number, i.e. if a < 0 then -1, else if 297 a > 0 then 1, else 0 299 abs(a) the absolute value of a, i.e. abs(a) = sign(a)*a 301 log2(a) the base-two logarithm of a 303 min(a,b) the smallest of two values a and b 305 max(a,b) the largest of two values a and b 307 median(a,b,c) the numerical middle value in a data set of a, b, and 308 c, i.e. a+b+c-min(a,b,c)-max(a,b,c) 310 a_{b} the b-th value of a sequence of a 312 a_{b,c} the 'b,c'-th value of a sequence of a 314 2.2.6. Order of Operation Precedence 316 When order of precedence is not indicated explicitly by use of 317 parentheses, operations are evaluated in the following order (from 318 top to bottom, operations of same precedence being evaluated from 319 left to right). This order of operations is based on the order of 320 operations used in Standard C. 322 a++, a-- 323 !a, -a 324 a * b, a / b, a % b 325 a + b, a - b 326 a << b, a >> b 327 a < b, a <= b, a > b, a >= b 328 a == b, a != b 329 a & b 330 a | b 331 a && b 332 a || b 333 a ? b : c 334 a = b, a += b, a -= b, a *= b 336 2.2.7. Range 338 "a...b" means any value starting from a to b, inclusive. 340 2.2.8. NumBytes 342 "NumBytes" is a non-negative integer that expresses the size in 8-bit 343 octets of particular FFV1 "Configuration Record" or "Frame". FFV1 344 relies on its "Container" to store the "NumBytes" values, see 345 Section 4.2.3. 347 2.2.9. Bitstream Functions 349 2.2.9.1. remaining_bits_in_bitstream 351 "remaining_bits_in_bitstream( )" means the count of remaining bits 352 after the pointer in that "Configuration Record" or "Frame". It is 353 computed from the "NumBytes" value multiplied by 8 minus the count of 354 bits of that "Configuration Record" or "Frame" already read by the 355 bitstream parser. 357 2.2.9.2. byte_aligned 359 "byte_aligned( )" is true if "remaining_bits_in_bitstream( NumBytes 360 )" is a multiple of 8, otherwise false. 362 2.2.9.3. get_bits 364 "get_bits( i )" is the action to read the next "i" bits in the 365 bitstream, from most significant bit to least significant bit, and to 366 return the corresponding value. The pointer is increased by "i". 368 3. General Description 370 Samples within a plane are coded in raster scan order (left->right, 371 top->bottom). Each sample is predicted by the median predictor from 372 samples in the same plane and the difference is stored see 373 Section 3.8. 375 3.1. Border 377 A border is assumed for each coded slice for the purpose of the 378 predictor and context according to the following rules: 380 o one column of samples to the left of the coded slice is assumed as 381 identical to the samples of the leftmost column of the coded slice 382 shifted down by one row. The value of the topmost sample of the 383 column of samples to the left of the coded slice is assumed to be 384 "0" 386 o one column of samples to the right of the coded slice is assumed 387 as identical to the samples of the rightmost column of the coded 388 slice 390 o an additional column of samples to the left of the coded slice and 391 two rows of samples above the coded slice are assumed to be "0" 393 The following table depicts a slice of samples "a,b,c,d,e,f,g,h,i" 394 along with its assumed border. 396 +---+---+---+---+---+---+---+---+ 397 | 0 | 0 | | 0 | 0 | 0 | | 0 | 398 +---+---+---+---+---+---+---+---+ 399 | 0 | 0 | | 0 | 0 | 0 | | 0 | 400 +---+---+---+---+---+---+---+---+ 401 | | | | | | | | | 402 +---+---+---+---+---+---+---+---+ 403 | 0 | 0 | | a | b | c | | c | 404 +---+---+---+---+---+---+---+---+ 405 | 0 | a | | d | e | f | | f | 406 +---+---+---+---+---+---+---+---+ 407 | 0 | d | | g | h | i | | i | 408 +---+---+---+---+---+---+---+---+ 410 3.2. Samples 412 Positions used for context and median predictor are: 414 +---+---+---+---+ 415 | | | T | | 416 +---+---+---+---+ 417 | |tl | t |tr | 418 +---+---+---+---+ 419 | L | l | X | | 420 +---+---+---+---+ 422 "X" is the current processed Sample. The identifiers are made of the 423 first letters of the words Top, Left and Right. 425 3.3. Median Predictor 427 The prediction for any sample value at position "X" may be computed 428 based upon the relative neighboring values of "l", "t", and "tl" via 429 this equation: 431 "median(l, t, l + t - tl)". 433 Note, this prediction template is also used in [ISO.14495-1.1999] and 434 [HuffYUV]. 436 Exception for the media predictor: if "colorspace_type == 0 && 437 bits_per_raw_sample == 16 && ( coder_type == 1 || coder_type == 2 )", 438 the following media predictor MUST be used: 440 "median(left16s, top16s, left16s + top16s - diag16s)" 442 where: 444 left16s = l >= 32768 ? ( l - 65536 ) : l 445 top16s = t >= 32768 ? ( t - 65536 ) : t 446 diag16s = tl >= 32768 ? ( tl - 65536 ) : tl 448 Background: a two's complement signed 16-bit signed integer was used 449 for storing sample values in all known implementations of FFV1 450 bitstream. So in some circumstances, the most significant bit was 451 wrongly interpreted (used as a sign bit instead of the 16th bit of an 452 unsigned integer). Note that when the issue is discovered, the only 453 configuration of all known implementations being impacted is 16-bit 454 YCbCr with no Pixel transformation with Range Coder coder, as other 455 potentially impacted configurations (e.g. 15/16-bit JPEG2000-RCT with 456 Range Coder coder, or 16-bit content with Golomb Rice coder) were 457 implemented nowhere [ISO.15444-1.2016]. In the meanwhile, 16-bit 458 JPEG2000-RCT with Range Coder coder was implemented without this 459 issue in one implementation and validated by one conformance checker. 460 It is expected (to be confirmed) to remove this exception for the 461 media predictor in the next version of the FFV1 bitstream. 463 3.4. Context 465 Relative to any sample "X", the Quantized Sample Differences "L-l", 466 "l-tl", "tl-t", "T-t", and "t-tr" are used as context: 468 context = Q_{0}[l - tl] + 469 Q_{1}[tl - t] + 470 Q_{2}[t - tr] + 471 Q_{3}[L - l] + 472 Q_{4}[T - t] 474 If "context >= 0" then "context" is used and the difference between 475 the sample and its predicted value is encoded as is, else "-context" 476 is used and the difference between the sample and its predicted value 477 is encoded with a flipped sign. 479 3.5. Quantization Table Sets 481 The FFV1 bitstream contains 1 or more Quantization Table Sets. Each 482 Quantization Table Set contains exactly 5 Quantization Tables, each 483 Quantization Table corresponding to 1 of the 5 Quantized Sample 484 Differences. For each Quantization Table, both the number of 485 quantization steps and their distribution are stored in the FFV1 486 bitstream; each Quantization Table has exactly 256 entries, and the 8 487 least significant bits of the Quantized Sample Difference are used as 488 index: 490 Q_{j}[k] = quant_tables[i][j][k&255] 492 In this formula, "i" is the Quantization Table Set index, "j" is the 493 Quantized Table index, "k" the Quantized Sample Difference. 495 3.6. Quantization Table Set Indexes 497 For each plane of each slice, a Quantization Table Set is selected 498 from an index: 500 o For Y plane, "quant_table_set_index [ 0 ]" index is used 502 o For Cb and Cr planes, "quant_table_set_index [ 1 ]" index is used 504 o For Alpha plane, "quant_table_set_index [ (version <= 3 || 505 chroma_planes) ? 2 : 1 ]" index is used 507 Background: in first implementations of FFV1 bitstream, the index for 508 Cb and Cr planes was stored even if it is not used (chroma_planes set 509 to 0), this index is kept for version <= 3 in order to keep 510 compatibility with FFV1 bitstreams in the wild. 512 3.7. Color spaces 514 FFV1 supports two color spaces: YCbCr and RGB. Both color spaces 515 allow an optional Alpha plane that can be used to code transparency 516 data. 518 3.7.1. YCbCr 520 In YCbCr color space, the Cb and Cr planes are optional, but if used 521 then MUST be used together. Omitting the Cb and Cr planes codes the 522 frames in grayscale without color data. An FFV1 "Frame" using YCbCr 523 MUST use one of the following arrangements: 525 o Y 526 o Y, Alpha 528 o Y, Cb, Cr 530 o Y, Cb, Cr, Alpha 532 The Y plane MUST be coded first. If the Cb and Cr planes are used 533 then they MUST be coded after the Y plane. If an Alpha 534 (transparency) plane is used, then it MUST be coded last. 536 3.7.2. RGB 538 JPEG2000-RCT is a Reversible Color Transform that codes RGB (red, 539 green, blue) planes losslessly in a modified YCbCr color space 540 [ISO.15444-1.2016]. Reversible Pixel transformations between YCbCr 541 and RGB use the following formulae. 543 Cb=b-g 545 Cr=r-g 547 Y=g+(Cb+Cr)>>2 549 g=Y-(Cb+Cr)>>2 551 r=Cr+g 553 b=Cb+g 555 Exception for the JPEG2000-RCT conversion: if bits_per_raw_sample is 556 between 9 and 15 inclusive and alpha_plane is 0, the following 557 formulae for reversible conversions between YCbCr and RGB MUST be 558 used instead of the ones above: 560 Cb=g-b 562 Cr=r-b 564 Y=b+(Cb+Cr)>>2 566 b=Y-(Cb+Cr)>>2 568 r=Cr+b 570 g=Cb+b 572 Background: At the time of this writing, in all known implementations 573 of FFV1 bitstream, when bits_per_raw_sample was between 9 and 15 574 inclusive and alpha_plane is 0, GBR planes were used as BGR planes 575 during both encoding and decoding. In the meanwhile, 16-bit 576 JPEG2000-RCT was implemented without this issue in one implementation 577 and validated by one conformance checker. Methods to address this 578 exception for the transform are under consideration for the next 579 version of the FFV1 bitstream. 581 When FFV1 uses the JPEG2000-RCT, the horizontal lines are interleaved 582 to improve caching efficiency since it is most likely that the RCT 583 will immediately be converted to RGB during decoding. The 584 interleaved coding order is also Y, then Cb, then Cr, and then if 585 used Alpha. 587 As an example, a "Frame" that is two pixels wide and two pixels high, 588 could be comprised of the following structure: 590 +------------------------+------------------------+ 591 | Pixel[1,1] | Pixel[2,1] | 592 | Y[1,1] Cb[1,1] Cr[1,1] | Y[2,1] Cb[2,1] Cr[2,1] | 593 +------------------------+------------------------+ 594 | Pixel[1,2] | Pixel[2,2] | 595 | Y[1,2] Cb[1,2] Cr[1,2] | Y[2,2] Cb[2,2] Cr[2,2] | 596 +------------------------+------------------------+ 598 In JPEG2000-RCT, the coding order would be left to right and then top 599 to bottom, with values interleaved by lines and stored in this order: 601 Y[1,1] Y[2,1] Cb[1,1] Cb[2,1] Cr[1,1] Cr[2,1] Y[1,2] Y[2,2] Cb[1,2] 602 Cb[2,2] Cr[1,2] Cr[2,2] 604 3.8. Coding of the Sample Difference 606 Instead of coding the n+1 bits of the Sample Difference with Huffman 607 or Range coding (or n+2 bits, in the case of RCT), only the n (or 608 n+1) least significant bits are used, since this is sufficient to 609 recover the original sample. In the equation below, the term "bits" 610 represents bits_per_raw_sample+1 for RCT or bits_per_raw_sample 611 otherwise: 613 coder_input = 614 [(sample_difference + 2^(bits-1)) & (2^bits - 1)] - 2^(bits-1) 616 3.8.1. Range Coding Mode 618 Early experimental versions of FFV1 used the CABAC Arithmetic coder 619 from H.264 as defined in [ISO.14496-10.2014] but due to the uncertain 620 patent/royalty situation, as well as its slightly worse performance, 621 CABAC was replaced by a Range coder based on an algorithm defined by 622 G. Nigel and N. Martin in 1979 [range-coding]. 624 3.8.1.1. Range Binary Values 626 To encode binary digits efficiently a Range coder is used. "C_{i}" 627 is the i-th Context. "B_{i}" is the i-th byte of the bytestream. 628 "b_{i}" is the i-th Range coded binary value, "S_{0,i}" is the i-th 629 initial state, which is 128. The length of the bytestream encoding n 630 binary symbols is "j_{n}" bytes. 632 r_{i} = floor( ( R_{i} * S_{i,C_{i}} ) / 2^8 ) 634 S_{i+1,C_{i}} = zero_state_{S_{i,C_{i}}} XOR 635 l_i = L_i XOR 636 t_i = R_i - r_i <== 637 b_i = 0 <==> 638 L_i < R_i - r_i 640 S_{i+1,C_{i}} = one_state_{S_{i,C_{i}}} XOR 641 l_i = L_i - R_i + r_i XOR 642 t_i = r_i <== 643 b_i = 1 <==> 644 L_i >= R_i - r_i 646 S_{i+1,k} = S_{i,k} <== C_i != k 648 R_{i+1} = 2^8 * t_{i} XOR 649 L_{i+1} = 2^8 * l_{i} + B_{j_{i}} XOR 650 j_{i+1} = j_{i} + 1 <== 651 t_{i} < 2^8 653 R_{i+1} = t_{i} XOR 654 L_{i+1} = l_{i} XOR 655 j_{i+1} = j_{i} <== 656 t_{i} >= 2^8 658 R_{0} = 65280 660 L_{0} = 2^8 * B_{0} + B_{1} 662 j_{0} = 2 664 3.8.1.2. Range Non Binary Values 666 To encode scalar integers, it would be possible to encode each bit 667 separately and use the past bits as context. However that would mean 668 255 contexts per 8-bit symbol that is not only a waste of memory but 669 also requires more past data to reach a reasonably good estimate of 670 the probabilities. Alternatively assuming a Laplacian distribution 671 and only dealing with its variance and mean (as in Huffman coding) 672 would also be possible, however, for maximum flexibility and 673 simplicity, the chosen method uses a single symbol to encode if a 674 number is 0 and if not encodes the number using its exponent, 675 mantissa and sign. The exact contexts used are best described by the 676 following code, followed by some comments. 678 pseudo-code | type 679 --------------------------------------------------------------|----- 680 void put_symbol(RangeCoder *c, uint8_t *state, int v, int \ | 681 is_signed) { | 682 int i; | 683 put_rac(c, state+0, !v); | 684 if (v) { | 685 int a= abs(v); | 686 int e= log2(a); | 687 | 688 for (i=0; i=0; i--) | 693 put_rac(c, state+22+min(i,9), (a>>i)&1); //22..31 | 694 | 695 if (is_signed) | 696 put_rac(c, state+11 + min(e, 10), v < 0); //11..21| 697 } | 698 } | 700 3.8.1.3. Initial Values for the Context Model 702 At keyframes all Range coder state variables are set to their initial 703 state. 705 3.8.1.4. State Transition Table 707 one_state_{i} = 708 default_state_transition_{i} + state_transition_delta_{i} 710 zero_state_{i} = 256 - one_state_{256-i} 712 3.8.1.5. default_state_transition 713 0, 0, 0, 0, 0, 0, 0, 0, 20, 21, 22, 23, 24, 25, 26, 27, 715 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 37, 38, 39, 40, 41, 42, 717 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 56, 57, 719 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 721 74, 75, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 723 89, 90, 91, 92, 93, 94, 94, 95, 96, 97, 98, 99,100,101,102,103, 725 104,105,106,107,108,109,110,111,112,113,114,114,115,116,117,118, 727 119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,133, 729 134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149, 731 150,151,152,152,153,154,155,156,157,158,159,160,161,162,163,164, 733 165,166,167,168,169,170,171,171,172,173,174,175,176,177,178,179, 735 180,181,182,183,184,185,186,187,188,189,190,190,191,192,194,194, 737 195,196,197,198,199,200,201,202,202,204,205,206,207,208,209,209, 739 210,211,212,213,215,215,216,217,218,219,220,220,222,223,224,225, 741 226,227,227,229,229,230,231,232,234,234,235,236,237,238,239,240, 743 241,242,243,244,245,246,247,248,248, 0, 0, 0, 0, 0, 0, 0, 745 3.8.1.6. Alternative State Transition Table 747 The alternative state transition table has been built using iterative 748 minimization of frame sizes and generally performs better than the 749 default. To use it, the coder_type MUST be set to 2 and the 750 difference to the default MUST be stored in the parameters. The 751 reference implementation of FFV1 in FFmpeg uses this table by default 752 at the time of this writing when Range coding is used. 754 0, 10, 10, 10, 10, 16, 16, 16, 28, 16, 16, 29, 42, 49, 20, 49, 756 59, 25, 26, 26, 27, 31, 33, 33, 33, 34, 34, 37, 67, 38, 39, 39, 758 40, 40, 41, 79, 43, 44, 45, 45, 48, 48, 64, 50, 51, 52, 88, 52, 760 53, 74, 55, 57, 58, 58, 74, 60,101, 61, 62, 84, 66, 66, 68, 69, 762 87, 82, 71, 97, 73, 73, 82, 75,111, 77, 94, 78, 87, 81, 83, 97, 764 85, 83, 94, 86, 99, 89, 90, 99,111, 92, 93,134, 95, 98,105, 98, 766 105,110,102,108,102,118,103,106,106,113,109,112,114,112,116,125, 768 115,116,117,117,126,119,125,121,121,123,145,124,126,131,127,129, 770 165,130,132,138,133,135,145,136,137,139,146,141,143,142,144,148, 772 147,155,151,149,151,150,152,157,153,154,156,168,158,162,161,160, 774 172,163,169,164,166,184,167,170,177,174,171,173,182,176,180,178, 776 175,189,179,181,186,183,192,185,200,187,191,188,190,197,193,196, 778 197,194,195,196,198,202,199,201,210,203,207,204,205,206,208,214, 780 209,211,221,212,213,215,224,216,217,218,219,220,222,228,223,225, 782 226,224,227,229,240,230,231,232,233,234,235,236,238,239,237,242, 784 241,243,242,244,245,246,247,248,249,250,251,252,252,253,254,255, 786 3.8.2. Golomb Rice Mode 788 This coding mode uses Golomb Rice codes. The VLC is split into 2 789 parts, the prefix stores the most significant bits and the suffix 790 stores the k least significant bits or stores the whole number in the 791 ESC case. The end of the bitstream of the "Frame" is filled with 792 0-bits until that the bitstream contains a multiple of 8 bits. 794 3.8.2.1. Prefix 795 +----------------+-------+ 796 | bits | value | 797 +----------------+-------+ 798 | 1 | 0 | 799 | 01 | 1 | 800 | ... | ... | 801 | 0000 0000 0001 | 11 | 802 | 0000 0000 0000 | ESC | 803 +----------------+-------+ 805 3.8.2.2. Suffix 807 +-------+-----------------------------------------------------------+ 808 | non | the k least significant bits MSB first | 809 | ESC | | 810 | ESC | the value - 11, in MSB first order, ESC may only be used | 811 | | if the value cannot be coded as non ESC | 812 +-------+-----------------------------------------------------------+ 814 3.8.2.3. Examples 816 +-----+-------------------------+-------+ 817 | k | bits | value | 818 +-----+-------------------------+-------+ 819 | 0 | "1" | 0 | 820 | 0 | "001" | 2 | 821 | 2 | "1 00" | 0 | 822 | 2 | "1 10" | 2 | 823 | 2 | "01 01" | 5 | 824 | any | "000000000000 10000000" | 139 | 825 +-----+-------------------------+-------+ 827 3.8.2.4. Run Mode 829 Run mode is entered when the context is 0 and left as soon as a non-0 830 difference is found. The level is identical to the predicted one. 831 The run and the first different level are coded. 833 3.8.2.5. Run Length Coding 835 The run value is encoded in 2 parts, the prefix part stores the more 836 significant part of the run as well as adjusting the run_index that 837 determines the number of bits in the less significant part of the 838 run. The 2nd part of the value stores the less significant part of 839 the run as it is. The run_index is reset for each plane and slice to 840 0. 842 pseudo-code | type 843 --------------------------------------------------------------|----- 844 log2_run[41]={ | 845 0, 0, 0, 0, 1, 1, 1, 1, | 846 2, 2, 2, 2, 3, 3, 3, 3, | 847 4, 4, 5, 5, 6, 6, 7, 7, | 848 8, 9,10,11,12,13,14,15, | 849 16,17,18,19,20,21,22,23, | 850 24, | 851 }; | 852 | 853 if (run_count == 0 && run_mode == 1) { | 854 if (get_bits(1)) { | 855 run_count = 1 << log2_run[run_index]; | 856 if (x + run_count <= w) | 857 run_index++; | 858 } else { | 859 if (log2_run[run_index]) | 860 run_count = get_bits(log2_run[run_index]); | 861 else | 862 run_count = 0; | 863 if (run_index) | 864 run_index--; | 865 run_mode = 2; | 866 } | 867 } | 869 The log2_run function is also used within [ISO.14495-1.1999]. 871 3.8.2.6. Level Coding 873 Level coding is identical to the normal difference coding with the 874 exception that the 0 value is removed as it cannot occur: 876 if (diff>0) diff--; 877 encode(diff); 879 Note, this is different from JPEG-LS, which doesn't use prediction in 880 run mode and uses a different encoding and context model for the last 881 difference On a small set of test samples the use of prediction 882 slightly improved the compression rate. 884 4. Bitstream 885 +--------+----------------------------------------------------------+ 886 | Symbol | Definition | 887 +--------+----------------------------------------------------------+ 888 | u(n) | unsigned big endian integer using n bits | 889 | sg | Golomb Rice coded signed scalar symbol coded with the | 890 | | method described in Section 3.8.2 | 891 | br | Range coded Boolean (1-bit) symbol with the method | 892 | | described in Section 3.8.1.1 | 893 | ur | Range coded unsigned scalar symbol coded with the method | 894 | | described in Section 3.8.1.2 | 895 | sr | Range coded signed scalar symbol coded with the method | 896 | | described in Section 3.8.1.2 | 897 +--------+----------------------------------------------------------+ 899 The same context that is initialized to 128 is used for all fields in 900 the header. 902 The following MUST be provided by external means during 903 initialization of the decoder: 905 "frame_pixel_width" is defined as "Frame" width in pixels. 907 "frame_pixel_height" is defined as "Frame" height in pixels. 909 Default values at the decoder initialization phase: 911 "ConfigurationRecordIsPresent" is set to 0. 913 4.1. Parameters 914 pseudo-code | type 915 --------------------------------------------------------------|----- 916 Parameters( ) { | 917 version | ur 918 if (version >= 3) | 919 micro_version | ur 920 coder_type | ur 921 if (coder_type > 1) | 922 for (i = 1; i < 256; i++) | 923 state_transition_delta[ i ] | sr 924 colorspace_type | ur 925 if (version >= 1) | 926 bits_per_raw_sample | ur 927 chroma_planes | br 928 log2_h_chroma_subsample | ur 929 log2_v_chroma_subsample | ur 930 alpha_plane | br 931 if (version >= 3) { | 932 num_h_slices - 1 | ur 933 num_v_slices - 1 | ur 934 quant_table_set_count | ur 935 } | 936 for( i = 0; i < quant_table_set_count; i++ ) | 937 QuantizationTableSet( i ) | 938 if (version >= 3) { | 939 for( i = 0; i < quant_table_set_count; i++ ) { | 940 states_coded | br 941 if (states_coded) | 942 for( j = 0; j < context_count[ i ]; j++ ) | 943 for( k = 0; k < CONTEXT_SIZE; k++ ) | 944 initial_state_delta[ i ][ j ][ k ] | sr 945 } | 946 ec | ur 947 intra | ur 948 } | 949 } | 951 4.1.1. version 953 "version" specifies the version of the FFV1 bitstream. 954 Each version is incompatible with others versions: decoders SHOULD 955 reject a file due to unknown version. 956 Decoders SHOULD reject a file with version <= 1 && 957 ConfigurationRecordIsPresent == 1. 958 Decoders SHOULD reject a file with version >= 3 && 959 ConfigurationRecordIsPresent == 0. 961 +-------+-------------------------+ 962 | value | version | 963 +-------+-------------------------+ 964 | 0 | FFV1 version 0 | 965 | 1 | FFV1 version 1 | 966 | 2 | reserved* | 967 | 3 | FFV1 version 3 | 968 | 4 | FFV1 version 4 | 969 | Other | reserved for future use | 970 +-------+-------------------------+ 972 * Version 2 was never enabled in the encoder thus version 2 files 973 SHOULD NOT exist, and this document does not describe them to keep 974 the text simpler. 976 4.1.2. micro_version 978 "micro_version" specifies the micro-version of the FFV1 bitstream. 979 After a version is considered stable (a micro-version value is 980 assigned to be the first stable variant of a specific version), each 981 new micro-version after this first stable variant is compatible with 982 the previous micro-version: decoders SHOULD NOT reject a file due to 983 an unknown micro-version equal or above the micro-version considered 984 as stable. 986 Meaning of micro_version for version 3: 988 +-------+-------------------------+ 989 | value | micro_version | 990 +-------+-------------------------+ 991 | 0...3 | reserved* | 992 | 4 | first stable variant | 993 | Other | reserved for future use | 994 +-------+-------------------------+ 996 * development versions may be incompatible with the stable variants. 998 Meaning of micro_version for version 4 (note: at the time of writing 999 of this specification, version 4 is not considered stable so the 1000 first stable version value is to be announced in the future): 1002 +---------+-------------------------+ 1003 | value | micro_version | 1004 +---------+-------------------------+ 1005 | 0...TBA | reserved* | 1006 | TBA | first stable variant | 1007 | Other | reserved for future use | 1008 +---------+-------------------------+ 1010 * development versions which may be incompatible with the stable 1011 variants. 1013 4.1.3. coder_type 1015 "coder_type" specifies the coder used. 1017 +-------+-------------------------------------------------+ 1018 | value | coder used | 1019 +-------+-------------------------------------------------+ 1020 | 0 | Golomb Rice | 1021 | 1 | Range Coder with default state transition table | 1022 | 2 | Range Coder with custom state transition table | 1023 | Other | reserved for future use | 1024 +-------+-------------------------------------------------+ 1026 4.1.4. state_transition_delta 1028 "state_transition_delta" specifies the Range coder custom state 1029 transition table. 1030 If state_transition_delta is not present in the FFV1 bitstream, all 1031 Range coder custom state transition table elements are assumed to be 1032 0. 1034 4.1.5. colorspace_type 1036 "colorspace_type" specifies color space losslessly encoded, Pixel 1037 transformation used by the encoder, as well as interleave method. 1039 +-------+---------------------+------------------+------------------+ 1040 | value | color space | transformation | interleave | 1041 | | losslessly encoded | | method | 1042 +-------+---------------------+------------------+------------------+ 1043 | 0 | YCbCr | No Pixel | plane then line | 1044 | | | transformation | | 1045 | 1 | RGB | JPEG2000-RCT | line then plane | 1046 | Other | reserved for future | reserved for | reserved for | 1047 | | use | future use | future use | 1048 +-------+---------------------+------------------+------------------+ 1050 Restrictions: 1051 If "colorspace_type" is 1, then "chroma_planes" MUST be 1, 1052 "log2_h_chroma_subsample" MUST be 0, and "log2_v_chroma_subsample" 1053 MUST be 0. 1055 4.1.6. chroma_planes 1057 "chroma_planes" indicates if chroma (color) planes are present. 1059 +-------+-------------------------------+ 1060 | value | presence | 1061 +-------+-------------------------------+ 1062 | 0 | chroma planes are not present | 1063 | 1 | chroma planes are present | 1064 +-------+-------------------------------+ 1066 4.1.7. bits_per_raw_sample 1068 "bits_per_raw_sample" indicates the number of bits for each sample. 1069 Inferred to be 8 if not present. 1071 +-------+---------------------------------+ 1072 | value | bits for each sample | 1073 +-------+---------------------------------+ 1074 | 0 | reserved* | 1075 | Other | the actual bits for each sample | 1076 +-------+---------------------------------+ 1078 * Encoders MUST NOT store bits_per_raw_sample = 0 Decoders SHOULD 1079 accept and interpret bits_per_raw_sample = 0 as 8. 1081 4.1.8. log2_h_chroma_subsample 1083 "log2_h_chroma_subsample" indicates the subsample factor, stored in 1084 powers to which the number 2 must be raised, between luma and chroma 1085 width ("chroma_width = 2^(-log2_h_chroma_subsample) * luma_width"). 1087 4.1.9. log2_v_chroma_subsample 1089 "log2_v_chroma_subsample" indicates the subsample factor, stored in 1090 powers to which the number 2 must be raised, between luma and chroma 1091 height ("chroma_height=2^(-log2_v_chroma_subsample) * luma_height"). 1093 4.1.10. alpha_plane 1095 "alpha_plane" indicates if a transparency plane is present. 1097 +-------+-----------------------------------+ 1098 | value | presence | 1099 +-------+-----------------------------------+ 1100 | 0 | transparency plane is not present | 1101 | 1 | transparency plane is present | 1102 +-------+-----------------------------------+ 1104 4.1.11. num_h_slices 1106 "num_h_slices" indicates the number of horizontal elements of the 1107 slice raster. 1108 Inferred to be 1 if not present. 1110 4.1.12. num_v_slices 1112 "num_v_slices" indicates the number of vertical elements of the slice 1113 raster. 1114 Inferred to be 1 if not present. 1116 4.1.13. quant_table_set_count 1118 "quant_table_set_count" indicates the number of Quantization 1119 Table Sets. 1120 Inferred to be 1 if not present. 1121 MUST NOT be 0. 1123 4.1.14. states_coded 1125 "states_coded" indicates if the respective Quantization Table Set has 1126 the initial states coded. 1127 Inferred to be 0 if not present. 1129 +-------+-----------------------------------------------------------+ 1130 | value | initial states | 1131 +-------+-----------------------------------------------------------+ 1132 | 0 | initial states are not present and are assumed to be all | 1133 | | 128 | 1134 | 1 | initial states are present | 1135 +-------+-----------------------------------------------------------+ 1137 4.1.15. initial_state_delta 1139 "initial_state_delta[ i ][ j ][ k ]" indicates the initial Range 1140 coder state, it is encoded using "k" as context index and 1142 pred = j ? initial_states[ i ][j - 1][ k ] : 128 1144 initial_state[ i ][ j ][ k ] = 1145 ( pred + initial_state_delta[ i ][ j ][ k ] ) & 255 1147 4.1.16. ec 1149 "ec" indicates the error detection/correction type. 1151 +-------+--------------------------------------------+ 1152 | value | error detection/correction type | 1153 +-------+--------------------------------------------+ 1154 | 0 | 32-bit CRC on the global header | 1155 | 1 | 32-bit CRC per slice and the global header | 1156 | Other | reserved for future use | 1157 +-------+--------------------------------------------+ 1159 4.1.17. intra 1161 "intra" indicates the relationship between the instances of "Frame". 1162 Inferred to be 0 if not present. 1164 +-------+-----------------------------------------------------------+ 1165 | value | relationship | 1166 +-------+-----------------------------------------------------------+ 1167 | 0 | Frames are independent or dependent (keyframes and non | 1168 | | keyframes) | 1169 | 1 | Frames are independent (keyframes only) | 1170 | Other | reserved for future use | 1171 +-------+-----------------------------------------------------------+ 1173 4.2. Configuration Record 1175 In the case of a FFV1 bitstream with "version >= 3", a "Configuration 1176 Record" is stored in the underlying "Container", at the track header 1177 level. It contains the parameters used for all instances of "Frame". 1178 The size of the "Configuration Record", "NumBytes", is supplied by 1179 the underlying "Container". 1181 pseudo-code | type 1182 --------------------------------------------------------------|----- 1183 ConfigurationRecord( NumBytes ) { | 1184 ConfigurationRecordIsPresent = 1 | 1185 Parameters( ) | 1186 while( remaining_bits_in_bitstream( NumBytes ) > 32 ) | 1187 reserved_for_future_use | u(1) 1188 configuration_record_crc_parity | u(32) 1189 } | 1191 4.2.1. reserved_for_future_use 1193 "reserved_for_future_use" has semantics that are reserved for future 1194 use. 1195 Encoders conforming to this version of this specification SHALL NOT 1196 write this value. 1197 Decoders conforming to this version of this specification SHALL 1198 ignore its value. 1200 4.2.2. configuration_record_crc_parity 1202 "configuration_record_crc_parity" 32 bits that are chosen so that the 1203 "Configuration Record" as a whole has a crc remainder of 0. 1204 This is equivalent to storing the crc remainder in the 32-bit parity. 1205 The CRC generator polynomial used is the standard IEEE CRC polynomial 1206 (0x104C11DB7) with initial value 0. 1208 4.2.3. Mapping FFV1 into Containers 1210 This "Configuration Record" can be placed in any file format 1211 supporting "Configuration Records", fitting as much as possible with 1212 how the file format uses to store "Configuration Records". The 1213 "Configuration Record" storage place and "NumBytes" are currently 1214 defined and supported by this version of this specification for the 1215 following formats: 1217 4.2.3.1. AVI File Format 1219 The "Configuration Record" extends the stream format chunk ("AVI ", 1220 "hdlr", "strl", "strf") with the ConfigurationRecord bitstream. 1221 See [AVI] for more information about chunks. 1223 "NumBytes" is defined as the size, in bytes, of the strf chunk 1224 indicated in the chunk header minus the size of the stream format 1225 structure. 1227 4.2.3.2. ISO Base Media File Format 1229 The "Configuration Record" extends the sample description box 1230 ("moov", "trak", "mdia", "minf", "stbl", "stsd") with a "glbl" box 1231 that contains the ConfigurationRecord bitstream. See 1232 [ISO.14496-12.2015] for more information about boxes. 1234 "NumBytes" is defined as the size, in bytes, of the "glbl" box 1235 indicated in the box header minus the size of the box header. 1237 4.2.3.3. NUT File Format 1239 The codec_specific_data element (in "stream_header" packet) contains 1240 the ConfigurationRecord bitstream. See [NUT] for more information 1241 about elements. 1243 "NumBytes" is defined as the size, in bytes, of the 1244 codec_specific_data element as indicated in the "length" field of 1245 codec_specific_data 1247 4.2.3.4. Matroska File Format 1249 FFV1 SHOULD use "V_FFV1" as the Matroska "Codec ID". For FFV1 1250 versions 2 or less, the Matroska "CodecPrivate" Element SHOULD NOT be 1251 used. For FFV1 versions 3 or greater, the Matroska "CodecPrivate" 1252 Element MUST contain the FFV1 "Configuration Record" structure and no 1253 other data. See [Matroska] for more information about elements. 1255 "NumBytes" is defined as the "Element Data Size" of the 1256 "CodecPrivate" Element. 1258 4.3. Frame 1260 A "Frame" consists of the keyframe field, parameters (if version 1261 <=1), and a sequence of independent slices. 1263 pseudo-code | type 1264 --------------------------------------------------------------|----- 1265 Frame( NumBytes ) { | 1266 keyframe | br 1267 if (keyframe && !ConfigurationRecordIsPresent | 1268 Parameters( ) | 1269 while ( remaining_bits_in_bitstream( NumBytes ) ) | 1270 Slice( ) | 1271 } | 1273 Architecture overview of slices in a "Frame": 1275 +-----------------------------------------------------------------+ 1276 | first slice header | 1277 | first slice content | 1278 | first slice footer | 1279 | --------------------------------------------------------------- | 1280 | second slice header | 1281 | second slice content | 1282 | second slice footer | 1283 | --------------------------------------------------------------- | 1284 | ... | 1285 | --------------------------------------------------------------- | 1286 | last slice header | 1287 | last slice content | 1288 | last slice footer | 1289 +-----------------------------------------------------------------+ 1291 4.4. Slice 1293 pseudo-code | type 1294 --------------------------------------------------------------|----- 1295 Slice( ) { | 1296 if (version >= 3) | 1297 SliceHeader( ) | 1298 SliceContent( ) | 1299 if (coder_type == 0) | 1300 while (!byte_aligned()) | 1301 padding | u(1) 1302 if (version <= 1) { | 1303 while (remaining_bits_in_bitstream( NumBytes ) != 0 ) | 1304 reserved | u(1) 1305 } | 1306 if (version >= 3) | 1307 SliceFooter( ) | 1308 } | 1310 "padding" specifies a bit without any significance and used only for 1311 byte alignment. MUST be 0. 1313 "reserved" specifies a bit without any significance in this revision 1314 of the specification and may have a significance in a later revision 1315 of this specification. 1316 Encoders SHOULD NOT fill these bits. 1317 Decoders SHOULD ignore these bits. 1318 Note in case these bits are used in a later revision of this 1319 specification: any revision of this specification SHOULD care about 1320 avoiding to add 40 bits of content after "SliceContent" for version 0 1321 and 1 of the bitstream. Background: due to some non conforming 1322 encoders, some bitstreams where found with 40 extra bits 1323 corresponding to "error_status" and "slice_crc_parity", a decoder 1324 conforming to the revised specification could not do the difference 1325 between a revised bitstream and a buggy bitstream. 1327 4.5. Slice Header 1328 pseudo-code | type 1329 --------------------------------------------------------------|----- 1330 SliceHeader( ) { | 1331 slice_x | ur 1332 slice_y | ur 1333 slice_width - 1 | ur 1334 slice_height - 1 | ur 1335 for( i = 0; i < quant_table_set_index_count; i++ ) | 1336 quant_table_set_index [ i ] | ur 1337 picture_structure | ur 1338 sar_num | ur 1339 sar_den | ur 1340 if (version >= 4) { | 1341 reset_contexts | br 1342 slice_coding_mode | ur 1343 } | 1344 } | 1346 4.5.1. slice_x 1348 "slice_x" indicates the x position on the slice raster formed by 1349 num_h_slices. 1350 Inferred to be 0 if not present. 1352 4.5.2. slice_y 1354 "slice_y" indicates the y position on the slice raster formed by 1355 num_v_slices. 1356 Inferred to be 0 if not present. 1358 4.5.3. slice_width 1360 "slice_width" indicates the width on the slice raster formed by 1361 num_h_slices. 1362 Inferred to be 1 if not present. 1364 4.5.4. slice_height 1366 "slice_height" indicates the height on the slice raster formed by 1367 num_v_slices. 1368 Inferred to be 1 if not present. 1370 4.5.5. quant_table_set_index_count 1372 "quant_table_set_index_count" is defined as "1 + ( ( chroma_planes || 1373 version \<= 3 ) ? 1 : 0 ) + ( alpha_plane ? 1 : 0 )". 1375 4.5.6. quant_table_set_index 1377 "quant_table_set_index" indicates the Quantization Table Set index to 1378 select the Quantization Table Set and the initial states for the 1379 slice. 1380 Inferred to be 0 if not present. 1382 4.5.7. picture_structure 1384 "picture_structure" specifies the temporal and spatial relationship 1385 of each line of the "Frame". 1386 Inferred to be 0 if not present. 1388 +-------+-------------------------+ 1389 | value | picture structure used | 1390 +-------+-------------------------+ 1391 | 0 | unknown | 1392 | 1 | top field first | 1393 | 2 | bottom field first | 1394 | 3 | progressive | 1395 | Other | reserved for future use | 1396 +-------+-------------------------+ 1398 4.5.8. sar_num 1400 "sar_num" specifies the sample aspect ratio numerator. 1401 Inferred to be 0 if not present. 1402 MUST be 0 if sample aspect ratio is unknown. 1404 4.5.9. sar_den 1406 "sar_den" specifies the sample aspect ratio denominator. 1407 Inferred to be 0 if not present. 1408 MUST be 0 if sample aspect ratio is unknown. 1410 4.5.10. reset_contexts 1412 "reset_contexts" indicates if slice contexts must be reset. 1413 Inferred to be 0 if not present. 1415 4.5.11. slice_coding_mode 1417 "slice_coding_mode" indicates the slice coding mode. 1418 Inferred to be 0 if not present. 1420 +-------+-----------------------------+ 1421 | value | slice coding mode | 1422 +-------+-----------------------------+ 1423 | 0 | Range Coding or Golomb Rice | 1424 | 1 | raw PCM | 1425 | Other | reserved for future use | 1426 +-------+-----------------------------+ 1428 4.6. Slice Content 1430 pseudo-code | type 1431 --------------------------------------------------------------|----- 1432 SliceContent( ) { | 1433 if (colorspace_type == 0) { | 1434 for( p = 0; p < primary_color_count; p++ ) | 1435 for( y = 0; y < plane_pixel_height[ p ]; y++ ) | 1436 Line( p, y ) | 1437 } else if (colorspace_type == 1) { | 1438 for( y = 0; y < slice_pixel_height; y++ ) | 1439 for( p = 0; p < primary_color_count; p++ ) | 1440 Line( p, y ) | 1441 } | 1442 } | 1444 4.6.1. primary_color_count 1446 "primary_color_count" is defined as "1 + ( chroma_planes ? 2 : 0 ) + 1447 ( alpha_plane ? 1 : 0 )". 1449 4.6.2. plane_pixel_height 1451 "plane_pixel_height[ p ]" is the height in pixels of plane p of the 1452 slice. 1453 "plane_pixel_height[ 0 ]" and "plane_pixel_height[ 1 + ( 1454 chroma_planes ? 2 : 0 ) ]" value is "slice_pixel_height". 1455 If "chroma_planes" is set to 1, "plane_pixel_height[ 1 ]" and 1456 "plane_pixel_height[ 2 ]" value is "ceil(slice_pixel_height / 1457 log2_v_chroma_subsample)". 1459 4.6.3. slice_pixel_height 1461 "slice_pixel_height" is the height in pixels of the slice. 1462 Its value is "floor(( slice_y + slice_height ) * slice_pixel_height / 1463 num_v_slices) - slice_pixel_y". 1465 4.6.4. slice_pixel_y 1467 "slice_pixel_y" is the slice vertical position in pixels. 1468 Its value is "floor(slice_y * frame_pixel_height / num_v_slices)". 1470 4.7. Line 1472 pseudo-code | type 1473 --------------------------------------------------------------|----- 1474 Line( p, y ) { | 1475 if (colorspace_type == 0) { | 1476 for( x = 0; x < plane_pixel_width[ p ]; x++ ) | 1477 sample_difference[ p ][ y ][ x ] | 1478 } else if (colorspace_type == 1) { | 1479 for( x = 0; x < slice_pixel_width; x++ ) | 1480 sample_difference[ p ][ y ][ x ] | 1481 } | 1482 } | 1484 4.7.1. plane_pixel_width 1486 "plane_pixel_width[ p ]" is the width in pixels of plane p of the 1487 slice. 1488 "plane_pixel_width[ 0 ]" and "plane_pixel_width[ 1 + ( chroma_planes 1489 ? 2 : 0 ) ]" value is "slice_pixel_width". 1490 If "chroma_planes" is set to 1, "plane_pixel_width[ 1 ]" and 1491 "plane_pixel_width[ 2 ]" value is "ceil(slice_pixel_width / (1 << 1492 log2_h_chroma_subsample))". 1494 4.7.2. slice_pixel_width 1496 "slice_pixel_width" is the width in pixels of the slice. 1497 Its value is "floor(( slice_x + slice_width ) * slice_pixel_width / 1498 num_h_slices) - slice_pixel_x". 1500 4.7.3. slice_pixel_x 1502 "slice_pixel_x" is the slice horizontal position in pixels. 1503 Its value is "floor(slice_x * frame_pixel_width / num_h_slices)". 1505 4.7.4. sample_difference 1507 "sample_difference[ p ][ y ][ x ]" is the sample difference for 1508 sample at plane "p", y position "y", and x position "x". The sample 1509 value is computed based on prediction and context described in 1510 Section 3.2. 1512 4.8. Slice Footer 1514 Note: slice footer is always byte aligned. 1516 pseudo-code | type 1517 --------------------------------------------------------------|----- 1518 SliceFooter( ) { | 1519 slice_size | u(24) 1520 if (ec) { | 1521 error_status | u(8) 1522 slice_crc_parity | u(32) 1523 } | 1524 } | 1526 4.8.1. slice_size 1528 "slice_size" indicates the size of the slice in bytes. 1529 Note: this allows finding the start of slices before previous slices 1530 have been fully decoded, and allows parallel decoding as well as 1531 error resilience. 1533 4.8.2. error_status 1535 "error_status" specifies the error status. 1537 +-------+--------------------------------------+ 1538 | value | error status | 1539 +-------+--------------------------------------+ 1540 | 0 | no error | 1541 | 1 | slice contains a correctable error | 1542 | 2 | slice contains a uncorrectable error | 1543 | Other | reserved for future use | 1544 +-------+--------------------------------------+ 1546 4.8.3. slice_crc_parity 1548 "slice_crc_parity" 32 bits that are chosen so that the slice as a 1549 whole has a crc remainder of 0. 1550 This is equivalent to storing the crc remainder in the 32-bit parity. 1551 The CRC generator polynomial used is the standard IEEE CRC polynomial 1552 (0x104C11DB7) with initial value 0. 1554 4.9. Quantization Table Set 1556 The Quantization Table Sets are stored by storing the number of equal 1557 entries -1 of the first half of the table (represented as "len - 1" 1558 in the pseudo-code below) using the method described in 1559 Section 3.8.1.2. The second half doesn't need to be stored as it is 1560 identical to the first with flipped sign. "scale" and "len_count[ i 1561 ][ j ]" are temporary values used for the computing of 1562 "context_count[ i ]" and are not used outside Quantization Table Set 1563 pseudo-code. 1565 example: 1567 Table: 0 0 1 1 1 1 2 2 -2 -2 -2 -1 -1 -1 -1 0 1569 Stored values: 1, 3, 1 1571 pseudo-code | type 1572 --------------------------------------------------------------|----- 1573 QuantizationTableSet( i ) { | 1574 scale = 1 | 1575 for( j = 0; j < MAX_CONTEXT_INPUTS; j++ ) { | 1576 QuantizationTable( i, j, scale ) | 1577 scale *= 2 * len_count[ i ][ j ] - 1 | 1578 } | 1579 context_count[ i ] = ceil ( scale / 2 ) | 1580 } | 1582 MAX_CONTEXT_INPUTS is 5. 1584 pseudo-code | type 1585 --------------------------------------------------------------|----- 1586 QuantizationTable(i, j, scale) { | 1587 v = 0 | 1588 for( k = 0; k < 128; ) { | 1589 len - 1 | ur 1590 for( a = 0; a < len; a++ ) { | 1591 quant_tables[ i ][ j ][ k ] = scale* v | 1592 k++ | 1593 } | 1594 v++ | 1595 } | 1596 for( k = 1; k < 128; k++ ) { | 1597 quant_tables[ i ][ j ][ 256 - k ] = \ | 1598 -quant_tables[ i ][ j ][ k ] | 1599 } | 1600 quant_tables[ i ][ j ][ 128 ] = \ | 1601 -quant_tables[ i ][ j ][ 127 ] | 1602 len_count[ i ][ j ] = v | 1603 } | 1605 4.9.1. quant_tables 1607 "quant_tables[ i ][ j ][ k ]" indicates the quantification table 1608 value of the Quantized Sample Difference "k" of the Quantization 1609 Table "j" of the Set Quantization Table Set "i". 1611 4.9.2. context_count 1613 "context_count[ i ]" indicates the count of contexts for Quantization 1614 Table Set "i". 1616 5. Restrictions 1618 To ensure that fast multithreaded decoding is possible, starting 1619 version 3 and if frame_pixel_width * frame_pixel_height is more than 1620 101376, slice_width * slice_height MUST be less or equal to 1621 num_h_slices * num_v_slices / 4. Note: 101376 is the frame size in 1622 pixels of a 352x288 frame also known as CIF ("Common Intermediate 1623 Format") frame size format. 1625 For each "Frame", each position in the slice raster MUST be filled by 1626 one and only one slice of the "Frame" (no missing slice position, no 1627 slice overlapping). 1629 For each "Frame" with keyframe value of 0, each slice MUST have the 1630 same value of slice_x, slice_y, slice_width, slice_height as a slice 1631 in the previous "Frame", except if reset_contexts is 1. 1633 6. Security Considerations 1635 Like any other codec, (such as [RFC6716]), FFV1 should not be used 1636 with insecure ciphers or cipher-modes that are vulnerable to known 1637 plaintext attacks. Some of the header bits as well as the padding 1638 are easily predictable. 1640 Implementations of the FFV1 codec need to take appropriate security 1641 considerations into account, as outlined in [RFC4732]. It is 1642 extremely important for the decoder to be robust against malicious 1643 payloads. Malicious payloads must not cause the decoder to overrun 1644 its allocated memory or to take an excessive amount of resources to 1645 decode. Although problems in encoders are typically rarer, the same 1646 applies to the encoder. Malicious video streams must not cause the 1647 encoder to misbehave because this would allow an attacker to attack 1648 transcoding gateways. A frequent security problem in image and video 1649 codecs is also to not check for integer overflows in Pixel count 1650 computations, that is to allocate width * height without considering 1651 that the multiplication result may have overflowed the arithmetic 1652 types range. 1654 The reference implementation [REFIMPL] contains no known buffer 1655 overflow or cases where a specially crafted packet or video segment 1656 could cause a significant increase in CPU load. 1658 The reference implementation [REFIMPL] was validated in the following 1659 conditions: 1661 o Sending the decoder valid packets generated by the reference 1662 encoder and verifying that the decoder's output matches the 1663 encoder's input. 1665 o Sending the decoder packets generated by the reference encoder and 1666 then subjected to random corruption. 1668 o Sending the decoder random packets that are not FFV1. 1670 In all of the conditions above, the decoder and encoder was run 1671 inside the [VALGRIND] memory debugger as well as clangs address 1672 sanitizer [Address-Sanitizer], which track reads and writes to 1673 invalid memory regions as well as the use of uninitialized memory. 1674 There were no errors reported on any of the tested conditions. 1676 7. Media Type Definition 1678 This registration is done using the template defined in [RFC6838] and 1679 following [RFC4855]. 1681 Type name: video 1683 Subtype name: FFV1 1685 Required parameters: None. 1687 Optional parameters: 1689 This parameter is used to signal the capabilities of a receiver 1690 implementation. This parameter MUST NOT be used for any other 1691 purpose. 1693 version: The version of the FFV1 encoding as defined by 1694 Section 4.1.1. 1696 micro_version: The micro_version of the FFV1 encoding as defined by 1697 Section 4.1.2. 1699 coder_type: The coder_type of the FFV1 encoding as defined by 1700 Section 4.1.3. 1702 colorspace_type: The colorspace_type of the FFV1 encoding as defined 1703 by Section 4.1.5. 1705 bits_per_raw_sample: The version of the FFV1 encoding as defined by 1706 Section 4.1.7. 1708 max-slices: The value of max-slices is an integer indicating the 1709 maximum count of slices with a frames of the FFV1 encoding. 1711 Encoding considerations: 1713 This media type is defined for encapsulation in several audiovisual 1714 container formats and contains binary data; see Section 4.2.3. This 1715 media type is framed binary data Section 4.8 of [RFC4288]. 1717 Security considerations: 1719 See Section 6 of this document. 1721 Interoperability considerations: None. 1723 Published specification: 1725 [I-D.ietf-cellar-ffv1] and RFC XXXX. 1727 [RFC Editor: Upon publication as an RFC, please replace "XXXX" with 1728 the number assigned to this document and remove this note.] 1730 Applications which use this media type: 1732 Any application that requires the transport of lossless video can use 1733 this media type. Some examples are, but not limited to screen 1734 recording, scientific imaging, and digital video preservation. 1736 Fragment identifier considerations: N/A. 1738 Additional information: None. 1740 Person & email address to contact for further information: Michael 1741 Niedermayer 1743 Intended usage: COMMON 1745 Restrictions on usage: None. 1747 Author: Dave Rice 1749 Change controller: IETF cellar working group delegated from the IESG. 1751 8. IANA Considerations 1753 The IANA is requested to register the following values: 1755 o Media type registration as described in Section 7. 1757 9. Appendixes 1759 9.1. Decoder implementation suggestions 1761 9.1.1. Multi-threading Support and Independence of Slices 1763 The FFV1 bitstream is parsable in two ways: in sequential order as 1764 described in this document or with the pre-analysis of the footer of 1765 each slice. Each slice footer contains a slice_size field so the 1766 boundary of each slice is computable without having to parse the 1767 slice content. That allows multi-threading as well as independence 1768 of slice content (a bitstream error in a slice header or slice 1769 content has no impact on the decoding of the other slices). 1771 After having checked keyframe field, a decoder SHOULD parse 1772 slice_size fields, from slice_size of the last slice at the end of 1773 the "Frame" up to slice_size of the first slice at the beginning of 1774 the "Frame", before parsing slices, in order to have slices 1775 boundaries. A decoder MAY fallback on sequential order e.g. in case 1776 of a corrupted "Frame" (frame size unknown, slice_size of slices not 1777 coherent...) or if there is no possibility of seek into the stream. 1779 10. Changelog 1781 See 1783 11. References 1785 11.1. Normative References 1787 [I-D.ietf-cellar-ffv1] 1788 Niedermayer, M., Rice, D., and J. Martinez, "FFV1 Video 1789 Coding Format Version 0, 1, and 3", draft-ietf-cellar- 1790 ffv1-03 (work in progress), June 2018. 1792 [ISO.15444-1.2016] 1793 International Organization for Standardization, 1794 "Information technology -- JPEG 2000 image coding system: 1795 Core coding system", October 2016. 1797 [ISO.9899.1990] 1798 International Organization for Standardization, 1799 "Programming languages - C", ISO Standard 9899, 1990. 1801 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1802 Requirement Levels", BCP 14, RFC 2119, 1803 DOI 10.17487/RFC2119, March 1997, 1804 . 1806 [RFC4288] Freed, N. and J. Klensin, "Media Type Specifications and 1807 Registration Procedures", RFC 4288, DOI 10.17487/RFC4288, 1808 December 2005, . 1810 [RFC4732] Handley, M., Ed., Rescorla, E., Ed., and IAB, "Internet 1811 Denial-of-Service Considerations", RFC 4732, 1812 DOI 10.17487/RFC4732, December 2006, 1813 . 1815 [RFC4855] Casner, S., "Media Type Registration of RTP Payload 1816 Formats", RFC 4855, DOI 10.17487/RFC4855, February 2007, 1817 . 1819 [RFC6716] Valin, JM., Vos, K., and T. Terriberry, "Definition of the 1820 Opus Audio Codec", RFC 6716, DOI 10.17487/RFC6716, 1821 September 2012, . 1823 [RFC6838] Freed, N., Klensin, J., and T. Hansen, "Media Type 1824 Specifications and Registration Procedures", BCP 13, 1825 RFC 6838, DOI 10.17487/RFC6838, January 2013, 1826 . 1828 11.2. Informative References 1830 [Address-Sanitizer] 1831 The Clang Team, "ASAN AddressSanitizer website", undated, 1832 . 1834 [AVI] Microsoft, "AVI RIFF File Reference", undated, 1835 . 1838 [HuffYUV] Rudiak-Gould, B., "HuffYUV", December 2003, 1839 . 1842 [ISO.14495-1.1999] 1843 International Organization for Standardization, 1844 "Information technology -- Lossless and near-lossless 1845 compression of continuous-tone still images: Baseline", 1846 December 1999. 1848 [ISO.14496-10.2014] 1849 International Organization for Standardization, 1850 "Information technology -- Coding of audio-visual objects 1851 -- Part 10: Advanced Video Coding", September 2014. 1853 [ISO.14496-12.2015] 1854 International Organization for Standardization, 1855 "Information technology -- Coding of audio-visual objects 1856 -- Part 12: ISO base media file format", December 2015. 1858 [Matroska] 1859 IETF, "Matroska", 2016, . 1862 [NUT] Niedermayer, M., "NUT Open Container Format", December 1863 2013, . 1865 [range-coding] 1866 Nigel, G. and N. Martin, "Range encoding: an algorithm for 1867 removing redundancy from a digitised message.", Proc. 1868 Institution of Electronic and Radio Engineers 1869 International Conference on Video and Data Recording , 1870 July 1979. 1872 [REFIMPL] Niedermayer, M., "The reference FFV1 implementation / the 1873 FFV1 codec in FFmpeg", undated, . 1875 [VALGRIND] 1876 Valgrind Developers, "Valgrind website", undated, 1877 . 1879 [YCbCr] Wikipedia, "YCbCr", undated, 1880 . 1882 Authors' Addresses 1884 Michael Niedermayer 1886 Email: michael@niedermayer.cc 1887 Dave Rice 1889 Email: dave@dericed.com 1891 Jerome Martinez 1893 Email: jerome@mediaarea.net