idnits 2.17.1 draft-ietf-cbor-7049bis-11.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (December 18, 2019) is 1591 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: '2' on line 2566 -- Looks like a reference, but probably isn't: '3' on line 2566 -- Looks like a reference, but probably isn't: '4' on line 2564 -- Looks like a reference, but probably isn't: '5' on line 2564 -- Looks like a reference, but probably isn't: '100' on line 1408 == Missing Reference: '-1' is mentioned on line 1404, but not defined -- Looks like a reference, but probably isn't: '1' on line 2850 == Missing Reference: 'RFCthis' is mentioned on line 2136, but not defined == Missing Reference: 'TM' is mentioned on line 2385, but not defined -- Looks like a reference, but probably isn't: '0' on line 2866 == Missing Reference: 'RFC4627' is mentioned on line 3005, but not defined ** Obsolete undefined reference: RFC 4627 (Obsoleted by RFC 7158, RFC 7159) == Missing Reference: 'CNN-TERMS' is mentioned on line 3007, but not defined -- Possible downref: Non-RFC (?) normative reference: ref. 'ECMA262' -- Possible downref: Non-RFC (?) normative reference: ref. 'IEEE754' -- Obsolete informational reference (is this intentional?): RFC 7049 (Obsoleted by RFC 8949) Summary: 1 error (**), 0 flaws (~~), 6 warnings (==), 12 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group C. Bormann 3 Internet-Draft Universitaet Bremen TZI 4 Obsoletes: 7049 (if approved) P. Hoffman 5 Intended status: Standards Track ICANN 6 Expires: June 20, 2020 December 18, 2019 8 Concise Binary Object Representation (CBOR) 9 draft-ietf-cbor-7049bis-11 11 Abstract 13 The Concise Binary Object Representation (CBOR) is a data format 14 whose design goals include the possibility of extremely small code 15 size, fairly small message size, and extensibility without the need 16 for version negotiation. These design goals make it different from 17 earlier binary serializations such as ASN.1 and MessagePack. 19 This document is a revised edition of RFC 7049, with editorial 20 improvements, added detail, and fixed errata. This revision formally 21 obsoletes RFC 7049, while keeping full compatibility of the 22 interchange format from RFC 7049. It does not create a new version 23 of the format. 25 Contributing 27 This document is being worked on in the CBOR Working Group. Please 28 contribute on the mailing list there, or in the GitHub repository for 29 this draft: https://github.com/cbor-wg/CBORbis 31 The charter for the CBOR Working Group says that the WG will update 32 RFC 7049 to fix verified errata. Security issues and clarifications 33 may be addressed, but changes to this document will ensure backward 34 compatibility for popular deployed codebases. This document will be 35 targeted at becoming an Internet Standard. 37 Status of This Memo 39 This Internet-Draft is submitted in full conformance with the 40 provisions of BCP 78 and BCP 79. 42 Internet-Drafts are working documents of the Internet Engineering 43 Task Force (IETF). Note that other groups may also distribute 44 working documents as Internet-Drafts. The list of current Internet- 45 Drafts is at https://datatracker.ietf.org/drafts/current/. 47 Internet-Drafts are draft documents valid for a maximum of six months 48 and may be updated, replaced, or obsoleted by other documents at any 49 time. It is inappropriate to use Internet-Drafts as reference 50 material or to cite them other than as "work in progress." 52 This Internet-Draft will expire on June 20, 2020. 54 Copyright Notice 56 Copyright (c) 2019 IETF Trust and the persons identified as the 57 document authors. All rights reserved. 59 This document is subject to BCP 78 and the IETF Trust's Legal 60 Provisions Relating to IETF Documents 61 (https://trustee.ietf.org/license-info) in effect on the date of 62 publication of this document. Please review these documents 63 carefully, as they describe your rights and restrictions with respect 64 to this document. Code Components extracted from this document must 65 include Simplified BSD License text as described in Section 4.e of 66 the Trust Legal Provisions and are provided without warranty as 67 described in the Simplified BSD License. 69 Table of Contents 71 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4 72 1.1. Objectives . . . . . . . . . . . . . . . . . . . . . . . 4 73 1.2. Terminology . . . . . . . . . . . . . . . . . . . . . . . 6 74 2. CBOR Data Models . . . . . . . . . . . . . . . . . . . . . . 7 75 2.1. Extended Generic Data Models . . . . . . . . . . . . . . 8 76 2.2. Specific Data Models . . . . . . . . . . . . . . . . . . 9 77 3. Specification of the CBOR Encoding . . . . . . . . . . . . . 9 78 3.1. Major Types . . . . . . . . . . . . . . . . . . . . . . . 11 79 3.2. Indefinite Lengths for Some Major Types . . . . . . . . . 13 80 3.2.1. The "break" Stop Code . . . . . . . . . . . . . . . . 13 81 3.2.2. Indefinite-Length Arrays and Maps . . . . . . . . . . 14 82 3.2.3. Indefinite-Length Byte Strings and Text Strings . . . 16 83 3.3. Floating-Point Numbers and Values with No Content . . . . 16 84 3.4. Tagging of Items . . . . . . . . . . . . . . . . . . . . 18 85 3.4.1. Standard Date/Time String . . . . . . . . . . . . . . 21 86 3.4.2. Epoch-based Date/Time . . . . . . . . . . . . . . . . 21 87 3.4.3. Bignums . . . . . . . . . . . . . . . . . . . . . . . 22 88 3.4.4. Decimal Fractions and Bigfloats . . . . . . . . . . . 22 89 3.4.5. Content Hints . . . . . . . . . . . . . . . . . . . . 24 90 3.4.5.1. Encoded CBOR Data Item . . . . . . . . . . . . . 24 91 3.4.5.2. Expected Later Encoding for CBOR-to-JSON 92 Converters . . . . . . . . . . . . . . . . . . . 24 93 3.4.5.3. Encoded Text . . . . . . . . . . . . . . . . . . 25 94 3.4.6. Self-Described CBOR . . . . . . . . . . . . . . . . . 26 95 4. Serialization Considerations . . . . . . . . . . . . . . . . 26 96 4.1. Preferred Serialization . . . . . . . . . . . . . . . . . 26 97 4.2. Deterministically Encoded CBOR . . . . . . . . . . . . . 27 98 4.2.1. Core Deterministic Encoding Requirements . . . . . . 28 99 4.2.2. Additional Deterministic Encoding Considerations . . 29 100 4.2.3. Length-first map key ordering . . . . . . . . . . . . 30 101 5. Creating CBOR-Based Protocols . . . . . . . . . . . . . . . . 31 102 5.1. CBOR in Streaming Applications . . . . . . . . . . . . . 32 103 5.2. Generic Encoders and Decoders . . . . . . . . . . . . . . 32 104 5.3. Validity of Items . . . . . . . . . . . . . . . . . . . . 33 105 5.3.1. Basic validity . . . . . . . . . . . . . . . . . . . 33 106 5.3.2. Tag validity . . . . . . . . . . . . . . . . . . . . 34 107 5.4. Validity and Evolution . . . . . . . . . . . . . . . . . 34 108 5.5. Numbers . . . . . . . . . . . . . . . . . . . . . . . . . 35 109 5.6. Specifying Keys for Maps . . . . . . . . . . . . . . . . 36 110 5.6.1. Equivalence of Keys . . . . . . . . . . . . . . . . . 37 111 5.7. Undefined Values . . . . . . . . . . . . . . . . . . . . 38 112 6. Converting Data between CBOR and JSON . . . . . . . . . . . . 38 113 6.1. Converting from CBOR to JSON . . . . . . . . . . . . . . 38 114 6.2. Converting from JSON to CBOR . . . . . . . . . . . . . . 39 115 7. Future Evolution of CBOR . . . . . . . . . . . . . . . . . . 41 116 7.1. Extension Points . . . . . . . . . . . . . . . . . . . . 41 117 7.2. Curating the Additional Information Space . . . . . . . . 42 118 8. Diagnostic Notation . . . . . . . . . . . . . . . . . . . . . 42 119 8.1. Encoding Indicators . . . . . . . . . . . . . . . . . . . 43 120 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 44 121 9.1. Simple Values Registry . . . . . . . . . . . . . . . . . 44 122 9.2. Tags Registry . . . . . . . . . . . . . . . . . . . . . . 44 123 9.3. Media Type ("MIME Type") . . . . . . . . . . . . . . . . 45 124 9.4. CoAP Content-Format . . . . . . . . . . . . . . . . . . . 46 125 9.5. The +cbor Structured Syntax Suffix Registration . . . . . 46 126 10. Security Considerations . . . . . . . . . . . . . . . . . . . 47 127 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 49 128 11.1. Normative References . . . . . . . . . . . . . . . . . . 49 129 11.2. Informative References . . . . . . . . . . . . . . . . . 50 130 Appendix A. Examples . . . . . . . . . . . . . . . . . . . . . . 53 131 Appendix B. Jump Table . . . . . . . . . . . . . . . . . . . . . 57 132 Appendix C. Pseudocode . . . . . . . . . . . . . . . . . . . . . 60 133 Appendix D. Half-Precision . . . . . . . . . . . . . . . . . . . 62 134 Appendix E. Comparison of Other Binary Formats to CBOR's Design 135 Objectives . . . . . . . . . . . . . . . . . . . . . 63 136 E.1. ASN.1 DER, BER, and PER . . . . . . . . . . . . . . . . . 64 137 E.2. MessagePack . . . . . . . . . . . . . . . . . . . . . . . 64 138 E.3. BSON . . . . . . . . . . . . . . . . . . . . . . . . . . 65 139 E.4. MSDTP: RFC 713 . . . . . . . . . . . . . . . . . . . . . 65 140 E.5. Conciseness on the Wire . . . . . . . . . . . . . . . . . 65 141 Appendix F. Changes from RFC 7049 . . . . . . . . . . . . . . . 66 142 Appendix G. Well-formedness errors and examples . . . . . . . . 66 143 G.1. Examples for CBOR data items that are not well-formed . . 67 144 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . 69 145 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 70 147 1. Introduction 149 There are hundreds of standardized formats for binary representation 150 of structured data (also known as binary serialization formats). Of 151 those, some are for specific domains of information, while others are 152 generalized for arbitrary data. In the IETF, probably the best-known 153 formats in the latter category are ASN.1's BER and DER [ASN.1]. 155 The format defined here follows some specific design goals that are 156 not well met by current formats. The underlying data model is an 157 extended version of the JSON data model [RFC8259]. It is important 158 to note that this is not a proposal that the grammar in RFC 8259 be 159 extended in general, since doing so would cause a significant 160 backwards incompatibility with already deployed JSON documents. 161 Instead, this document simply defines its own data model that starts 162 from JSON. 164 Appendix E lists some existing binary formats and discusses how well 165 they do or do not fit the design objectives of the Concise Binary 166 Object Representation (CBOR). 168 This document is a revised edition of [RFC7049], with editorial 169 improvements, added detail, and fixed errata. This revision formally 170 obsoletes RFC 7049, while keeping full compatibility of the 171 interchange format from RFC 7049. It does not create a new version 172 of the format. 174 1.1. Objectives 176 The objectives of CBOR, roughly in decreasing order of importance, 177 are: 179 1. The representation must be able to unambiguously encode most 180 common data formats used in Internet standards. 182 - It must represent a reasonable set of basic data types and 183 structures using binary encoding. "Reasonable" here is 184 largely influenced by the capabilities of JSON, with the major 185 addition of binary byte strings. The structures supported are 186 limited to arrays and trees; loops and lattice-style graphs 187 are not supported. 189 - There is no requirement that all data formats be uniquely 190 encoded; that is, it is acceptable that the number "7" might 191 be encoded in multiple different ways. 193 2. The code for an encoder or decoder must be able to be compact in 194 order to support systems with very limited memory, processor 195 power, and instruction sets. 197 - An encoder and a decoder need to be implementable in a very 198 small amount of code (for example, in class 1 constrained 199 nodes as defined in [RFC7228]). 201 - The format should use contemporary machine representations of 202 data (for example, not requiring binary-to-decimal 203 conversion). 205 3. Data must be able to be decoded without a schema description. 207 - Similar to JSON, encoded data should be self-describing so 208 that a generic decoder can be written. 210 4. The serialization must be reasonably compact, but data 211 compactness is secondary to code compactness for the encoder and 212 decoder. 214 - "Reasonable" here is bounded by JSON as an upper bound in 215 size, and by implementation complexity maintaining a lower 216 bound. Using either general compression schemes or extensive 217 bit-fiddling violates the complexity goals. 219 5. The format must be applicable to both constrained nodes and high- 220 volume applications. 222 - This means it must be reasonably frugal in CPU usage for both 223 encoding and decoding. This is relevant both for constrained 224 nodes and for potential usage in applications with a very high 225 volume of data. 227 6. The format must support all JSON data types for conversion to and 228 from JSON. 230 - It must support a reasonable level of conversion as long as 231 the data represented is within the capabilities of JSON. It 232 must be possible to define a unidirectional mapping towards 233 JSON for all types of data. 235 7. The format must be extensible, and the extended data must be 236 decodable by earlier decoders. 238 - The format is designed for decades of use. 240 - The format must support a form of extensibility that allows 241 fallback so that a decoder that does not understand an 242 extension can still decode the message. 244 - The format must be able to be extended in the future by later 245 IETF standards. 247 1.2. Terminology 249 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 250 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 251 "OPTIONAL" in this document are to be interpreted as described in 252 BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all 253 capitals, as shown here. 255 The term "byte" is used in its now-customary sense as a synonym for 256 "octet". All multi-byte values are encoded in network byte order 257 (that is, most significant byte first, also known as "big-endian"). 259 This specification makes use of the following terminology: 261 Data item: A single piece of CBOR data. The structure of a data 262 item may contain zero, one, or more nested data items. The term 263 is used both for the data item in representation format and for 264 the abstract idea that can be derived from that by a decoder; the 265 former can be addressed specifically by using "encoded data item". 267 Decoder: A process that decodes a well-formed encoded CBOR data item 268 and makes it available to an application. Formally speaking, a 269 decoder contains a parser to break up the input using the syntax 270 rules of CBOR, as well as a semantic processor to prepare the data 271 in a form suitable to the application. 273 Encoder: A process that generates the (well-formed) representation 274 format of a CBOR data item from application information. 276 Data Stream: A sequence of zero or more data items, not further 277 assembled into a larger containing data item. The independent 278 data items that make up a data stream are sometimes also referred 279 to as "top-level data items". 281 Well-formed: A data item that follows the syntactic structure of 282 CBOR. A well-formed data item uses the initial bytes and the byte 283 strings and/or data items that are implied by their values as 284 defined in CBOR and does not include following extraneous data. 285 CBOR decoders by definition only return contents from well-formed 286 data items. 288 Valid: A data item that is well-formed and also follows the semantic 289 restrictions that apply to CBOR data items. 291 Expected: Besides its normal English meaning, the term "expected" is 292 used to describe requirements beyond CBOR validity that an 293 application has on its input data. Well-formed (processable at 294 all), valid (checked by a validity-checking generic decoder), and 295 expected (checked by the application) form a hierarchy of layers 296 of acceptability. 298 Stream decoder: A process that decodes a data stream and makes each 299 of the data items in the sequence available to an application as 300 they are received. 302 Where bit arithmetic or data types are explained, this document uses 303 the notation familiar from the programming language C, except that 304 "**" denotes exponentiation. Similar to the "0x" notation for 305 hexadecimal numbers, numbers in binary notation are prefixed with 306 "0b". Underscores can be added to a number solely for readability, 307 so 0b00100001 (0x21) might be written 0b001_00001 to emphasize the 308 desired interpretation of the bits in the byte; in this case, it is 309 split into three bits and five bits. Encoded CBOR data items are 310 sometimes given in the "0x" or "0b" notation; these values are first 311 interpreted as numbers as in C and are then interpreted as byte 312 strings in network byte order, including any leading zero bytes 313 expressed in the notation. 315 2. CBOR Data Models 317 CBOR is explicit about its generic data model, which defines the set 318 of all data items that can be represented in CBOR. Its basic generic 319 data model is extensible by the registration of simple type values 320 and tags. Applications can then subset the resulting extended 321 generic data model to build their specific data models. 323 Within environments that can represent the data items in the generic 324 data model, generic CBOR encoders and decoders can be implemented 325 (which usually involves defining additional implementation data types 326 for those data items that do not already have a natural 327 representation in the environment). The ability to provide generic 328 encoders and decoders is an explicit design goal of CBOR; however 329 many applications will provide their own application-specific 330 encoders and/or decoders. 332 In the basic (un-extended) generic data model, a data item is one of: 334 * an integer in the range -2**64..2**64-1 inclusive 335 * a simple value, identified by a number between 0 and 255, but 336 distinct from that number 338 * a floating-point value, distinct from an integer, out of the set 339 representable by IEEE 754 binary64 (including non-finites) 340 [IEEE754] 342 * a sequence of zero or more bytes ("byte string") 344 * a sequence of zero or more Unicode code points ("text string") 346 * a sequence of zero or more data items ("array") 348 * a mapping (mathematical function) from zero or more data items 349 ("keys") each to a data item ("values"), ("map") 351 * a tagged data item ("tag"), comprising a tag number (an integer in 352 the range 0..2**64-1) and a tagged value (a data item) 354 Note that integer and floating-point values are distinct in this 355 model, even if they have the same numeric value. 357 Also note that serialization variants, such as the number of bytes of 358 the encoded floating value, or the choice of one of the ways in which 359 an integer, the length of a text or byte string, the number of 360 elements in an array or pairs in a map, or a tag number, 361 (collectively "the argument", see Section 3) can be encoded, are not 362 visible at the generic data model level. 364 2.1. Extended Generic Data Models 366 This basic generic data model comes pre-extended by the registration 367 of a number of simple values and tag numbers right in this document, 368 such as: 370 * "false", "true", "null", and "undefined" (simple values identified 371 by 20..23) 373 * integer and floating-point values with a larger range and 374 precision than the above (tag numbers 2 to 5) 376 * application data types such as a point in time or an RFC 3339 377 date/time string (tag numbers 1, 0) 379 Further elements of the extended generic data model can be (and have 380 been) defined via the IANA registries created for CBOR. Even if such 381 an extension is unknown to a generic encoder or decoder, data items 382 using that extension can be passed to or from the application by 383 representing them at the interface to the application within the 384 basic generic data model, i.e., as generic values of a simple type or 385 generic tags. 387 In other words, the basic generic data model is stable as defined in 388 this document, while the extended generic data model expands by the 389 registration of new simple values or tag numbers, but never shrinks. 391 While there is a strong expectation that generic encoders and 392 decoders can represent "false", "true", and "null" ("undefined" is 393 intentionally omitted) in the form appropriate for their programming 394 environment, implementation of the data model extensions created by 395 tags is truly optional and a matter of implementation quality. 397 2.2. Specific Data Models 399 The specific data model for a CBOR-based protocol usually subsets the 400 extended generic data model and assigns application semantics to the 401 data items within this subset and its components. When documenting 402 such specific data models, where it is desired to specify the types 403 of data items, it is preferred to identify the types by the names 404 they have in the generic data model ("negative integer", "array") 405 instead of by referring to aspects of their CBOR representation 406 ("major type 1", "major type 4"). 408 Specific data models can also specify what values (including values 409 of different types) are equivalent for the purposes of map keys and 410 encoder freedom. For example, in the generic data model, a valid map 411 MAY have both "0" and "0.0" as keys, and an encoder MUST NOT encode 412 "0.0" as an integer (major type 0, Section 3.1). However, if a 413 specific data model declares that floating-point and integer 414 representations of integral values are equivalent, using both map 415 keys "0" and "0.0" in a single map would be considered duplicates, 416 even while encoded as different major types, and so invalid; and an 417 encoder could encode integral-valued floats as integers or vice 418 versa, perhaps to save encoded bytes. 420 3. Specification of the CBOR Encoding 422 A CBOR data item (Section 2) is encoded to or decoded from a byte 423 string carrying a well-formed encoded data item as described in this 424 section. The encoding is summarized in Table 6, indexed by the 425 initial byte. An encoder MUST produce only well-formed encoded data 426 items. A decoder MUST NOT return a decoded data item when it 427 encounters input that is not a well-formed encoded CBOR data item 428 (this does not detract from the usefulness of diagnostic and recovery 429 tools that might make available some information from a damaged 430 encoded CBOR data item). 432 The initial byte of each encoded data item contains both information 433 about the major type (the high-order 3 bits, described in 434 Section 3.1) and additional information (the low-order 5 bits). With 435 a few exceptions, the additional information's value describes how to 436 load an unsigned integer "argument": 438 Less than 24: The argument's value is the value of the additional 439 information. 441 24, 25, 26, or 27: The argument's value is held in the following 1, 442 2, 4, or 8 bytes, respectively, in network byte order. For major 443 type 7 and additional information value 25, 26, 27, these bytes 444 are not used as an integer argument, but as a floating-point value 445 (see Section 3.3). 447 28, 29, 30: These values are reserved for future additions to the 448 CBOR format. In the present version of CBOR, the encoded item is 449 not well-formed. 451 31: No argument value is derived. If the major type is 0, 1, or 6, 452 the encoded item is not well-formed. For major types 2 to 5, the 453 item's length is indefinite, and for major type 7, the byte does 454 not consitute a data item at all but terminates an indefinite 455 length item; both are described in Section 3.2. 457 The initial byte and any additional bytes consumed to construct the 458 argument are collectively referred to as the "head" of the data item. 460 The meaning of this argument depends on the major type. For example, 461 in major type 0, the argument is the value of the data item itself 462 (and in major type 1 the value of the data item is computed from the 463 argument); in major type 2 and 3 it gives the length of the string 464 data in bytes that follows; and in major types 4 and 5 it is used to 465 determine the number of data items enclosed. 467 If the encoded sequence of bytes ends before the end of a data item, 468 that item is not well-formed. If the encoded sequence of bytes still 469 has bytes remaining after the outermost encoded item is decoded, that 470 encoding is not a single well-formed CBOR item; depending on the 471 application, the decoder may either treat the encoding as not well- 472 formed or just identify the start of the remaining bytes to the 473 application. 475 A CBOR decoder implementation can be based on a jump table with all 476 256 defined values for the initial byte (Table 6). A decoder in a 477 constrained implementation can instead use the structure of the 478 initial byte and following bytes for more compact code (see 479 Appendix C for a rough impression of how this could look). 481 3.1. Major Types 483 The following lists the major types and the additional information 484 and other bytes associated with the type. 486 Major type 0: an integer in the range 0..2**64-1 inclusive. The 487 value of the encoded item is the argument itself. For example, 488 the integer 10 is denoted as the one byte 0b000_01010 (major type 489 0, additional information 10). The integer 500 would be 490 0b000_11001 (major type 0, additional information 25) followed by 491 the two bytes 0x01f4, which is 500 in decimal. 493 Major type 1: a negative integer in the range -2**64..-1 inclusive. 494 The value of the item is -1 minus the argument. For example, the 495 integer -500 would be 0b001_11001 (major type 1, additional 496 information 25) followed by the two bytes 0x01f3, which is 499 in 497 decimal. 499 Major type 2: a byte string. The number of bytes in the string is 500 equal to the argument. For example, a byte string whose length is 501 5 would have an initial byte of 0b010_00101 (major type 2, 502 additional information 5 for the length), followed by 5 bytes of 503 binary content. A byte string whose length is 500 would have 3 504 initial bytes of 0b010_11001 (major type 2, additional information 505 25 to indicate a two-byte length) followed by the two bytes 0x01f4 506 for a length of 500, followed by 500 bytes of binary content. 508 Major type 3: a text string (Section 2), encoded as UTF-8 509 ([RFC3629]). The number of bytes in the string is equal to the 510 argument. A string containing an invalid UTF-8 sequence is well- 511 formed but invalid. This type is provided for systems that need 512 to interpret or display human-readable text, and allows the 513 differentiation between unstructured bytes and text that has a 514 specified repertoire and encoding. In contrast to formats such as 515 JSON, the Unicode characters in this type are never escaped. 516 Thus, a newline character (U+000A) is always represented in a 517 string as the byte 0x0a, and never as the bytes 0x5c6e (the 518 characters "\" and "n") or as 0x5c7530303061 (the characters "\", 519 "u", "0", "0", "0", and "a"). 521 Major type 4: an array of data items. Arrays are also called lists, 522 sequences, or tuples. The argument is the number of data items in 523 the array. Items in an array do not need to all be of the same 524 type. For example, an array that contains 10 items of any type 525 would have an initial byte of 0b100_01010 (major type of 4, 526 additional information of 10 for the length) followed by the 10 527 remaining items. 529 Major type 5: a map of pairs of data items. Maps are also called 530 tables, dictionaries, hashes, or objects (in JSON). A map is 531 comprised of pairs of data items, each pair consisting of a key 532 that is immediately followed by a value. The argument is the 533 number of _pairs_ of data items in the map. For example, a map 534 that contains 9 pairs would have an initial byte of 0b101_01001 535 (major type of 5, additional information of 9 for the number of 536 pairs) followed by the 18 remaining items. The first item is the 537 first key, the second item is the first value, the third item is 538 the second key, and so on. Because items in a map come in pairs, 539 their total number is always even: A map that contains an odd 540 number of items (no value data present after the last key data 541 item) is not well-formed. A map that has duplicate keys may be 542 well-formed, but it is not valid, and thus it causes indeterminate 543 decoding; see also Section 5.6. 545 Major type 6: a tagged data item ("tag") whose tag number is the 546 argument and whose enclosed data item ("tag content") is the 547 single encoded data item that follows the head. See Section 3.4. 549 Major type 7: floating-point numbers and simple values, as well as 550 the "break" stop code. See Section 3.3. 552 These eight major types lead to a simple table showing which of the 553 256 possible values for the initial byte of a data item are used 554 (Table 6). 556 In major types 6 and 7, many of the possible values are reserved for 557 future specification. See Section 9 for more information on these 558 values. 560 Table 1 summarizes the major types defined by CBOR, ignoring the next 561 section for now. The number N in this table stands for the argument, 562 mt for the major type. 564 +----+-----------------------+---------------------------------+ 565 | mt | Meaning | Content | 566 +----+-----------------------+---------------------------------+ 567 | 0 | unsigned integer N | - | 568 | | | | 569 | 1 | negative integer -1-N | - | 570 | | | | 571 | 2 | byte string | N bytes | 572 | | | | 573 | 3 | text string | N bytes (UTF-8 text) | 574 | | | | 575 | 4 | array | N data items (elements) | 576 | | | | 577 | 5 | map | 2N data items (key/value pairs) | 578 | | | | 579 | 6 | tag of number N | 1 data item | 580 | | | | 581 | 7 | simple/float | - | 582 +----+-----------------------+---------------------------------+ 584 Table 1: Overview over CBOR major types (definite length encoded) 586 3.2. Indefinite Lengths for Some Major Types 588 Four CBOR items (arrays, maps, byte strings, and text strings) can be 589 encoded with an indefinite length using additional information value 590 31. This is useful if the encoding of the item needs to begin before 591 the number of items inside the array or map, or the total length of 592 the string, is known. (The application of this is often referred to 593 as "streaming" within a data item.) 595 Indefinite-length arrays and maps are dealt with differently than 596 indefinite-length byte strings and text strings. 598 3.2.1. The "break" Stop Code 600 The "break" stop code is encoded with major type 7 and additional 601 information value 31 (0b111_11111). It is not itself a data item: it 602 is just a syntactic feature to close an indefinite-length item. 604 If the "break" stop code appears anywhere where a data item is 605 expected, other than directly inside an indefinite-length string, 606 array, or map -- for example directly inside a definite-length array 607 or map -- the enclosing item is not well-formed. 609 3.2.2. Indefinite-Length Arrays and Maps 611 Indefinite-length arrays and maps are represented using their major 612 type with the additional information value of 31, followed by an 613 arbitrary-length sequence of zero or more items for an array or key/ 614 value pairs for a map, followed by the "break" stop code 615 (Section 3.2.1). In other words, indefinite-length arrays and maps 616 look identical to other arrays and maps except for beginning with the 617 additional information value of 31 and ending with the "break" stop 618 code. 620 If the break stop code appears after a key in a map, in place of that 621 key's value, the map is not well-formed. 623 There is no restriction against nesting indefinite-length array or 624 map items. A "break" only terminates a single item, so nested 625 indefinite-length items need exactly as many "break" stop codes as 626 there are type bytes starting an indefinite-length item. 628 For example, assume an encoder wants to represent the abstract array 629 [1, [2, 3], [4, 5]]. The definite-length encoding would be 630 0x8301820203820405: 632 83 -- Array of length 3 633 01 -- 1 634 82 -- Array of length 2 635 02 -- 2 636 03 -- 3 637 82 -- Array of length 2 638 04 -- 4 639 05 -- 5 641 Indefinite-length encoding could be applied independently to each of 642 the three arrays encoded in this data item, as required, leading to 643 representations such as: 645 0x9f018202039f0405ffff 646 9F -- Start indefinite-length array 647 01 -- 1 648 82 -- Array of length 2 649 02 -- 2 650 03 -- 3 651 9F -- Start indefinite-length array 652 04 -- 4 653 05 -- 5 654 FF -- "break" (inner array) 655 FF -- "break" (outer array) 657 0x9f01820203820405ff 658 9F -- Start indefinite-length array 659 01 -- 1 660 82 -- Array of length 2 661 02 -- 2 662 03 -- 3 663 82 -- Array of length 2 664 04 -- 4 665 05 -- 5 666 FF -- "break" 668 0x83018202039f0405ff 669 83 -- Array of length 3 670 01 -- 1 671 82 -- Array of length 2 672 02 -- 2 673 03 -- 3 674 9F -- Start indefinite-length array 675 04 -- 4 676 05 -- 5 677 FF -- "break" 679 0x83019f0203ff820405 680 83 -- Array of length 3 681 01 -- 1 682 9F -- Start indefinite-length array 683 02 -- 2 684 03 -- 3 685 FF -- "break" 686 82 -- Array of length 2 687 04 -- 4 688 05 -- 5 690 An example of an indefinite-length map (that happens to have two key/ 691 value pairs) might be: 693 0xbf6346756ef563416d7421ff 694 BF -- Start indefinite-length map 695 63 -- First key, UTF-8 string length 3 696 46756e -- "Fun" 697 F5 -- First value, true 698 63 -- Second key, UTF-8 string length 3 699 416d74 -- "Amt" 700 21 -- Second value, -2 701 FF -- "break" 703 3.2.3. Indefinite-Length Byte Strings and Text Strings 705 Indefinite-length strings are represented by a byte containing the 706 major type and additional information value of 31, followed by a 707 series of zero or more byte or text strings ("chunks") that have 708 definite lengths, followed by the "break" stop code (Section 3.2.1). 709 The data item represented by the indefinite-length string is the 710 concatenation of the chunks (i.e., the empty byte or text string, 711 respectively, if no chunk is present). (Note that zero-length 712 chunks, while not particularly useful, are permitted.) 714 If any item between the indefinite-length string indicator 715 (0b010_11111 or 0b011_11111) and the "break" stop code is not a 716 definite-length string item of the same major type, the string is not 717 well-formed. 719 If any definite-length text string inside an indefinite-length text 720 string is invalid, the indefinite-length text string is invalid. 721 Note that this implies that the bytes of a single UTF-8 character 722 cannot be spread between chunks: a new chunk can only be started at a 723 character boundary. 725 For example, assume the sequence: 727 0b010_11111 0b010_00100 0xaabbccdd 0b010_00011 0xeeff99 0b111_11111 729 5F -- Start indefinite-length byte string 730 44 -- Byte string of length 4 731 aabbccdd -- Bytes content 732 43 -- Byte string of length 3 733 eeff99 -- Bytes content 734 FF -- "break" 736 After decoding, this results in a single byte string with seven 737 bytes: 0xaabbccddeeff99. 739 3.3. Floating-Point Numbers and Values with No Content 741 Major type 7 is for two types of data: floating-point numbers and 742 "simple values" that do not need any content. Each value of the 743 5-bit additional information in the initial byte has its own separate 744 meaning, as defined in Table 2. Like the major types for integers, 745 items of this major type do not carry content data; all the 746 information is in the initial bytes. 748 +------------+------------------------------------------------------+ 749 | 5-Bit | Semantics | 750 | Value | | 751 +------------+------------------------------------------------------+ 752 | 0..23 | Simple value (value 0..23) | 753 | | | 754 | 24 | Simple value (value 32..255 in following byte) | 755 | | | 756 | 25 | IEEE 754 Half-Precision Float (16 bits follow) | 757 | | | 758 | 26 | IEEE 754 Single-Precision Float (32 bits follow) | 759 | | | 760 | 27 | IEEE 754 Double-Precision Float (64 bits follow) | 761 | | | 762 | 28-30 | Reserved, not well-formed in the present document | 763 | | | 764 | 31 | "break" stop code for indefinite-length items | 765 | | (Section 3.2.1) | 766 +------------+------------------------------------------------------+ 768 Table 2: Values for Additional Information in Major Type 7 770 As with all other major types, the 5-bit value 24 signifies a single- 771 byte extension: it is followed by an additional byte to represent the 772 simple value. (To minimize confusion, only the values 32 to 255 are 773 used.) This maintains the structure of the initial bytes: as for the 774 other major types, the length of these always depends on the 775 additional information in the first byte. Table 3 lists the values 776 assigned and available for simple types. 778 +---------+-----------------+ 779 | Value | Semantics | 780 +---------+-----------------+ 781 | 0..19 | (Unassigned) | 782 | | | 783 | 20 | False | 784 | | | 785 | 21 | True | 786 | | | 787 | 22 | Null | 788 | | | 789 | 23 | Undefined value | 790 | | | 791 | 24..31 | (Reserved) | 792 | | | 793 | 32..255 | (Unassigned) | 794 +---------+-----------------+ 796 Table 3: Simple Values 798 An encoder MUST NOT issue two-byte sequences that start with 0xf8 799 (major type = 7, additional information = 24) and continue with a 800 byte less than 0x20 (32 decimal). Such sequences are not well- 801 formed. (This implies that an encoder cannot encode false, true, 802 null, or undefined in two-byte sequences, only the one-byte variants 803 of these are well-formed.) 805 The 5-bit values of 25, 26, and 27 are for 16-bit, 32-bit, and 64-bit 806 IEEE 754 binary floating-point values [IEEE754]. These floating- 807 point values are encoded in the additional bytes of the appropriate 808 size. (See Appendix D for some information about 16-bit floating 809 point.) 811 3.4. Tagging of Items 813 In CBOR, a data item can be enclosed by a tag to give it additional 814 semantics while retaining its structure. The tag is major type 6, 815 and represents an unsigned integer as indicated by the tag's argument 816 (Section 3); the (sole) enclosed data item is carried as content 817 data. If a tag requires structured data, this structure is encoded 818 into the nested data item. The definition of a tag number usually 819 restricts what kinds of nested data item or items are valid for tags 820 using this tag number. 822 For example, assume that a byte string of length 12 is marked with a 823 tag of number 2 to indicate it is a positive bignum (Section 3.4.3). 824 This would be marked as 0b110_00010 (major type 6, additional 825 information 2 for the tag number) followed by 0b010_01100 (major type 826 2, additional information of 12 for the length) followed by the 12 827 bytes of the bignum. 829 Decoders do not need to understand tags of every tag number, and tags 830 may be of little value in applications where the implementation 831 creating a particular CBOR data item and the implementation decoding 832 that stream know the semantic meaning of each item in the data flow. 833 Their primary purpose in this specification is to define common data 834 types such as dates. A secondary purpose is to provide conversion 835 hints when it is foreseen that the CBOR data item needs to be 836 translated into a different format, requiring hints about the content 837 of items. Understanding the semantics of tags is optional for a 838 decoder; it can just jump over the initial bytes of the tag (that 839 encode the tag number) and interpret the tag content itself, 840 presenting both tag number and tag content to the application. 842 A tag applies semantics to the data item it encloses. Thus, if tag A 843 encloses tag B, which encloses data item C, tag A applies to the 844 result of applying tag B on data item C. That is, a tag is a data 845 item consisting of a tag number and an enclosed value. The content 846 of the tag (the enclosed data item) is the data item (the value) that 847 is being tagged. 849 IANA maintains a registry of tag numbers as described in Section 9.2. 850 Table 4 provides a list of tag numbers that were defined in 851 [RFC7049], with definitions in the rest of this section. Note that 852 many other tag numbers have been defined since the publication of 853 [RFC7049]; see the registry described at Section 9.2 for the complete 854 list. 856 +----------+----------+---------------------------------------------+ 857 | Tag | Data | Semantics | 858 | Number | Item | | 859 +----------+----------+---------------------------------------------+ 860 | 0 | text | Standard date/time string; see | 861 | | string | Section 3.4.1 | 862 | | | | 863 | 1 | multiple | Epoch-based date/time; see Section 3.4.2 | 864 | | | | 865 | 2 | byte | Positive bignum; see Section 3.4.3 | 866 | | string | | 867 | | | | 868 | 3 | byte | Negative bignum; see Section 3.4.3 | 869 | | string | | 870 | | | | 871 | 4 | array | Decimal fraction; see Section 3.4.4 | 872 | | | | 873 | 5 | array | Bigfloat; see Section 3.4.4 | 874 | | | | 875 | 21 | multiple | Expected conversion to base64url encoding; | 876 | | | see Section 3.4.5.2 | 877 | | | | 878 | 22 | multiple | Expected conversion to base64 encoding; see | 879 | | | Section 3.4.5.2 | 880 | | | | 881 | 23 | multiple | Expected conversion to base16 encoding; see | 882 | | | Section 3.4.5.2 | 883 | | | | 884 | 24 | byte | Encoded CBOR data item; see Section 3.4.5.1 | 885 | | string | | 886 | | | | 887 | 32 | text | URI; see Section 3.4.5.3 | 888 | | string | | 889 | | | | 890 | 33 | text | base64url; see Section 3.4.5.3 | 891 | | string | | 892 | | | | 893 | 34 | text | base64; see Section 3.4.5.3 | 894 | | string | | 895 | | | | 896 | 35 | text | Regular expression; see Section 3.4.5.3 | 897 | | string | | 898 | | | | 899 | 36 | text | MIME message; see Section 3.4.5.3 | 900 | | string | | 901 | | | | 902 | 55799 | multiple | Self-described CBOR; see Section 3.4.6 | 903 +----------+----------+---------------------------------------------+ 905 Table 4: Tag numbers defined in RFC 7049 907 Conceptually, tags are interpreted in the generic data model, not at 908 (de-)serialization time. A small number of tags (specifically, tag 909 number 25 and tag number 29) have been registered with semantics that 910 may require processing at (de-)serialization time: The decoder needs 911 to be aware and the encoder needs to be in control of the exact 912 sequence in which data items are encoded into the CBOR data stream. 913 This means these tags cannot be implemented on top of every generic 914 CBOR encoder/decoder (which might not reflect the serialization order 915 for entries in a map at the data model level and vice versa); their 916 implementation therefore typically needs to be integrated into the 917 generic encoder/decoder. The definition of new tags with this 918 property is NOT RECOMMENDED. 920 Protocols using tag numbers 0 and 1 extend the generic data model 921 (Section 2) with data items representing points in time; tag numbers 922 2 and 3, with arbitrarily sized integers; and tag numbers 4 and 5, 923 with floating point values of arbitrary size and precision. 925 3.4.1. Standard Date/Time String 927 Tag number 0 contains a text string in the standard format described 928 by the "date-time" production in [RFC3339], as refined by Section 3.3 929 of [RFC4287], representing the point in time described there. A 930 nested item of another type or that doesn't match the [RFC4287] 931 format is invalid. 933 3.4.2. Epoch-based Date/Time 935 Tag number 1 contains a numerical value counting the number of 936 seconds from 1970-01-01T00:00Z in UTC time to the represented point 937 in civil time. 939 The enclosed item MUST be an unsigned or negative integer (major 940 types 0 and 1), or a floating-point number (major type 7 with 941 additional information 25, 26, or 27). Other contained types are 942 invalid. 944 Non-negative values (major type 0 and non-negative floating-point 945 numbers) stand for time values on or after 1970-01-01T00:00Z UTC and 946 are interpreted according to POSIX [TIME_T]. (POSIX time is also 947 known as UNIX Epoch time. Note that leap seconds are handled 948 specially by POSIX time and this results in a 1 second discontinuity 949 several times per decade.) Note that applications that require the 950 expression of times beyond early 2106 cannot leave out support of 951 64-bit integers for the enclosed value. 953 Negative values (major type 1 and negative floating-point numbers) 954 are interpreted as determined by the application requirements as 955 there is no universal standard for UTC count-of-seconds time before 956 1970-01-01T00:00Z (this is particularly true for points in time that 957 precede discontinuities in national calendars). The same applies to 958 non-finite values. 960 To indicate fractional seconds, floating-point values can be used 961 within tag number 1 instead of integer values. Note that this 962 generally requires binary64 support, as binary16 and binary32 provide 963 non-zero fractions of seconds only for a short period of time around 964 early 1970. An application that requires tag number 1 support may 965 restrict the enclosed value to be an integer (or a floating-point 966 value) only. 968 3.4.3. Bignums 970 Protocols using tag numbers 2 and 3 extend the generic data model 971 (Section 2) with "bignums" representing arbitrarily sized integers. 972 In the generic data model, bignum values are not equal to integers 973 from the basic data model, but specific data models can define that 974 equivalence, and preferred encoding never makes use of bignums that 975 also can be expressed as basic integers (see below). 977 Bignums are encoded as a byte string data item, which is interpreted 978 as an unsigned integer n in network byte order. Contained items of 979 other types are invalid. For tag number 2, the value of the bignum 980 is n. For tag number 3, the value of the bignum is -1 - n. The 981 preferred encoding of the byte string is to leave out any leading 982 zeroes (note that this means the preferred encoding for n = 0 is the 983 empty byte string, but see below). Decoders that understand these 984 tags MUST be able to decode bignums that do have leading zeroes. The 985 preferred encoding of an integer that can be represented using major 986 type 0 or 1 is to encode it this way instead of as a bignum (which 987 means that the empty string never occurs in a bignum when using 988 preferred encoding). Note that this means the non-preferred choice 989 of a bignum representation instead of a basic integer for encoding a 990 number is not intended to have application semantics (just as the 991 choice of a longer basic integer representation than needed, such as 992 0x1800 for 0x00 does not). 994 For example, the number 18446744073709551616 (2**64) is represented 995 as 0b110_00010 (major type 6, tag number 2), followed by 0b010_01001 996 (major type 2, length 9), followed by 0x010000000000000000 (one byte 997 0x01 and eight bytes 0x00). In hexadecimal: 999 C2 -- Tag 2 1000 49 -- Byte string of length 9 1001 010000000000000000 -- Bytes content 1003 3.4.4. Decimal Fractions and Bigfloats 1005 Protocols using tag number 4 extend the generic data model with data 1006 items representing arbitrary-length decimal fractions of the form 1007 m*(10**e). Protocols using tag number 5 extend the generic data 1008 model with data items representing arbitrary-length binary fractions 1009 of the form m*(2**e). As with bignums, values of different types are 1010 not equal in the generic data model. 1012 Decimal fractions combine an integer mantissa with a base-10 scaling 1013 factor. They are most useful if an application needs the exact 1014 representation of a decimal fraction such as 1.1 because there is no 1015 exact representation for many decimal fractions in binary floating 1016 point. 1018 Bigfloats combine an integer mantissa with a base-2 scaling factor. 1019 They are binary floating-point values that can exceed the range or 1020 the precision of the three IEEE 754 formats supported by CBOR 1021 (Section 3.3). Bigfloats may also be used by constrained 1022 applications that need some basic binary floating-point capability 1023 without the need for supporting IEEE 754. 1025 A decimal fraction or a bigfloat is represented as a tagged array 1026 that contains exactly two integer numbers: an exponent e and a 1027 mantissa m. Decimal fractions (tag number 4) use base-10 exponents; 1028 the value of a decimal fraction data item is m*(10**e). Bigfloats 1029 (tag number 5) use base-2 exponents; the value of a bigfloat data 1030 item is m*(2**e). The exponent e MUST be represented in an integer 1031 of major type 0 or 1, while the mantissa also can be a bignum 1032 (Section 3.4.3). Contained items with other structures are invalid. 1034 An example of a decimal fraction is that the number 273.15 could be 1035 represented as 0b110_00100 (major type of 6 for the tag, additional 1036 information of 4 for the number of tag), followed by 0b100_00010 1037 (major type of 4 for the array, additional information of 2 for the 1038 length of the array), followed by 0b001_00001 (major type of 1 for 1039 the first integer, additional information of 1 for the value of -2), 1040 followed by 0b000_11001 (major type of 0 for the second integer, 1041 additional information of 25 for a two-byte value), followed by 1042 0b0110101010110011 (27315 in two bytes). In hexadecimal: 1044 C4 -- Tag 4 1045 82 -- Array of length 2 1046 21 -- -2 1047 19 6ab3 -- 27315 1049 An example of a bigfloat is that the number 1.5 could be represented 1050 as 0b110_00101 (major type of 6 for the tag, additional information 1051 of 5 for the number of tag), followed by 0b100_00010 (major type of 4 1052 for the array, additional information of 2 for the length of the 1053 array), followed by 0b001_00000 (major type of 1 for the first 1054 integer, additional information of 0 for the value of -1), followed 1055 by 0b000_00011 (major type of 0 for the second integer, additional 1056 information of 3 for the value of 3). In hexadecimal: 1058 C5 -- Tag 5 1059 82 -- Array of length 2 1060 20 -- -1 1061 03 -- 3 1063 Decimal fractions and bigfloats provide no representation of 1064 Infinity, -Infinity, or NaN; if these are needed in place of a 1065 decimal fraction or bigfloat, the IEEE 754 half-precision 1066 representations from Section 3.3 can be used. For constrained 1067 applications, where there is a choice between representing a specific 1068 number as an integer and as a decimal fraction or bigfloat (such as 1069 when the exponent is small and non-negative), there is a quality-of- 1070 implementation expectation that the integer representation is used 1071 directly. 1073 3.4.5. Content Hints 1075 The tags in this section are for content hints that might be used by 1076 generic CBOR processors. These content hints do not extend the 1077 generic data model. 1079 3.4.5.1. Encoded CBOR Data Item 1081 Sometimes it is beneficial to carry an embedded CBOR data item that 1082 is not meant to be decoded immediately at the time the enclosing data 1083 item is being decoded. Tag number 24 (CBOR data item) can be used to 1084 tag the embedded byte string as a data item encoded in CBOR format. 1085 Contained items that aren't byte strings are invalid. A contained 1086 byte string is valid if it encodes a well-formed CBOR item; validity 1087 checking of the decoded CBOR item is not required for tag validity 1088 (but could be offered by a generic decoder as a special option). 1090 3.4.5.2. Expected Later Encoding for CBOR-to-JSON Converters 1092 Tags number 21 to 23 indicate that a byte string might require a 1093 specific encoding when interoperating with a text-based 1094 representation. These tags are useful when an encoder knows that the 1095 byte string data it is writing is likely to be later converted to a 1096 particular JSON-based usage. That usage specifies that some strings 1097 are encoded as base64, base64url, and so on. The encoder uses byte 1098 strings instead of doing the encoding itself to reduce the message 1099 size, to reduce the code size of the encoder, or both. The encoder 1100 does not know whether or not the converter will be generic, and 1101 therefore wants to say what it believes is the proper way to convert 1102 binary strings to JSON. 1104 The data item tagged can be a byte string or any other data item. In 1105 the latter case, the tag applies to all of the byte string data items 1106 contained in the data item, except for those contained in a nested 1107 data item tagged with an expected conversion. 1109 These three tag numbers suggest conversions to three of the base data 1110 encodings defined in [RFC4648]. For base64url encoding (tag number 1111 21), padding is not used (see Section 3.2 of RFC 4648); that is, all 1112 trailing equals signs ("=") are removed from the encoded string. For 1113 base64 encoding (tag number 22), padding is used as defined in RFC 1114 4648. For both base64url and base64, padding bits are set to zero 1115 (see Section 3.5 of RFC 4648), and encoding is performed without the 1116 inclusion of any line breaks, whitespace, or other additional 1117 characters. Note that, for all three tag numbers, the encoding of 1118 the empty byte string is the empty text string. 1120 3.4.5.3. Encoded Text 1122 Some text strings hold data that have formats widely used on the 1123 Internet, and sometimes those formats can be validated and presented 1124 to the application in appropriate form by the decoder. There are 1125 tags for some of these formats. As with tag numbers 21 to 23, if 1126 these tags are applied to an item other than a text string, they 1127 apply to all text string data items it contains. 1129 * Tag number 32 is for URIs, as defined in [RFC3986]. If the text 1130 string doesn't match the "URI-reference" production, the string is 1131 invalid. 1133 * Tag numbers 33 and 34 are for base64url- and base64-encoded text 1134 strings, as defined in [RFC4648]. If any of: 1136 - the encoded text string contains non-alphabet characters or 1137 only 1 character in the last block of 4, or 1139 - the padding bits in a 2- or 3-character block are not 0, or 1141 - the base64 encoding has the wrong number of padding characters, 1142 or 1144 - the base64url encoding has padding characters, 1146 the string is invalid. 1148 * Tag number 35 is for regular expressions that are roughly in Perl 1149 Compatible Regular Expressions (PCRE/PCRE2) form [PCRE] or a 1150 version of the JavaScript regular expression syntax [ECMA262]. 1151 (Note that more specific identification may be necessary if the 1152 actual version of the specification underlying the regular 1153 expression, or more than just the text of the regular expression 1154 itself, need to be conveyed.) Any contained string value is 1155 valid. 1157 * Tag number 36 is for MIME messages (including all headers), as 1158 defined in [RFC2045]. A text string that isn't a valid MIME 1159 message is invalid. (For this tag, validity checking may be 1160 particularly onerous for a generic decoder and might therefore not 1161 be offered. Note that many MIME messages are general binary data 1162 and can therefore not be represented in a text string; 1163 [IANA.cbor-tags] lists a registration for tag number 257 that is 1164 similar to tag number 36 but is used with an enclosed byte 1165 string.) 1167 Note that tag numbers 33 and 34 differ from 21 and 22 in that the 1168 data is transported in base-encoded form for the former and in raw 1169 byte string form for the latter. 1171 3.4.6. Self-Described CBOR 1173 In many applications, it will be clear from the context that CBOR is 1174 being employed for encoding a data item. For instance, a specific 1175 protocol might specify the use of CBOR, or a media type is indicated 1176 that specifies its use. However, there may be applications where 1177 such context information is not available, such as when CBOR data is 1178 stored in a file that does not have disambiguating metadata. Here, 1179 it may help to have some distinguishing characteristics for the data 1180 itself. 1182 Tag number 55799 is defined for this purpose. It does not impart any 1183 special semantics on the data item that it encloses; that is, the 1184 semantics of a data item enclosed in tag number 55799 is exactly 1185 identical to the semantics of the data item itself. 1187 The serialization of this tag's head is 0xd9d9f7, which does not 1188 appear to be in use as a distinguishing mark for any frequently used 1189 file types. In particular, 0xd9d9f7 is not a valid start of a 1190 Unicode text in any Unicode encoding if it is followed by a valid 1191 CBOR data item. 1193 For instance, a decoder might be able to decode both CBOR and JSON. 1194 Such a decoder would need to mechanically distinguish the two 1195 formats. An easy way for an encoder to help the decoder would be to 1196 tag the entire CBOR item with tag number 55799, the serialization of 1197 which will never be found at the beginning of a JSON text. 1199 4. Serialization Considerations 1201 4.1. Preferred Serialization 1203 For some values at the data model level, CBOR provides multiple 1204 serializations. For many applications, it is desirable that an 1205 encoder always chooses a preferred serialization (preferred 1206 encoding); however, the present specification does not put the burden 1207 of enforcing this preference on either encoder or decoder. 1209 Some constrained decoders may be limited in their ability to decode 1210 non-preferred serializations: For example, if only integers below 1211 1_000_000_000 are expected in an application, the decoder may leave 1212 out the code that would be needed to decode 64-bit arguments in 1213 integers. An encoder that always uses preferred serialization 1214 ("preferred encoder") interoperates with this decoder for the numbers 1215 that can occur in this application. More generally speaking, it 1216 therefore can be said that a preferred encoder is more universally 1217 interoperable (and also less wasteful) than one that, say, always 1218 uses 64-bit integers. 1220 Similarly, a constrained encoder may be limited in the variety of 1221 representation variants it supports in such a way that it does not 1222 emit preferred serializations ("variant encoder"): Say, it could be 1223 designed to always use the 32-bit variant for an integer that it 1224 encodes even if a short representation is available (again, assuming 1225 that there is no application need for integers that can only be 1226 represented with the 64-bit variant). A decoder that does not rely 1227 on only ever receiving preferred serializations ("variation-tolerant 1228 decoder") can there be said to be more universally interoperable (it 1229 might very well optimize for the case of receiving preferred 1230 serializations, though). Full implementations of CBOR decoders are 1231 by definition variation-tolerant; the distinction is only relevant if 1232 a constrained implementation of a CBOR decoder meets a variant 1233 encoder. 1235 The preferred serialization always uses the shortest form of 1236 representing the argument (Section 3)); it also uses the shortest 1237 floating-point encoding that preserves the value being encoded (see 1238 Section 5.5). Definite length encoding is preferred whenever the 1239 length is known at the time the serialization of the item starts. 1241 4.2. Deterministically Encoded CBOR 1243 Some protocols may want encoders to only emit CBOR in a particular 1244 deterministic format; those protocols might also have the decoders 1245 check that their input is in that deterministic format. Those 1246 protocols are free to define what they mean by a "deterministic 1247 format" and what encoders and decoders are expected to do. This 1248 section defines a set of restrictions that can serve as the base of 1249 such a deterministic format. 1251 4.2.1. Core Deterministic Encoding Requirements 1253 A CBOR encoding satisfies the "core deterministic encoding 1254 requirements" if it satisfies the following restrictions: 1256 * Preferred serialization MUST be used. In particular, this means 1257 that arguments (see Section 3) for integers, lengths in major 1258 types 2 through 5, and tags MUST be as short as possible, for 1259 instance: 1261 - 0 to 23 and -1 to -24 MUST be expressed in the same byte as the 1262 major type; 1264 - 24 to 255 and -25 to -256 MUST be expressed only with an 1265 additional uint8_t; 1267 - 256 to 65535 and -257 to -65536 MUST be expressed only with an 1268 additional uint16_t; 1270 - 65536 to 4294967295 and -65537 to -4294967296 MUST be expressed 1271 only with an additional uint32_t. 1273 Floating point values also MUST use the shortest form that 1274 preserves the value, e.g. 1.5 is encoded as 0xf93e00 and 1000000.5 1275 as 0xfa49742408. 1277 * Indefinite-length items MUST NOT appear. They can be encoded as 1278 definite-length items instead. 1280 * The keys in every map MUST be sorted in the bytewise lexicographic 1281 order of their deterministic encodings. For example, the 1282 following keys are sorted correctly: 1284 1. 10, encoded as 0x0a. 1286 2. 100, encoded as 0x1864. 1288 3. -1, encoded as 0x20. 1290 4. "z", encoded as 0x617a. 1292 5. "aa", encoded as 0x626161. 1294 6. [100], encoded as 0x811864. 1296 7. [-1], encoded as 0x8120. 1298 8. false, encoded as 0xf4. 1300 4.2.2. Additional Deterministic Encoding Considerations 1302 If a protocol allows for IEEE floats, then additional deterministic 1303 encoding rules might need to be added. One example rule might be to 1304 have all floats start as a 64-bit float, then do a test conversion to 1305 a 32-bit float; if the result is the same numeric value, use the 1306 shorter value and repeat the process with a test conversion to a 1307 16-bit float. (This rule selects 16-bit float for positive and 1308 negative Infinity as well.) Although IEEE floats can represent both 1309 positive and negative zero as distinct values, the application might 1310 not distinguish these and might decide to represent all zero values 1311 with a positive sign, disallowing negative zero. 1313 CBOR tags present additional considerations for deterministic 1314 encoding. If a CBOR-based protocol were to provide the same 1315 semantics for the presence and absence of a specific tag (e.g., by 1316 allowing both tag 1 data items and raw numbers in a date/time 1317 position, treating the latter as if they were tagged), the 1318 deterministic format would not allow them. In a protocol that 1319 requires tags in certain places to obtain specific semantics, the tag 1320 needs to appear in the deterministic format as well. Deterministic 1321 encoding considerations also apply to the content of tags. 1323 Protocols that include floating, big integer, or other complex values 1324 need to define extra requirements on their deterministic encodings. 1325 For example: 1327 * If a protocol includes a field that can express floating-point 1328 values (Section 3.3), the protocol's deterministic encoding needs 1329 to specify whether the integer 1.0 is encoded as 0x01, 0xf93c00, 1330 0xfa3f800000, or 0xfb3ff0000000000000. Three sensible rules for 1331 this are: 1333 1. Encode integral values that fit in 64 bits as values from 1334 major types 0 and 1, and other values as the smallest of 16-, 1335 32-, or 64-bit floating point that accurately represents the 1336 value, 1338 2. Encode all values as the smallest of 16-, 32-, or 64-bit 1339 floating point that accurately represents the value, even for 1340 integral values, or 1342 3. Encode all values as 64-bit floating point. 1344 Rule 1 straddles the boundaries between integers and floating 1345 point values, and Rule 3 does not use preferred encoding, so Rule 1346 2 may be a good choice in many cases. 1348 If NaN is an allowed value and there is no intent to support NaN 1349 payloads or signaling NaNs, the protocol needs to pick a single 1350 representation, for example 0xf97e00. If that simple choice is 1351 not possible, specific attention will be needed for NaN handling. 1353 Subnormal numbers (nonzero numbers with the lowest possible 1354 exponent of a given IEEE 754 number format) may be flushed to zero 1355 outputs or be treated as zero inputs in some floating point 1356 implementations. A protocol's deterministic encoding may want to 1357 exclude them from interchange, interchanging zero instead. 1359 * If a protocol includes a field that can express integers with an 1360 absolute value of 2^64 or larger using tag numbers 2 or 3 1361 (Section 3.4.3), the protocol's deterministic encoding needs to 1362 specify whether small integers are expressed using the tag or 1363 major types 0 and 1. 1365 * A protocol might give encoders the choice of representing a URL as 1366 either a text string or, using Section 3.4.5.3, tag number 32 1367 containing a text string. This protocol's deterministic encoding 1368 needs to either require that the tag is present or require that 1369 it's absent, not allow either one. 1371 4.2.3. Length-first map key ordering 1373 The core deterministic encoding requirements sort map keys in a 1374 different order from the one suggested by Section 3.9 of [RFC7049] 1375 (called "Canonical CBOR" there). Protocols that need to be 1376 compatible with [RFC7049]'s order can instead be specified in terms 1377 of this specification's "length-first core deterministic encoding 1378 requirements": 1380 A CBOR encoding satisfies the "length-first core deterministic 1381 encoding requirements" if it satisfies the core deterministic 1382 encoding requirements except that the keys in every map MUST be 1383 sorted such that: 1385 1. If two keys have different lengths, the shorter one sorts 1386 earlier; 1388 2. If two keys have the same length, the one with the lower value in 1389 (byte-wise) lexical order sorts earlier. 1391 For example, under the length-first core deterministic encoding 1392 requirements, the following keys are sorted correctly: 1394 1. 10, encoded as 0x0a. 1396 2. -1, encoded as 0x20. 1398 3. false, encoded as 0xf4. 1400 4. 100, encoded as 0x1864. 1402 5. "z", encoded as 0x617a. 1404 6. [-1], encoded as 0x8120. 1406 7. "aa", encoded as 0x626161. 1408 8. [100], encoded as 0x811864. 1410 (Although [RFC7049] used the term "Canonical CBOR" for its form of 1411 requirements on deterministic encoding, this document avoids this 1412 term because "canonicalization" is often associated with specific 1413 uses of deterministic encoding only. The terms are essentially 1414 interchangeable, however, and the set of core requirements in this 1415 document could also be called "Canonical CBOR", while the length- 1416 first-ordered version of that could be called "Old Canonical CBOR".) 1418 5. Creating CBOR-Based Protocols 1420 Data formats such as CBOR are often used in environments where there 1421 is no format negotiation. A specific design goal of CBOR is to not 1422 need any included or assumed schema: a decoder can take a CBOR item 1423 and decode it with no other knowledge. 1425 Of course, in real-world implementations, the encoder and the decoder 1426 will have a shared view of what should be in a CBOR data item. For 1427 example, an agreed-to format might be "the item is an array whose 1428 first value is a UTF-8 string, second value is an integer, and 1429 subsequent values are zero or more floating-point numbers" or "the 1430 item is a map that has byte strings for keys and contains at least 1431 one pair whose key is 0xab01". 1433 CBOR-based protocols MUST specify how their decoders handle invalid 1434 and other unexpected data. CBOR-based protocols MAY specify that 1435 they treat arbitrary valid data as unexpected. Encoders for CBOR- 1436 based protocols MUST produce only valid items, that is, the protocol 1437 cannot be designed to make use of invalid items. An encoder can be 1438 capable of encoding as many or as few types of values as is required 1439 by the protocol in which it is used; a decoder can be capable of 1440 understanding as many or as few types of values as is required by the 1441 protocols in which it is used. This lack of restrictions allows CBOR 1442 to be used in extremely constrained environments. 1444 This section discusses some considerations in creating CBOR-based 1445 protocols. With few exceptions, it is advisory only and explicitly 1446 excludes any language from BCP 14 other than words that could be 1447 interpreted as "MAY" in the sense of BCP 14. The exceptions aim at 1448 facilitating interoperability of CBOR-based protocols while making 1449 use of a wide variety of both generic and application-specific 1450 encoders and decoders. 1452 5.1. CBOR in Streaming Applications 1454 In a streaming application, a data stream may be composed of a 1455 sequence of CBOR data items concatenated back-to-back. In such an 1456 environment, the decoder immediately begins decoding a new data item 1457 if data is found after the end of a previous data item. 1459 Not all of the bytes making up a data item may be immediately 1460 available to the decoder; some decoders will buffer additional data 1461 until a complete data item can be presented to the application. 1462 Other decoders can present partial information about a top-level data 1463 item to an application, such as the nested data items that could 1464 already be decoded, or even parts of a byte string that hasn't 1465 completely arrived yet. 1467 Note that some applications and protocols will not want to use 1468 indefinite-length encoding. Using indefinite-length encoding allows 1469 an encoder to not need to marshal all the data for counting, but it 1470 requires a decoder to allocate increasing amounts of memory while 1471 waiting for the end of the item. This might be fine for some 1472 applications but not others. 1474 5.2. Generic Encoders and Decoders 1476 A generic CBOR decoder can decode all well-formed CBOR data and 1477 present them to an application. See Appendix C. 1479 Even though CBOR attempts to minimize these cases, not all well- 1480 formed CBOR data is valid: for example, the encoded text string 1481 "0x62c0ae" does not contain valid UTF-8 and so is not a valid CBOR 1482 item. Also, specific tags may make semantic constraints that may be 1483 violated, such as a bignum tag enclosing another tag, or an instance 1484 of tag number 0 containing a byte string, or containing a text string 1485 with contents that do not match [RFC3339]'s "date-time" production. 1486 There is no requirement that generic encoders and decoders make 1487 unnatural choices for their application interface to enable the 1488 processing of invalid data. Generic encoders and decoders are 1489 expected to forward simple values and tags even if their specific 1490 codepoints are not registered at the time the encoder/decoder is 1491 written (Section 5.4). 1493 Generic decoders provide ways to present well-formed CBOR values, 1494 both valid and invalid, to an application. The diagnostic notation 1495 (Section 8) may be used to present well-formed CBOR values to humans. 1497 Generic encoders provide an application interface that allows the 1498 application to specify any well-formed value, including simple values 1499 and tags unknown to the encoder. 1501 5.3. Validity of Items 1503 A well-formed but invalid CBOR data item presents a problem with 1504 interpreting the data encoded in it in the CBOR data model. A CBOR- 1505 based protocol could be specified in several layers, in which the 1506 lower layers don't process the semantics of some of the CBOR data 1507 they forward. These layers can't notice any validity errors in data 1508 they don't process and MUST forward that data as-is. The first layer 1509 that does process the semantics of an invalid CBOR item MUST take one 1510 of two choices: 1512 1. Replace the problematic item with an error marker and continue 1513 with the next item, or 1515 2. Issue an error and stop processing altogether. 1517 A CBOR-based protocol MUST specify which of these options its 1518 decoders take, for each kind of invalid item they might encounter. 1520 Such problems might occur at the basic validity level of CBOR or in 1521 the context of tags (tag validity). 1523 5.3.1. Basic validity 1525 Two kinds of validity errors can occur in the basic generic data 1526 model: 1528 Duplicate keys in a map: Generic decoders (Section 5.2) make data 1529 available to applications using the native CBOR data model. That 1530 data model includes maps (key-value mappings with unique keys), 1531 not multimaps (key-value mappings where multiple entries can have 1532 the same key). Thus, a generic decoder that gets a CBOR map item 1533 that has duplicate keys will decode to a map with only one 1534 instance of that key, or it might stop processing altogether. On 1535 the other hand, a "streaming decoder" may not even be able to 1536 notice (Section 5.6). 1538 Invalid UTF-8 string: A decoder might or might not want to verify 1539 that the sequence of bytes in a UTF-8 string (major type 3) is 1540 actually valid UTF-8 and react appropriately. 1542 5.3.2. Tag validity 1544 Two additional kinds of validity errors are introduced by adding tags 1545 to the basic generic data model: 1547 Inadmissible type for tag content: Tags (Section 3.4) specify what 1548 type of data item is supposed to be enclosed by the tag; for 1549 example, the tags for positive or negative bignums are supposed to 1550 be put on byte strings. A decoder that decodes the tagged data 1551 item into a native representation (a native big integer in this 1552 example) is expected to check the type of the data item being 1553 tagged. Even decoders that don't have such native representations 1554 available in their environment may perform the check on those tags 1555 known to them and react appropriately. 1557 Inadmissible value for tag content: The type of data item may be 1558 admissible for a tag's content, but the specific value may not be; 1559 e.g., a value of "yesterday" is not acceptable for the content of 1560 tag 0, even though it properly is a text string. A decoder that 1561 normally ingests such tags into equivalent platform types might 1562 present this tag to the application in a similar way to how it 1563 would present a tag with an unknown tag number (Section 5.4). 1565 5.4. Validity and Evolution 1567 A decoder with validity checking will expend the effort to reliably 1568 detect data items with validity errors. For example, such a decoder 1569 needs to have an API that reports an error (and does not return data) 1570 for a CBOR data item that contains any of the validity errors listed 1571 in the previous subsection. 1573 The set of tags defined in the tag registry (Section 9.2), as well as 1574 the set of simple values defined in the simple values registry 1575 (Section 9.1), can grow at any time beyond the set understood by a 1576 generic decoder. A validity-checking decoder can do one of two 1577 things when it encounters such a case that it does not recognize: 1579 * It can report an error (and not return data). Note that this 1580 error is not a validity error per se. This kind of error is more 1581 likely to be raised by a decoder that would be performing validity 1582 checking if this were a known case. 1584 * It can emit the unknown item (type, value, and, for tags, the 1585 decoded tagged data item) to the application calling the decoder, 1586 with an indication that the decoder did not recognize that tag 1587 number or simple value. 1589 The latter approach, which is also appropriate for decoders that do 1590 not support validity checking, provides forward compatibility with 1591 newly registered tags and simple values without the requirement to 1592 update the encoder at the same time as the calling application. (For 1593 this, the API for the decoder needs to have a way to mark unknown 1594 items so that the calling application can handle them in a manner 1595 appropriate for the program.) 1597 Since some of the processing needed for validity checking may have an 1598 appreciable cost (in particular with duplicate detection for maps), 1599 support of validity checking is not a requirement placed on all CBOR 1600 decoders. 1602 Some encoders will rely on their applications to provide input data 1603 in such a way that valid CBOR results from the encoder. A generic 1604 encoder also may want to provide a validity-checking mode where it 1605 reliably limits its output to valid CBOR, independent of whether or 1606 not its application is indeed providing API-conformant data. 1608 5.5. Numbers 1610 CBOR-based protocols should take into account that different language 1611 environments pose different restrictions on the range and precision 1612 of numbers that are representable. For example, the JavaScript 1613 number system treats all numbers as floating point, which may result 1614 in silent loss of precision in decoding integers with more than 53 1615 significant bits. A protocol that uses numbers should define its 1616 expectations on the handling of non-trivial numbers in decoders and 1617 receiving applications. 1619 A CBOR-based protocol that includes floating-point numbers can 1620 restrict which of the three formats (half-precision, single- 1621 precision, and double-precision) are to be supported. For an 1622 integer-only application, a protocol may want to completely exclude 1623 the use of floating-point values. 1625 A CBOR-based protocol designed for compactness may want to exclude 1626 specific integer encodings that are longer than necessary for the 1627 application, such as to save the need to implement 64-bit integers. 1628 There is an expectation that encoders will use the most compact 1629 integer representation that can represent a given value. However, a 1630 compact application should accept values that use a longer-than- 1631 needed encoding (such as encoding "0" as 0b000_11001 followed by two 1632 bytes of 0x00) as long as the application can decode an integer of 1633 the given size. 1635 The preferred encoding for a floating-point value is the shortest 1636 floating-point encoding that preserves its value, e.g., 0xf94580 for 1637 the number 5.5, and 0xfa45ad9c00 for the number 5555.5, unless the 1638 CBOR-based protocol specifically excludes the use of the shorter 1639 floating-point encodings. For NaN values, a shorter encoding is 1640 preferred if zero-padding the shorter significand towards the right 1641 reconstitutes the original NaN value (for many applications, the 1642 single NaN encoding 0xf97e00 will suffice). 1644 5.6. Specifying Keys for Maps 1646 The encoding and decoding applications need to agree on what types of 1647 keys are going to be used in maps. In applications that need to 1648 interwork with JSON-based applications, keys probably should be 1649 limited to UTF-8 strings only; otherwise, there has to be a specified 1650 mapping from the other CBOR types to Unicode characters, and this 1651 often leads to implementation errors. In applications where keys are 1652 numeric in nature and numeric ordering of keys is important to the 1653 application, directly using the numbers for the keys is useful. 1655 If multiple types of keys are to be used, consideration should be 1656 given to how these types would be represented in the specific 1657 programming environments that are to be used. For example, in 1658 JavaScript Maps [ECMA262], a key of integer 1 cannot be distinguished 1659 from a key of floating-point 1.0. This means that, if integer keys 1660 are used, the protocol needs to avoid use of floating-point keys the 1661 values of which happen to be integer numbers in the same map. 1663 Decoders that deliver data items nested within a CBOR data item 1664 immediately on decoding them ("streaming decoders") often do not keep 1665 the state that is necessary to ascertain uniqueness of a key in a 1666 map. Similarly, an encoder that can start encoding data items before 1667 the enclosing data item is completely available ("streaming encoder") 1668 may want to reduce its overhead significantly by relying on its data 1669 source to maintain uniqueness. 1671 A CBOR-based protocol MUST define what to do when a receiving 1672 application does see multiple identical keys in a map. The resulting 1673 rule in the protocol MUST respect the CBOR data model: it cannot 1674 prescribe a specific handling of the entries with the identical keys, 1675 except that it might have a rule that having identical keys in a map 1676 indicates a malformed map and that the decoder has to stop with an 1677 error. Duplicate keys are also prohibited by CBOR decoders that 1678 enforce validity (Section 5.4). 1680 The CBOR data model for maps does not allow ascribing semantics to 1681 the order of the key/value pairs in the map representation. Thus, a 1682 CBOR-based protocol MUST NOT specify that changing the key/value pair 1683 order in a map would change the semantics, except to specify that 1684 some, orders are disallowed, for example where they would not meet 1685 the requirements of a deterministic encoding (Section 4.2). (Any 1686 secondary effects of map ordering such as on timing, cache usage, and 1687 other potential side channels are not considered part of the 1688 semantics but may be enough reason on its own for a protocol to 1689 require a deterministic encoding format.) 1691 Applications for constrained devices that have maps where a small 1692 number of frequently used keys can be identified should consider 1693 using small integers as keys; for instance, a set of 24 or fewer 1694 frequent keys can be encoded in a single byte as unsigned integers, 1695 up to 48 if negative integers are also used. Less frequently 1696 occurring keys can then use integers with longer encodings. 1698 5.6.1. Equivalence of Keys 1700 The specific data model applying to a CBOR data item is used to 1701 determine whether keys occurring in maps are duplicates or distinct. 1703 At the generic data model level, numerically equivalent integer and 1704 floating-point values are distinct from each other, as they are from 1705 the various big numbers (Tags 2 to 5). Similarly, text strings are 1706 distinct from byte strings, even if composed of the same bytes. A 1707 tagged value is distinct from an untagged value or from a value 1708 tagged with a different tag number. 1710 Within each of these groups, numeric values are distinct unless they 1711 are numerically equal (specifically, -0.0 is equal to 0.0); for the 1712 purpose of map key equivalence, NaN (not a number) values are 1713 equivalent if they have the same significand after zero-extending 1714 both significands at the right to 64 bits. 1716 (Byte and text) strings are compared byte by byte, arrays element by 1717 element, and are equal if they have the same number of bytes/elements 1718 and the same values at the same positions. Two maps are equal if 1719 they have the same set of pairs regardless of their order; pairs are 1720 equal if both the key and value are equal. 1722 Tagged values are equal if both the tag number and the enclosed item 1723 are equal. (Note that a generic decoder that provides processing for 1724 a specific tag may not be able to distinguish some semantically 1725 equivalent values, e.g. if leading zeroes occur in the content of tag 1726 2/3 (Section 3.4.3).) Simple values are equal if they simply have 1727 the same value. Nothing else is equal in the generic data model, a 1728 simple value 2 is not equivalent to an integer 2 and an array is 1729 never equivalent to a map. 1731 As discussed in Section 2.2, specific data models can make values 1732 equivalent for the purpose of comparing map keys that are distinct in 1733 the generic data model. Note that this implies that a generic 1734 decoder may deliver a decoded map to an application that needs to be 1735 checked for duplicate map keys by that application (alternatively, 1736 the decoder may provide a programming interface to perform this 1737 service for the application). Specific data models cannot 1738 distinguish values for map keys that are equal for this purpose at 1739 the generic data model level. 1741 5.7. Undefined Values 1743 In some CBOR-based protocols, the simple value (Section 3.3) of 1744 Undefined might be used by an encoder as a substitute for a data item 1745 with an encoding problem, in order to allow the rest of the enclosing 1746 data items to be encoded without harm. 1748 6. Converting Data between CBOR and JSON 1750 This section gives non-normative advice about converting between CBOR 1751 and JSON. Implementations of converters are free to use whichever 1752 advice here they want. 1754 It is worth noting that a JSON text is a sequence of characters, not 1755 an encoded sequence of bytes, while a CBOR data item consists of 1756 bytes, not characters. 1758 6.1. Converting from CBOR to JSON 1760 Most of the types in CBOR have direct analogs in JSON. However, some 1761 do not, and someone implementing a CBOR-to-JSON converter has to 1762 consider what to do in those cases. The following non-normative 1763 advice deals with these by converting them to a single substitute 1764 value, such as a JSON null. 1766 * An integer (major type 0 or 1) becomes a JSON number. 1768 * A byte string (major type 2) that is not embedded in a tag that 1769 specifies a proposed encoding is encoded in base64url without 1770 padding and becomes a JSON string. 1772 * A UTF-8 string (major type 3) becomes a JSON string. Note that 1773 JSON requires escaping certain characters ([RFC8259], Section 7): 1774 quotation mark (U+0022), reverse solidus (U+005C), and the "C0 1775 control characters" (U+0000 through U+001F). All other characters 1776 are copied unchanged into the JSON UTF-8 string. 1778 * An array (major type 4) becomes a JSON array. 1780 * A map (major type 5) becomes a JSON object. This is possible 1781 directly only if all keys are UTF-8 strings. A converter might 1782 also convert other keys into UTF-8 strings (such as by converting 1783 integers into strings containing their decimal representation); 1784 however, doing so introduces a danger of key collision. Note also 1785 that, if tags on UTF-8 strings are ignored as proposed below, this 1786 will cause a key collision if the tags are different but the 1787 strings are the same. 1789 * False (major type 7, additional information 20) becomes a JSON 1790 false. 1792 * True (major type 7, additional information 21) becomes a JSON 1793 true. 1795 * Null (major type 7, additional information 22) becomes a JSON 1796 null. 1798 * A floating-point value (major type 7, additional information 25 1799 through 27) becomes a JSON number if it is finite (that is, it can 1800 be represented in a JSON number); if the value is non-finite (NaN, 1801 or positive or negative Infinity), it is represented by the 1802 substitute value. 1804 * Any other simple value (major type 7, any additional information 1805 value not yet discussed) is represented by the substitute value. 1807 * A bignum (major type 6, tag number 2 or 3) is represented by 1808 encoding its byte string in base64url without padding and becomes 1809 a JSON string. For tag number 3 (negative bignum), a "~" (ASCII 1810 tilde) is inserted before the base-encoded value. (The conversion 1811 to a binary blob instead of a number is to prevent a likely 1812 numeric overflow for the JSON decoder.) 1814 * A byte string with an encoding hint (major type 6, tag number 21 1815 through 23) is encoded as described and becomes a JSON string. 1817 * For all other tags (major type 6, any other tag number), the 1818 enclosed CBOR item is represented as a JSON value; the tag number 1819 is ignored. 1821 * Indefinite-length items are made definite before conversion. 1823 6.2. Converting from JSON to CBOR 1825 All JSON values, once decoded, directly map into one or more CBOR 1826 values. As with any kind of CBOR generation, decisions have to be 1827 made with respect to number representation. In a suggested 1828 conversion: 1830 * JSON numbers without fractional parts (integer numbers) are 1831 represented as integers (major types 0 and 1, possibly major type 1832 6 tag number 2 and 3), choosing the shortest form; integers longer 1833 than an implementation-defined threshold may instead be 1834 represented as floating-point values. The default range that is 1835 represented as integer is -2**53+1..2**53-1 (fully exploiting the 1836 range for exact integers in the binary64 representation often used 1837 for decoding JSON [RFC7493]). A CBOR-based protocol, or a generic 1838 converter implementation, may choose -2**32..2**32-1 or 1839 -2**64..2**64-1 (fully using the integer ranges available in CBOR 1840 with uint32_t or uint64_t, respectively) or even -2**31..2**31-1 1841 or -2**63..2**63-1 (using popular ranges for two's complement 1842 signed integers). (If the JSON was generated from a JavaScript 1843 implementation, its precision is already limited to 53 bits 1844 maximum.) 1846 * Numbers with fractional parts are represented as floating-point 1847 values, performing the decimal-to-binary conversion based on the 1848 precision provided by IEEE 754 binary64. Then, when encoding in 1849 CBOR, the preferred serialization uses the shortest floating-point 1850 representation exactly representing this conversion result; for 1851 instance, 1.5 is represented in a 16-bit floating-point value (not 1852 all implementations will be capable of efficiently finding the 1853 minimum form, though). Instead of using the default binary64 1854 precision, there may be an implementation-defined limit to the 1855 precision of the conversion that will affect the precision of the 1856 represented values. Decimal representation should only be used on 1857 the CBOR side if that is specified in a protocol. 1859 CBOR has been designed to generally provide a more compact encoding 1860 than JSON. One implementation strategy that might come to mind is to 1861 perform a JSON-to-CBOR encoding in place in a single buffer. This 1862 strategy would need to carefully consider a number of pathological 1863 cases, such as that some strings represented with no or very few 1864 escapes and longer (or much longer) than 255 bytes may expand when 1865 encoded as UTF-8 strings in CBOR. Similarly, a few of the binary 1866 floating-point representations might cause expansion from some short 1867 decimal representations (1.1, 1e9) in JSON. This may be hard to get 1868 right, and any ensuing vulnerabilities may be exploited by an 1869 attacker. 1871 7. Future Evolution of CBOR 1873 Successful protocols evolve over time. New ideas appear, 1874 implementation platforms improve, related protocols are developed and 1875 evolve, and new requirements from applications and protocols are 1876 added. Facilitating protocol evolution is therefore an important 1877 design consideration for any protocol development. 1879 For protocols that will use CBOR, CBOR provides some useful 1880 mechanisms to facilitate their evolution. Best practices for this 1881 are well known, particularly from JSON format development of JSON- 1882 based protocols. Therefore, such best practices are outside the 1883 scope of this specification. 1885 However, facilitating the evolution of CBOR itself is very well 1886 within its scope. CBOR is designed to both provide a stable basis 1887 for development of CBOR-based protocols and to be able to evolve. 1888 Since a successful protocol may live for decades, CBOR needs to be 1889 designed for decades of use and evolution. This section provides 1890 some guidance for the evolution of CBOR. It is necessarily more 1891 subjective than other parts of this document. It is also necessarily 1892 incomplete, lest it turn into a textbook on protocol development. 1894 7.1. Extension Points 1896 In a protocol design, opportunities for evolution are often included 1897 in the form of extension points. For example, there may be a 1898 codepoint space that is not fully allocated from the outset, and the 1899 protocol is designed to tolerate and embrace implementations that 1900 start using more codepoints than initially allocated. 1902 Sizing the codepoint space may be difficult because the range 1903 required may be hard to predict. An attempt should be made to make 1904 the codepoint space large enough so that it can slowly be filled over 1905 the intended lifetime of the protocol. 1907 CBOR has three major extension points: 1909 * the "simple" space (values in major type 7). Of the 24 efficient 1910 (and 224 slightly less efficient) values, only a small number have 1911 been allocated. Implementations receiving an unknown simple data 1912 item may be able to process it as such, given that the structure 1913 of the value is indeed simple. The IANA registry in Section 9.1 1914 is the appropriate way to address the extensibility of this 1915 codepoint space. 1917 * the "tag" space (values in major type 6). Again, only a small 1918 part of the codepoint space has been allocated, and the space is 1919 abundant (although the early numbers are more efficient than the 1920 later ones). Implementations receiving an unknown tag number can 1921 choose to simply ignore it or to process it as an unknown tag 1922 number wrapping the enclosed data item. The IANA registry in 1923 Section 9.2 is the appropriate way to address the extensibility of 1924 this codepoint space. 1926 * the "additional information" space. An implementation receiving 1927 an unknown additional information value has no way to continue 1928 decoding, so allocating codepoints to this space is a major step. 1929 There are also very few codepoints left. 1931 7.2. Curating the Additional Information Space 1933 The human mind is sometimes drawn to filling in little perceived gaps 1934 to make something neat. We expect the remaining gaps in the 1935 codepoint space for the additional information values to be an 1936 attractor for new ideas, just because they are there. 1938 The present specification does not manage the additional information 1939 codepoint space by an IANA registry. Instead, allocations out of 1940 this space can only be done by updating this specification. 1942 For an additional information value of n >= 24, the size of the 1943 additional data typically is 2**(n-24) bytes. Therefore, additional 1944 information values 28 and 29 should be viewed as candidates for 1945 128-bit and 256-bit quantities, in case a need arises to add them to 1946 the protocol. Additional information value 30 is then the only 1947 additional information value available for general allocation, and 1948 there should be a very good reason for allocating it before assigning 1949 it through an update of this protocol. 1951 8. Diagnostic Notation 1953 CBOR is a binary interchange format. To facilitate documentation and 1954 debugging, and in particular to facilitate communication between 1955 entities cooperating in debugging, this section defines a simple 1956 human-readable diagnostic notation. All actual interchange always 1957 happens in the binary format. 1959 Note that this truly is a diagnostic format; it is not meant to be 1960 parsed. Therefore, no formal definition (as in ABNF) is given in 1961 this document. (Implementers looking for a text-based format for 1962 representing CBOR data items in configuration files may also want to 1963 consider YAML [YAML].) 1965 The diagnostic notation is loosely based on JSON as it is defined in 1966 RFC 8259, extending it where needed. 1968 The notation borrows the JSON syntax for numbers (integer and 1969 floating point), True (>true<), False (>false<), Null (>null<), UTF-8 1970 strings, arrays, and maps (maps are called objects in JSON; the 1971 diagnostic notation extends JSON here by allowing any data item in 1972 the key position). Undefined is written >undefined< as in 1973 JavaScript. The non-finite floating-point numbers Infinity, 1974 -Infinity, and NaN are written exactly as in this sentence (this is 1975 also a way they can be written in JavaScript, although JSON does not 1976 allow them). A tag is written as an integer number for the tag 1977 number, followed by the tag content in parentheses; for instance, an 1978 RFC 3339 (ISO 8601) date could be notated as: 1980 0("2013-03-21T20:04:00Z") 1982 or the equivalent relative time as 1984 1(1363896240) 1986 Byte strings are notated in one of the base encodings, without 1987 padding, enclosed in single quotes, prefixed by >h< for base16, >b32< 1988 for base32, >h32< for base32hex, >b64< for base64 or base64url (the 1989 actual encodings do not overlap, so the string remains unambiguous). 1990 For example, the byte string 0x12345678 could be written h'12345678', 1991 b32'CI2FM6A', or b64'EjRWeA'. 1993 Unassigned simple values are given as "simple()" with the appropriate 1994 integer in the parentheses. For example, "simple(42)" indicates 1995 major type 7, value 42. 1997 8.1. Encoding Indicators 1999 Sometimes it is useful to indicate in the diagnostic notation which 2000 of several alternative representations were actually used; for 2001 example, a data item written >1.5< by a diagnostic decoder might have 2002 been encoded as a half-, single-, or double-precision float. 2004 The convention for encoding indicators is that anything starting with 2005 an underscore and all following characters that are alphanumeric or 2006 underscore, is an encoding indicator, and can be ignored by anyone 2007 not interested in this information. Encoding indicators are always 2008 optional. 2010 A single underscore can be written after the opening brace of a map 2011 or the opening bracket of an array to indicate that the data item was 2012 represented in indefinite-length format. For example, [_ 1, 2] 2013 contains an indicator that an indefinite-length representation was 2014 used to represent the data item [1, 2]. 2016 An underscore followed by a decimal digit n indicates that the 2017 preceding item (or, for arrays and maps, the item starting with the 2018 preceding bracket or brace) was encoded with an additional 2019 information value of 24+n. For example, 1.5_1 is a half-precision 2020 floating-point number, while 1.5_3 is encoded as double precision. 2021 This encoding indicator is not shown in Appendix A. (Note that the 2022 encoding indicator "_" is thus an abbreviation of the full form "_7", 2023 which is not used.) 2025 As a special case, byte and text strings of indefinite length can be 2026 notated in the form (_ h'0123', h'4567') and (_ "foo", "bar"). 2028 9. IANA Considerations 2030 IANA has created two registries for new CBOR values. The registries 2031 are separate, that is, not under an umbrella registry, and follow the 2032 rules in [RFC8126]. IANA has also assigned a new MIME media type and 2033 an associated Constrained Application Protocol (CoAP) Content-Format 2034 entry. 2036 [To be removed by RFC editor:] IANA is requested to update these 2037 registries to point to the present document instead of RFC 7049. 2039 9.1. Simple Values Registry 2041 IANA has created the "Concise Binary Object Representation (CBOR) 2042 Simple Values" registry at [IANA.cbor-simple-values]. The initial 2043 values are shown in Table 3. 2045 New entries in the range 0 to 19 are assigned by Standards Action. 2046 It is suggested that these Standards Actions allocate values starting 2047 with the number 16 in order to reserve the lower numbers for 2048 contiguous blocks (if any). 2050 New entries in the range 32 to 255 are assigned by Specification 2051 Required. 2053 9.2. Tags Registry 2055 IANA has created the "Concise Binary Object Representation (CBOR) 2056 Tags" registry at [IANA.cbor-tags]. The tags that were defined in 2057 [RFC7049] are described in detail in Section 3.4, but other tags have 2058 already been defined. 2060 New entries in the range 0 to 23 are assigned by Standards Action. 2061 New entries in the range 24 to 255 are assigned by Specification 2062 Required. New entries in the range 256 to 18446744073709551615 are 2063 assigned by First Come First Served. The template for registration 2064 requests is: 2066 * Data item 2068 * Semantics (short form) 2070 In addition, First Come First Served requests should include: 2072 * Point of contact 2074 * Description of semantics (URL) - This description is optional; the 2075 URL can point to something like an Internet-Draft or a web page. 2077 9.3. Media Type ("MIME Type") 2079 The Internet media type [RFC6838] for a single encoded CBOR data item 2080 is application/cbor. 2082 Type name: application 2084 Subtype name: cbor 2086 Required parameters: n/a 2088 Optional parameters: n/a 2090 Encoding considerations: binary 2092 Security considerations: See Section 10 of this document 2094 Interoperability considerations: n/a 2096 Published specification: This document 2098 Applications that use this media type: None yet, but it is expected 2099 that this format will be deployed in protocols and applications. 2101 Additional information: 2102 Magic number(s): n/a 2103 File extension(s): .cbor 2104 Macintosh file type code(s): n/a 2106 Person & email address to contact for further information: 2107 Carsten Bormann 2108 cabo@tzi.org 2110 Intended usage: COMMON 2112 Restrictions on usage: none 2114 Author: 2115 Carsten Bormann 2117 Change controller: 2118 The IESG 2120 9.4. CoAP Content-Format 2122 Media Type: application/cbor 2124 Encoding: - 2126 Id: 60 2128 Reference: [RFCthis] 2130 9.5. The +cbor Structured Syntax Suffix Registration 2132 Name: Concise Binary Object Representation (CBOR) 2134 +suffix: +cbor 2136 References: [RFCthis] 2138 Encoding Considerations: CBOR is a binary format. 2140 Interoperability Considerations: n/a 2141 Fragment Identifier Considerations: 2142 The syntax and semantics of fragment identifiers specified for 2143 +cbor SHOULD be as specified for "application/cbor". (At 2144 publication of this document, there is no fragment identification 2145 syntax defined for "application/cbor".) 2147 The syntax and semantics for fragment identifiers for a specific 2148 "xxx/yyy+cbor" SHOULD be processed as follows: 2150 For cases defined in +cbor, where the fragment identifier resolves 2151 per the +cbor rules, then process as specified in +cbor. 2153 For cases defined in +cbor, where the fragment identifier does 2154 not resolve per the +cbor rules, then process as specified in 2155 "xxx/yyy+cbor". 2157 For cases not defined in +cbor, then process as specified in 2158 "xxx/yyy+cbor". 2160 Security Considerations: See Section 10 of this document 2162 Contact: 2163 Apps Area Working Group (apps-discuss@ietf.org) 2165 Author/Change Controller: 2166 The Apps Area Working Group. 2167 The IESG has change control over this registration. 2169 10. Security Considerations 2171 A network-facing application can exhibit vulnerabilities in its 2172 processing logic for incoming data. Complex parsers are well known 2173 as a likely source of such vulnerabilities, such as the ability to 2174 remotely crash a node, or even remotely execute arbitrary code on it. 2175 CBOR attempts to narrow the opportunities for introducing such 2176 vulnerabilities by reducing parser complexity, by giving the entire 2177 range of encodable values a meaning where possible. 2179 Because CBOR decoders are often used as a first step in processing 2180 unvalidated input, they need to be fully prepared for all types of 2181 hostile input that may be designed to corrupt, overrun, or achieve 2182 control of the system decoding the CBOR data item. A CBOR decoder 2183 needs to assume that all input may be hostile even if it has been 2184 checked by a firewall, has come over a secure channel such as TLS, is 2185 encrypted or signed, or has come from some other source that is 2186 presumed trusted. 2188 Hostile input may be constructed to overrun buffers, overflow or 2189 underflow integer arithmetic, or cause other decoding disruption. 2190 CBOR data items might have lengths or sizes that are intentionally 2191 extremely large or too short. Resource exhaustion attacks might 2192 attempt to lure a decoder into allocating very big data items 2193 (strings, arrays, maps, or even arbitrary precision numbers) or 2194 exhaust the stack depth by setting up deeply nested items. Decoders 2195 need to have appropriate resource management to mitigate these 2196 attacks. (Items for which very large sizes are given can also 2197 attempt to exploit integer overflow vulnerabilities.) 2199 A CBOR decoder, by definition, only accepts well-formed CBOR; this is 2200 the first step to its robustness. Input that is not well-formed CBOR 2201 causes no further processing from the point where the lack of well- 2202 formedness was detected. If possible, any data decoded up to this 2203 point should have no impact on the application using the CBOR 2204 decoder. 2206 In addition to ascertaining well-formedness, a CBOR decoder might 2207 also perform validity checks on the CBOR data. Alternatively, it can 2208 leave those checks to the application using the decoder. This choice 2209 needs to be clearly documented in the decoder. Beyond the validity 2210 at the CBOR level, an application also needs to ascertain that the 2211 input is in alignment with the application protocol that is 2212 serialized in CBOR. 2214 The input check itself may consume resources. This is usually linear 2215 in the size of the input, which means that an attacker has to spend 2216 resources that are commensurate to the resources spent by the 2217 defender on input validation. Processing for arbitrary-precision 2218 numbers may exceed linear effort. Also, some hash-table 2219 implementations that are used by decoders to build in-memory 2220 representations of maps can be attacked to spend quadratic effort, 2221 unless a secret key is employed (see Section 7 of [SIPHASH]). Such 2222 superlinear efforts can be employed by an attacker to exhaust 2223 resources at or before the input validator; they therefore need to be 2224 avoided in a CBOR decoder implementation. Note that tag number 2225 definitions and their implementations can add security considerations 2226 of this kind; this should then be discussed in the security 2227 considerations of the tag number definition. 2229 CBOR encoders do not receive input directly from the network and are 2230 thus not directly attackable in the same way as CBOR decoders. 2231 However, CBOR encoders often have an API that takes input from 2232 another level in the implementation and can be attacked through that 2233 API. The design and implementation of that API should assume the 2234 behavior of its caller may be based on hostile input or on coding 2235 mistakes. It should check inputs for buffer overruns, overflow and 2236 underflow of integer arithmetic, and other such errors that are aimed 2237 to disrupt the encoder. 2239 Protocols should be defined in such a way that potential multiple 2240 interpretations are reliably reduced to a single interpretation. For 2241 example, an attacker could make use of invalid input such as 2242 duplicate keys in maps, or exploit different precision in processing 2243 numbers to make one application base its decisions on a different 2244 interpretation than the one that will be used by a second 2245 application. To facilitate consistent interpretation, encoder and 2246 decoder implementations should provide a validity checking mode of 2247 operation (Section 5.4). Note, however, that a generic decoder 2248 cannot know about all requirements that an application poses on its 2249 input data; it is therefore not relieving the application from 2250 performing its own input checking. Also, since the set of defined 2251 tag numbers evolves, the application may employ a tag number that is 2252 not yet supported for validity checking by the generic decoder it 2253 uses. Generic decoders therefore need to provide documentation which 2254 tag numbers they support and what validity checking they can provide 2255 for each of them as well as for basic CBOR validity (UTF-8 checking, 2256 duplicate map key checking). 2258 11. References 2260 11.1. Normative References 2262 [ECMA262] Ecma International, "ECMAScript 2018 Language 2263 Specification", ECMA Standard ECMA-262, 9th Edition, June 2264 2018, . 2268 [IEEE754] IEEE, "IEEE Standard for Floating-Point Arithmetic", IEEE 2269 Std 754-2008. 2271 [RFC2045] Freed, N. and N. Borenstein, "Multipurpose Internet Mail 2272 Extensions (MIME) Part One: Format of Internet Message 2273 Bodies", RFC 2045, DOI 10.17487/RFC2045, November 1996, 2274 . 2276 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 2277 Requirement Levels", BCP 14, RFC 2119, 2278 DOI 10.17487/RFC2119, March 1997, 2279 . 2281 [RFC3339] Klyne, G. and C. Newman, "Date and Time on the Internet: 2282 Timestamps", RFC 3339, DOI 10.17487/RFC3339, July 2002, 2283 . 2285 [RFC3629] Yergeau, F., "UTF-8, a transformation format of ISO 2286 10646", STD 63, RFC 3629, DOI 10.17487/RFC3629, November 2287 2003, . 2289 [RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform 2290 Resource Identifier (URI): Generic Syntax", STD 66, 2291 RFC 3986, DOI 10.17487/RFC3986, January 2005, 2292 . 2294 [RFC4287] Nottingham, M., Ed. and R. Sayre, Ed., "The Atom 2295 Syndication Format", RFC 4287, DOI 10.17487/RFC4287, 2296 December 2005, . 2298 [RFC4648] Josefsson, S., "The Base16, Base32, and Base64 Data 2299 Encodings", RFC 4648, DOI 10.17487/RFC4648, October 2006, 2300 . 2302 [RFC8126] Cotton, M., Leiba, B., and T. Narten, "Guidelines for 2303 Writing an IANA Considerations Section in RFCs", BCP 26, 2304 RFC 8126, DOI 10.17487/RFC8126, June 2017, 2305 . 2307 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2308 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 2309 May 2017, . 2311 [TIME_T] The Open Group Base Specifications, "Vol. 1: Base 2312 Definitions, Issue 7", Section 4.15 'Seconds Since the 2313 Epoch', IEEE Std 1003.1, 2013 Edition, 2013, 2314 . 2317 11.2. Informative References 2319 [ASN.1] International Telecommunication Union, "Information 2320 Technology -- ASN.1 encoding rules: Specification of Basic 2321 Encoding Rules (BER), Canonical Encoding Rules (CER) and 2322 Distinguished Encoding Rules (DER)", ITU-T Recommendation 2323 X.690, 1994. 2325 [BSON] Various, "BSON - Binary JSON", 2013, 2326 . 2328 [I-D.ietf-cbor-sequence] 2329 Bormann, C., "Concise Binary Object Representation (CBOR) 2330 Sequences", draft-ietf-cbor-sequence-02 (work in 2331 progress), September 2019. 2333 [IANA.cbor-simple-values] 2334 IANA, "Concise Binary Object Representation (CBOR) Simple 2335 Values", 2336 . 2338 [IANA.cbor-tags] 2339 IANA, "Concise Binary Object Representation (CBOR) Tags", 2340 . 2342 [MessagePack] 2343 Furuhashi, S., "MessagePack", 2013, . 2345 [PCRE] Ho, A., "PCRE - Perl Compatible Regular Expressions", 2346 2018, . 2348 [RFC0713] Haverty, J., "MSDTP-Message Services Data Transmission 2349 Protocol", RFC 713, DOI 10.17487/RFC0713, April 1976, 2350 . 2352 [RFC6838] Freed, N., Klensin, J., and T. Hansen, "Media Type 2353 Specifications and Registration Procedures", BCP 13, 2354 RFC 6838, DOI 10.17487/RFC6838, January 2013, 2355 . 2357 [RFC7049] Bormann, C. and P. Hoffman, "Concise Binary Object 2358 Representation (CBOR)", RFC 7049, DOI 10.17487/RFC7049, 2359 October 2013, . 2361 [RFC7228] Bormann, C., Ersue, M., and A. Keranen, "Terminology for 2362 Constrained-Node Networks", RFC 7228, 2363 DOI 10.17487/RFC7228, May 2014, 2364 . 2366 [RFC7493] Bray, T., Ed., "The I-JSON Message Format", RFC 7493, 2367 DOI 10.17487/RFC7493, March 2015, 2368 . 2370 [RFC8259] Bray, T., Ed., "The JavaScript Object Notation (JSON) Data 2371 Interchange Format", STD 90, RFC 8259, 2372 DOI 10.17487/RFC8259, December 2017, 2373 . 2375 [RFC8618] Dickinson, J., Hague, J., Dickinson, S., Manderson, T., 2376 and J. Bond, "Compacted-DNS (C-DNS): A Format for DNS 2377 Packet Capture", RFC 8618, DOI 10.17487/RFC8618, September 2378 2019, . 2380 [SIPHASH] Aumasson, J. and D. Bernstein, "SipHash: A Fast Short- 2381 Input PRF", Lecture Notes in Computer Science pp. 489-508, 2382 DOI 10.1007/978-3-642-34931-7_28, 2012. 2384 [YAML] Ben-Kiki, O., Evans, C., and I. Net, "YAML Ain't Markup 2385 Language (YAML[TM]) Version 1.2", 3rd Edition, October 2386 2009, . 2388 Appendix A. Examples 2390 The following table provides some CBOR-encoded values in hexadecimal 2391 (right column), together with diagnostic notation for these values 2392 (left column). Note that the string "\u00fc" is one form of 2393 diagnostic notation for a UTF-8 string containing the single Unicode 2394 character U+00FC, LATIN SMALL LETTER U WITH DIAERESIS (u umlaut). 2395 Similarly, "\u6c34" is a UTF-8 string in diagnostic notation with a 2396 single character U+6C34 (CJK UNIFIED IDEOGRAPH-6C34, often 2397 representing "water"), and "\ud800\udd51" is a UTF-8 string in 2398 diagnostic notation with a single character U+10151 (GREEK ACROPHONIC 2399 ATTIC FIFTY STATERS). (Note that all these single-character strings 2400 could also be represented in native UTF-8 in diagnostic notation, 2401 just not in an ASCII-only specification like the present one.) In 2402 the diagnostic notation provided for bignums, their intended numeric 2403 value is shown as a decimal number (such as 18446744073709551616) 2404 instead of showing a tagged byte string (such as 2405 2(h'010000000000000000')). 2407 +------------------------------+------------------------------------+ 2408 | Diagnostic | Encoded | 2409 +------------------------------+------------------------------------+ 2410 | 0 | 0x00 | 2411 | | | 2412 | 1 | 0x01 | 2413 | | | 2414 | 10 | 0x0a | 2415 | | | 2416 | 23 | 0x17 | 2417 | | | 2418 | 24 | 0x1818 | 2419 | | | 2420 | 25 | 0x1819 | 2421 | | | 2422 | 100 | 0x1864 | 2423 | | | 2424 | 1000 | 0x1903e8 | 2425 | | | 2426 | 1000000 | 0x1a000f4240 | 2427 | | | 2428 | 1000000000000 | 0x1b000000e8d4a51000 | 2429 | | | 2430 | 18446744073709551615 | 0x1bffffffffffffffff | 2431 | | | 2432 | 18446744073709551616 | 0xc249010000000000000000 | 2433 | | | 2434 | -18446744073709551616 | 0x3bffffffffffffffff | 2435 | | | 2436 | -18446744073709551617 | 0xc349010000000000000000 | 2437 | | | 2438 | -1 | 0x20 | 2439 | | | 2440 | -10 | 0x29 | 2441 | | | 2442 | -100 | 0x3863 | 2443 | | | 2444 | -1000 | 0x3903e7 | 2445 | | | 2446 | 0.0 | 0xf90000 | 2447 | | | 2448 | -0.0 | 0xf98000 | 2449 | | | 2450 | 1.0 | 0xf93c00 | 2451 | | | 2452 | 1.1 | 0xfb3ff199999999999a | 2453 | | | 2454 | 1.5 | 0xf93e00 | 2455 | | | 2456 | 65504.0 | 0xf97bff | 2457 | | | 2458 | 100000.0 | 0xfa47c35000 | 2459 | | | 2460 | 3.4028234663852886e+38 | 0xfa7f7fffff | 2461 | | | 2462 | 1.0e+300 | 0xfb7e37e43c8800759c | 2463 | | | 2464 | 5.960464477539063e-8 | 0xf90001 | 2465 | | | 2466 | 0.00006103515625 | 0xf90400 | 2467 | | | 2468 | -4.0 | 0xf9c400 | 2469 | | | 2470 | -4.1 | 0xfbc010666666666666 | 2471 | | | 2472 | Infinity | 0xf97c00 | 2473 | | | 2474 | NaN | 0xf97e00 | 2475 | | | 2476 | -Infinity | 0xf9fc00 | 2477 | | | 2478 | Infinity | 0xfa7f800000 | 2479 | | | 2480 | NaN | 0xfa7fc00000 | 2481 | | | 2482 | -Infinity | 0xfaff800000 | 2483 | | | 2484 | Infinity | 0xfb7ff0000000000000 | 2485 | | | 2486 | NaN | 0xfb7ff8000000000000 | 2487 | | | 2488 | -Infinity | 0xfbfff0000000000000 | 2489 | | | 2490 | false | 0xf4 | 2491 | | | 2492 | true | 0xf5 | 2493 | | | 2494 | null | 0xf6 | 2495 | | | 2496 | undefined | 0xf7 | 2497 | | | 2498 | simple(16) | 0xf0 | 2499 | | | 2500 | simple(255) | 0xf8ff | 2501 | | | 2502 | 0("2013-03-21T20:04:00Z") | 0xc074323031332d30332d32315432303a | 2503 | | 30343a30305a | 2504 | | | 2505 | 1(1363896240) | 0xc11a514b67b0 | 2506 | | | 2507 | 1(1363896240.5) | 0xc1fb41d452d9ec200000 | 2508 | | | 2509 | 23(h'01020304') | 0xd74401020304 | 2510 | | | 2511 | 24(h'6449455446') | 0xd818456449455446 | 2512 | | | 2513 | 32("http://www.example.com") | 0xd82076687474703a2f2f7777772e6578 | 2514 | | 616d706c652e636f6d | 2515 | | | 2516 | h'' | 0x40 | 2517 | | | 2518 | h'01020304' | 0x4401020304 | 2519 | | | 2520 | "" | 0x60 | 2521 | | | 2522 | "a" | 0x6161 | 2523 | | | 2524 | "IETF" | 0x6449455446 | 2525 | | | 2526 | "\"\\" | 0x62225c | 2527 | | | 2528 | "\u00fc" | 0x62c3bc | 2529 | | | 2530 | "\u6c34" | 0x63e6b0b4 | 2531 | | | 2532 | "\ud800\udd51" | 0x64f0908591 | 2533 | | | 2534 | [] | 0x80 | 2535 | | | 2536 | [1, 2, 3] | 0x83010203 | 2537 | | | 2538 | [1, [2, 3], [4, 5]] | 0x8301820203820405 | 2539 | | | 2540 | [1, 2, 3, 4, 5, 6, 7, 8, 9, | 0x98190102030405060708090a0b0c0d0e | 2541 | 10, 11, 12, 13, 14, 15, 16, | 0f101112131415161718181819 | 2542 | 17, 18, 19, 20, 21, 22, 23, | | 2543 | 24, 25] | | 2544 | | | 2545 | {} | 0xa0 | 2546 | | | 2547 | {1: 2, 3: 4} | 0xa201020304 | 2548 | | | 2549 | {"a": 1, "b": [2, 3]} | 0xa26161016162820203 | 2550 | | | 2551 | ["a", {"b": "c"}] | 0x826161a161626163 | 2552 | | | 2553 | {"a": "A", "b": "B", "c": | 0xa5616161416162614261636143616461 | 2554 | "C", "d": "D", "e": "E"} | 4461656145 | 2555 | | | 2556 | (_ h'0102', h'030405') | 0x5f42010243030405ff | 2557 | | | 2558 | (_ "strea", "ming") | 0x7f657374726561646d696e67ff | 2559 | | | 2560 | [_ ] | 0x9fff | 2561 | | | 2562 | [_ 1, [2, 3], [_ 4, 5]] | 0x9f018202039f0405ffff | 2563 | | | 2564 | [_ 1, [2, 3], [4, 5]] | 0x9f01820203820405ff | 2565 | | | 2566 | [1, [2, 3], [_ 4, 5]] | 0x83018202039f0405ff | 2567 | | | 2568 | [1, [_ 2, 3], [4, 5]] | 0x83019f0203ff820405 | 2569 | | | 2570 | [_ 1, 2, 3, 4, 5, 6, 7, 8, | 0x9f0102030405060708090a0b0c0d0e0f | 2571 | 9, 10, 11, 12, 13, 14, 15, | 101112131415161718181819ff | 2572 | 16, 17, 18, 19, 20, 21, 22, | | 2573 | 23, 24, 25] | | 2574 | | | 2575 | {_ "a": 1, "b": [_ 2, 3]} | 0xbf61610161629f0203ffff | 2576 | | | 2577 | ["a", {_ "b": "c"}] | 0x826161bf61626163ff | 2578 | | | 2579 | {_ "Fun": true, "Amt": -2} | 0xbf6346756ef563416d7421ff | 2580 +------------------------------+------------------------------------+ 2582 Table 5: Examples of Encoded CBOR Data Items 2584 Appendix B. Jump Table 2586 For brevity, this jump table does not show initial bytes that are 2587 reserved for future extension. It also only shows a selection of the 2588 initial bytes that can be used for optional features. (All unsigned 2589 integers are in network byte order.) 2591 +------------+------------------------------------------------------+ 2592 | Byte | Structure/Semantics | 2593 +------------+------------------------------------------------------+ 2594 | 0x00..0x17 | Unsigned integer 0x00..0x17 (0..23) | 2595 | | | 2596 | 0x18 | Unsigned integer (one-byte uint8_t follows) | 2597 | | | 2598 | 0x19 | Unsigned integer (two-byte uint16_t follows) | 2599 | | | 2600 | 0x1a | Unsigned integer (four-byte uint32_t follows) | 2601 | | | 2602 | 0x1b | Unsigned integer (eight-byte uint64_t follows) | 2603 | | | 2604 | 0x20..0x37 | Negative integer -1-0x00..-1-0x17 (-1..-24) | 2605 | | | 2606 | 0x38 | Negative integer -1-n (one-byte uint8_t for n | 2607 | | follows) | 2608 | | | 2609 | 0x39 | Negative integer -1-n (two-byte uint16_t for n | 2610 | | follows) | 2611 | | | 2612 | 0x3a | Negative integer -1-n (four-byte uint32_t for n | 2613 | | follows) | 2614 | | | 2615 | 0x3b | Negative integer -1-n (eight-byte uint64_t for n | 2616 | | follows) | 2617 | | | 2618 | 0x40..0x57 | byte string (0x00..0x17 bytes follow) | 2619 | | | 2620 | 0x58 | byte string (one-byte uint8_t for n, and then n | 2621 | | bytes follow) | 2622 | | | 2623 | 0x59 | byte string (two-byte uint16_t for n, and then n | 2624 | | bytes follow) | 2625 | | | 2626 | 0x5a | byte string (four-byte uint32_t for n, and then n | 2627 | | bytes follow) | 2628 | | | 2629 | 0x5b | byte string (eight-byte uint64_t for n, and then n | 2630 | | bytes follow) | 2631 | | | 2632 | 0x5f | byte string, byte strings follow, terminated by | 2633 | | "break" | 2634 | | | 2635 | 0x60..0x77 | UTF-8 string (0x00..0x17 bytes follow) | 2636 | | | 2637 | 0x78 | UTF-8 string (one-byte uint8_t for n, and then n | 2638 | | bytes follow) | 2639 | | | 2640 | 0x79 | UTF-8 string (two-byte uint16_t for n, and then n | 2641 | | bytes follow) | 2642 | | | 2643 | 0x7a | UTF-8 string (four-byte uint32_t for n, and then n | 2644 | | bytes follow) | 2645 | | | 2646 | 0x7b | UTF-8 string (eight-byte uint64_t for n, and then n | 2647 | | bytes follow) | 2648 | | | 2649 | 0x7f | UTF-8 string, UTF-8 strings follow, terminated by | 2650 | | "break" | 2651 | | | 2652 | 0x80..0x97 | array (0x00..0x17 data items follow) | 2653 | | | 2654 | 0x98 | array (one-byte uint8_t for n, and then n data items | 2655 | | follow) | 2656 | | | 2657 | 0x99 | array (two-byte uint16_t for n, and then n data | 2658 | | items follow) | 2659 | | | 2660 | 0x9a | array (four-byte uint32_t for n, and then n data | 2661 | | items follow) | 2662 | | | 2663 | 0x9b | array (eight-byte uint64_t for n, and then n data | 2664 | | items follow) | 2665 | | | 2666 | 0x9f | array, data items follow, terminated by "break" | 2667 | | | 2668 | 0xa0..0xb7 | map (0x00..0x17 pairs of data items follow) | 2669 | | | 2670 | 0xb8 | map (one-byte uint8_t for n, and then n pairs of | 2671 | | data items follow) | 2672 | | | 2673 | 0xb9 | map (two-byte uint16_t for n, and then n pairs of | 2674 | | data items follow) | 2675 | | | 2676 | 0xba | map (four-byte uint32_t for n, and then n pairs of | 2677 | | data items follow) | 2678 | | | 2679 | 0xbb | map (eight-byte uint64_t for n, and then n pairs of | 2680 | | data items follow) | 2681 | | | 2682 | 0xbf | map, pairs of data items follow, terminated by | 2683 | | "break" | 2684 | | | 2685 | 0xc0 | Text-based date/time (data item follows; see | 2686 | | Section 3.4.1) | 2687 | | | 2688 | 0xc1 | Epoch-based date/time (data item follows; see | 2689 | | Section 3.4.2) | 2690 | | | 2691 | 0xc2 | Positive bignum (data item "byte string" follows) | 2692 | | | 2693 | 0xc3 | Negative bignum (data item "byte string" follows) | 2694 | | | 2695 | 0xc4 | Decimal Fraction (data item "array" follows; see | 2696 | | Section 3.4.4) | 2697 | | | 2698 | 0xc5 | Bigfloat (data item "array" follows; see | 2699 | | Section 3.4.4) | 2700 | | | 2701 | 0xc6..0xd4 | (tag) | 2702 | | | 2703 | 0xd5..0xd7 | Expected Conversion (data item follows; see | 2704 | | Section 3.4.5.2) | 2705 | | | 2706 | 0xd8..0xdb | (more tags, 1/2/4/8 bytes and then a data item | 2707 | | follow) | 2708 | | | 2709 | 0xe0..0xf3 | (simple value) | 2710 | | | 2711 | 0xf4 | False | 2712 | | | 2713 | 0xf5 | True | 2714 | | | 2715 | 0xf6 | Null | 2716 | | | 2717 | 0xf7 | Undefined | 2718 | | | 2719 | 0xf8 | (simple value, one byte follows) | 2720 | | | 2721 | 0xf9 | Half-Precision Float (two-byte IEEE 754) | 2722 | | | 2723 | 0xfa | Single-Precision Float (four-byte IEEE 754) | 2724 | | | 2725 | 0xfb | Double-Precision Float (eight-byte IEEE 754) | 2726 | | | 2727 | 0xff | "break" stop code | 2728 +------------+------------------------------------------------------+ 2730 Table 6: Jump Table for Initial Byte 2732 Appendix C. Pseudocode 2734 The well-formedness of a CBOR item can be checked by the pseudocode 2735 in Figure 1. The data is well-formed if and only if: 2737 * the pseudocode does not "fail"; 2739 * after execution of the pseudocode, no bytes are left in the input 2740 (except in streaming applications) 2742 The pseudocode has the following prerequisites: 2744 * take(n) reads n bytes from the input data and returns them as a 2745 byte string. If n bytes are no longer available, take(n) fails. 2747 * uint() converts a byte string into an unsigned integer by 2748 interpreting the byte string in network byte order. 2750 * Arithmetic works as in C. 2752 * All variables are unsigned integers of sufficient range. 2754 Note that "well_formed" returns the major type for well-formed 2755 definite length items, but 0 for an indefinite length item (or -1 for 2756 a break stop code, only if "breakable" is set). This is used in 2757 "well_formed_indefinite" to ascertain that indefinite length strings 2758 only contain definite length strings as chunks. 2760 well_formed (breakable = false) { 2761 // process initial bytes 2762 ib = uint(take(1)); 2763 mt = ib >> 5; 2764 val = ai = ib & 0x1f; 2765 switch (ai) { 2766 case 24: val = uint(take(1)); break; 2767 case 25: val = uint(take(2)); break; 2768 case 26: val = uint(take(4)); break; 2769 case 27: val = uint(take(8)); break; 2770 case 28: case 29: case 30: fail(); 2771 case 31: 2772 return well_formed_indefinite(mt, breakable); 2773 } 2774 // process content 2775 switch (mt) { 2776 // case 0, 1, 7 do not have content; just use val 2777 case 2: case 3: take(val); break; // bytes/UTF-8 2778 case 4: for (i = 0; i < val; i++) well_formed(); break; 2779 case 5: for (i = 0; i < val*2; i++) well_formed(); break; 2780 case 6: well_formed(); break; // 1 embedded data item 2781 case 7: if (ai == 24 && val < 32) fail(); // bad simple 2782 } 2783 return mt; // finite data item 2784 } 2786 well_formed_indefinite(mt, breakable) { 2787 switch (mt) { 2788 case 2: case 3: 2789 while ((it = well_formed(true)) != -1) 2790 if (it != mt) // need finite-length chunk 2791 fail(); // of same type 2792 break; 2793 case 4: while (well_formed(true) != -1); break; 2794 case 5: while (well_formed(true) != -1) well_formed(); break; 2795 case 7: 2796 if (breakable) 2797 return -1; // signal break out 2798 else fail(); // no enclosing indefinite 2799 default: fail(); // wrong mt 2800 } 2801 return 0; // no break out 2802 } 2804 Figure 1: Pseudocode for Well-Formedness Check 2806 Note that the remaining complexity of a complete CBOR decoder is 2807 about presenting data that has been decoded to the application in an 2808 appropriate form. 2810 Major types 0 and 1 are designed in such a way that they can be 2811 encoded in C from a signed integer without actually doing an if-then- 2812 else for positive/negative (Figure 2). This uses the fact that 2813 (-1-n), the transformation for major type 1, is the same as ~n 2814 (bitwise complement) in C unsigned arithmetic; ~n can then be 2815 expressed as (-1)^n for the negative case, while 0^n leaves n 2816 unchanged for non-negative. The sign of a number can be converted to 2817 -1 for negative and 0 for non-negative (0 or positive) by arithmetic- 2818 shifting the number by one bit less than the bit length of the number 2819 (for example, by 63 for 64-bit numbers). 2821 void encode_sint(int64_t n) { 2822 uint64t ui = n >> 63; // extend sign to whole length 2823 mt = ui & 0x20; // extract major type 2824 ui ^= n; // complement negatives 2825 if (ui < 24) 2826 *p++ = mt + ui; 2827 else if (ui < 256) { 2828 *p++ = mt + 24; 2829 *p++ = ui; 2830 } else 2831 ... 2833 Figure 2: Pseudocode for Encoding a Signed Integer 2835 Appendix D. Half-Precision 2837 As half-precision floating-point numbers were only added to IEEE 754 2838 in 2008 [IEEE754], today's programming platforms often still only 2839 have limited support for them. It is very easy to include at least 2840 decoding support for them even without such support. An example of a 2841 small decoder for half-precision floating-point numbers in the C 2842 language is shown in Figure 3. A similar program for Python is in 2843 Figure 4; this code assumes that the 2-byte value has already been 2844 decoded as an (unsigned short) integer in network byte order (as 2845 would be done by the pseudocode in Appendix C). 2847 #include 2849 double decode_half(unsigned char *halfp) { 2850 int half = (halfp[0] << 8) + halfp[1]; 2851 int exp = (half >> 10) & 0x1f; 2852 int mant = half & 0x3ff; 2853 double val; 2854 if (exp == 0) val = ldexp(mant, -24); 2855 else if (exp != 31) val = ldexp(mant + 1024, exp - 25); 2856 else val = mant == 0 ? INFINITY : NAN; 2857 return half & 0x8000 ? -val : val; 2858 } 2860 Figure 3: C Code for a Half-Precision Decoder 2862 import struct 2863 from math import ldexp 2865 def decode_single(single): 2866 return struct.unpack("!f", struct.pack("!I", single))[0] 2868 def decode_half(half): 2869 valu = (half & 0x7fff) << 13 | (half & 0x8000) << 16 2870 if ((half & 0x7c00) != 0x7c00): 2871 return ldexp(decode_single(valu), 112) 2872 return decode_single(valu | 0x7f800000) 2874 Figure 4: Python Code for a Half-Precision Decoder 2876 Appendix E. Comparison of Other Binary Formats to CBOR's Design 2877 Objectives 2879 The proposal for CBOR follows a history of binary formats that is as 2880 long as the history of computers themselves. Different formats have 2881 had different objectives. In most cases, the objectives of the 2882 format were never stated, although they can sometimes be implied by 2883 the context where the format was first used. Some formats were meant 2884 to be universally usable, although history has proven that no binary 2885 format meets the needs of all protocols and applications. 2887 CBOR differs from many of these formats due to it starting with a set 2888 of objectives and attempting to meet just those. This section 2889 compares a few of the dozens of formats with CBOR's objectives in 2890 order to help the reader decide if they want to use CBOR or a 2891 different format for a particular protocol or application. 2893 Note that the discussion here is not meant to be a criticism of any 2894 format: to the best of our knowledge, no format before CBOR was meant 2895 to cover CBOR's objectives in the priority we have assigned them. A 2896 brief recap of the objectives from Section 1.1 is: 2898 1. unambiguous encoding of most common data formats from Internet 2899 standards 2901 2. code compactness for encoder or decoder 2903 3. no schema description needed 2905 4. reasonably compact serialization 2907 5. applicability to constrained and unconstrained applications 2909 6. good JSON conversion 2911 7. extensibility 2913 A discussion of CBOR and other formats with respect to a different 2914 set of design objectives is provided in Section 5 and Appendix C of 2915 [RFC8618]. 2917 E.1. ASN.1 DER, BER, and PER 2919 [ASN.1] has many serializations. In the IETF, DER and BER are the 2920 most common. The serialized output is not particularly compact for 2921 many items, and the code needed to decode numeric items can be 2922 complex on a constrained device. 2924 Few (if any) IETF protocols have adopted one of the several variants 2925 of Packed Encoding Rules (PER). There could be many reasons for 2926 this, but one that is commonly stated is that PER makes use of the 2927 schema even for parsing the surface structure of the data stream, 2928 requiring significant tool support. There are different versions of 2929 the ASN.1 schema language in use, which has also hampered adoption. 2931 E.2. MessagePack 2933 [MessagePack] is a concise, widely implemented counted binary 2934 serialization format, similar in many properties to CBOR, although 2935 somewhat less regular. While the data model can be used to represent 2936 JSON data, MessagePack has also been used in many remote procedure 2937 call (RPC) applications and for long-term storage of data. 2939 MessagePack has been essentially stable since it was first published 2940 around 2011; it has not yet had a transition. The evolution of 2941 MessagePack is impeded by an imperative to maintain complete 2942 backwards compatibility with existing stored data, while only few 2943 bytecodes are still available for extension. Repeated requests over 2944 the years from the MessagePack user community to separate out binary 2945 and text strings in the encoding recently have led to an extension 2946 proposal that would leave MessagePack's "raw" data ambiguous between 2947 its usages for binary and text data. The extension mechanism for 2948 MessagePack remains unclear. 2950 E.3. BSON 2952 [BSON] is a data format that was developed for the storage of JSON- 2953 like maps (JSON objects) in the MongoDB database. Its major 2954 distinguishing feature is the capability for in-place update, which 2955 prevents a compact representation. BSON uses a counted 2956 representation except for map keys, which are null-byte terminated. 2957 While BSON can be used for the representation of JSON-like objects on 2958 the wire, its specification is dominated by the requirements of the 2959 database application and has become somewhat baroque. The status of 2960 how BSON extensions will be implemented remains unclear. 2962 E.4. MSDTP: RFC 713 2964 Message Services Data Transmission (MSDTP) is a very early example of 2965 a compact message format; it is described in [RFC0713], written in 2966 1976. It is included here for its historical value, not because it 2967 was ever widely used. 2969 E.5. Conciseness on the Wire 2971 While CBOR's design objective of code compactness for encoders and 2972 decoders is a higher priority than its objective of conciseness on 2973 the wire, many people focus on the wire size. Table 7 shows some 2974 encoding examples for the simple nested array [1, [2, 3]]; where some 2975 form of indefinite-length encoding is supported by the encoding, 2976 [_ 1, [2, 3]] (indefinite length on the outer array) is also shown. 2978 +-------------+--------------------------+--------------------------+ 2979 | Format | [1, [2, 3]] | [_ 1, [2, 3]] | 2980 +-------------+--------------------------+--------------------------+ 2981 | RFC 713 | c2 05 81 c2 02 82 83 | | 2982 | | | | 2983 | ASN.1 BER | 30 0b 02 01 01 30 06 02 | 30 80 02 01 01 30 06 02 | 2984 | | 01 02 02 01 03 | 01 02 02 01 03 00 00 | 2985 | | | | 2986 | MessagePack | 92 01 92 02 03 | | 2987 | | | | 2988 | BSON | 22 00 00 00 10 30 00 01 | | 2989 | | 00 00 00 04 31 00 13 00 | | 2990 | | 00 00 10 30 00 02 00 00 | | 2991 | | 00 10 31 00 03 00 00 00 | | 2992 | | 00 00 | | 2993 | | | | 2994 | CBOR | 82 01 82 02 03 | 9f 01 82 02 03 ff | 2995 +-------------+--------------------------+--------------------------+ 2997 Table 7: Examples for Different Levels of Conciseness 2999 Appendix F. Changes from RFC 7049 3001 The following is a list of known changes from RFC 7049. This list is 3002 non-authoritative. It is meant to help reviewers see the significant 3003 differences. 3005 * Updated reference for [RFC4627] to [RFC8259] in many places 3007 * Updated reference for [CNN-TERMS] to [RFC7228] 3009 * Added a comment to the last example in Section 2.2.1 (added 3010 "Second value") 3012 * Fixed a bug in the example in Section 2.4.2 ("29" -> "49") 3014 * Fixed a bug in the last paragraph of Section 3.6 ("0b000_11101" -> 3015 "0b000_11001") 3017 Appendix G. Well-formedness errors and examples 3019 There are three basic kinds of well-formedness errors that can occur 3020 in decoding a CBOR data item: 3022 * Too much data: There are input bytes left that were not consumed. 3023 This is only an error if the application assumed that the input 3024 bytes would span exactly one data item. Where the application 3025 uses the self-delimiting nature of CBOR encoding to permit 3026 additional data after the data item, as is for example done in 3027 CBOR sequences [I-D.ietf-cbor-sequence], the CBOR decoder can 3028 simply indicate what part of the input has not been consumed. 3030 * Too little data: The input data available would need additional 3031 bytes added at their end for a complete CBOR data item. This may 3032 indicate the input is truncated; it is also a common error when 3033 trying to decode random data as CBOR. For some applications 3034 however, this may not be actually be an error, as the application 3035 may not be certain it has all the data yet and can obtain or wait 3036 for additional input bytes. Some of these applications may have 3037 an upper limit for how much additional data can show up; here the 3038 decoder may be able to indicate that the encoded CBOR data item 3039 cannot be completed within this limit. 3041 * Syntax error: The input data are not consistent with the 3042 requirements of the CBOR encoding, and this cannot be remedied by 3043 adding (or removing) data at the end. 3045 In Appendix C, errors of the first kind are addressed in the first 3046 paragraph/bullet list (requiring "no bytes are left"), and errors of 3047 the second kind are addressed in the second paragraph/bullet list 3048 (failing "if n bytes are no longer available"). Errors of the third 3049 kind are identified in the pseudocode by specific instances of 3050 calling fail(), in order: 3052 * a reserved value is used for additional information (28, 29, 30) 3054 * major type 7, additional information 24, value < 32 (incorrect or 3055 incorrectly encoded simple type) 3057 * incorrect substructure of indefinite length byte/text string (may 3058 only contain definite length strings of the same major type) 3060 * break stop code (mt=7, ai=31) occurs in a value position of a map 3061 or except at a position directly in an indefinite length item 3062 where also another enclosed data item could occur 3064 * additional information 31 used with major type 0, 1, or 6 3066 G.1. Examples for CBOR data items that are not well-formed 3068 This subsection shows a few examples for CBOR data items that are not 3069 well-formed. Each example is a sequence of bytes each shown in 3070 hexadecimal; multiple examples in a list are separated by commas. 3072 Examples for well-formedness error kind 1 (too much data) can easily 3073 be formed by adding data to a well-formed encoded CBOR data item. 3075 Similarly, examples for well-formedness error kind 2 (too little 3076 data) can be formed by truncating a well-formed encoded CBOR data 3077 item. In test suites, it may be beneficial to specifically test with 3078 incomplete data items that would require large amounts of addition to 3079 be completed (for instance by starting the encoding of a string of a 3080 very large size). 3082 A premature end of the input can occur in a head or within the 3083 enclosed data, which may be bare strings or enclosed data items that 3084 are either counted or should have been ended by a break stop code. 3086 * End of input in a head: 18, 19, 1a, 1b, 19 01, 1a 01 02, 1b 01 02 3087 03 04 05 06 07, 38, 58, 78, 98, 9a 01 ff 00, b8, d8, f8, f9 00, fa 3088 00 00, fb 00 00 00 3090 * Definite length strings with short data: 41, 61, 5a ff ff ff ff 3091 00, 5b ff ff ff ff ff ff ff ff 01 02 03, 7a ff ff ff ff 00, 7b 7f 3092 ff ff ff ff ff ff ff 01 02 03 3094 * Definite length maps and arrays not closed with enough items: 81, 3095 81 81 81 81 81 81 81 81 81, 82 00, a1, a2 01 02, a1 00, a2 00 00 3096 00 3098 * Indefinite length strings not closed by a break stop code: 5f 41 3099 00, 7f 61 00 3101 * Indefinite length maps and arrays not closed by a break stop code: 3102 9f, 9f 01 02, bf, bf 01 02 01 02, 81 9f, 9f 80 00, 9f 9f 9f 9f 9f 3103 ff ff ff ff, 9f 81 9f 81 9f 9f ff ff ff 3105 A few examples for the five subkinds of well-formedness error kind 3 3106 (syntax error) are shown below. 3108 Subkind 1: 3110 * Reserved additional information values: 1c, 1d, 1e, 3c, 3d, 3e, 3111 5c, 5d, 5e, 7c, 7d, 7e, 9c, 9d, 9e, bc, bd, be, dc, dd, de, fc, 3112 fd, fe, 3114 Subkind 2: 3116 * Reserved two-byte encodings of simple types: f8 00, f8 01, f8 18, 3117 f8 1f 3119 Subkind 3: 3121 * Indefinite length string chunks not of the correct type: 5f 00 ff, 3122 5f 21 ff, 5f 61 00 ff, 5f 80 ff, 5f a0 ff, 5f c0 00 ff, 5f e0 ff, 3123 7f 41 00 ff 3125 * Indefinite length string chunks not definite length: 5f 5f 41 00 3126 ff ff, 7f 7f 61 00 ff ff 3128 Subkind 4: 3130 * Break occurring on its own outside of an indefinite length item: 3131 ff 3133 * Break occurring in a definite length array or map or a tag: 81 ff, 3134 82 00 ff, a1 ff, a1 ff 00, a1 00 ff, a2 00 00 ff, 9f 81 ff, 9f 82 3135 9f 81 9f 9f ff ff ff ff 3137 * Break in indefinite length map would lead to odd number of items 3138 (break in a value position): bf 00 ff, bf 00 00 00 ff 3140 Subkind 5: 3142 * Major type 0, 1, 6 with additional information 31: 1f, 3f, df 3144 Acknowledgements 3146 CBOR was inspired by MessagePack. MessagePack was developed and 3147 promoted by Sadayuki Furuhashi ("frsyuki"). This reference to 3148 MessagePack is solely for attribution; CBOR is not intended as a 3149 version of or replacement for MessagePack, as it has different design 3150 goals and requirements. 3152 The need for functionality beyond the original MessagePack 3153 Specification became obvious to many people at about the same time 3154 around the year 2012. BinaryPack is a minor derivation of 3155 MessagePack that was developed by Eric Zhang for the binaryjs 3156 project. A similar, but different, extension was made by Tim Caswell 3157 for his msgpack-js and msgpack-js-browser projects. Many people have 3158 contributed to the discussion about extending MessagePack to separate 3159 text string representation from byte string representation. 3161 The encoding of the additional information in CBOR was inspired by 3162 the encoding of length information designed by Klaus Hartke for CoAP. 3164 This document also incorporates suggestions made by many people, 3165 notably Dan Frost, James Manger, Jeffrey Yasskin, Joe Hildebrand, 3166 Keith Moore, Laurence Lundblade, Matthew Lepinski, Michael 3167 Richardson, Nico Williams, Peter Occil, Phillip Hallam-Baker, Ray 3168 Polk, Tim Bray, Tony Finch, Tony Hansen, and Yaron Sheffer. 3170 Authors' Addresses 3172 Carsten Bormann 3173 Universitaet Bremen TZI 3174 Postfach 330440 3175 D-28359 Bremen 3176 Germany 3178 Phone: +49-421-218-63921 3179 EMail: cabo@tzi.org 3181 Paul Hoffman 3182 ICANN 3184 EMail: paul.hoffman@icann.org