idnits 2.17.1 draft-ietf-cbor-7049bis-09.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (November 05, 2019) is 1627 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: '2' on line 2558 -- Looks like a reference, but probably isn't: '3' on line 2558 -- Looks like a reference, but probably isn't: '4' on line 2556 -- Looks like a reference, but probably isn't: '5' on line 2556 -- Looks like a reference, but probably isn't: '100' on line 1406 == Missing Reference: '-1' is mentioned on line 1402, but not defined -- Looks like a reference, but probably isn't: '1' on line 2842 == Missing Reference: 'RFCthis' is mentioned on line 2128, but not defined == Missing Reference: 'TM' is mentioned on line 2377, but not defined -- Looks like a reference, but probably isn't: '0' on line 2858 == Missing Reference: 'RFC4627' is mentioned on line 2997, but not defined ** Obsolete undefined reference: RFC 4627 (Obsoleted by RFC 7158, RFC 7159) == Missing Reference: 'CNN-TERMS' is mentioned on line 2999, but not defined -- Possible downref: Non-RFC (?) normative reference: ref. 'ECMA262' -- Possible downref: Non-RFC (?) normative reference: ref. 'IEEE754' -- Obsolete informational reference (is this intentional?): RFC 7049 (Obsoleted by RFC 8949) Summary: 1 error (**), 0 flaws (~~), 6 warnings (==), 12 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group C. Bormann 3 Internet-Draft Universitaet Bremen TZI 4 Obsoletes: 7049 (if approved) P. Hoffman 5 Intended status: Standards Track ICANN 6 Expires: May 8, 2020 November 05, 2019 8 Concise Binary Object Representation (CBOR) 9 draft-ietf-cbor-7049bis-09 11 Abstract 13 The Concise Binary Object Representation (CBOR) is a data format 14 whose design goals include the possibility of extremely small code 15 size, fairly small message size, and extensibility without the need 16 for version negotiation. These design goals make it different from 17 earlier binary serializations such as ASN.1 and MessagePack. 19 This document is a revised edition of RFC 7049, with editorial 20 improvements, added detail, and fixed errata. This revision formally 21 obsoletes RFC 7049, while keeping full compatibility of the 22 interchange format from RFC 7049. It does not create a new version 23 of the format. 25 Contributing 27 This document is being worked on in the CBOR Working Group. Please 28 contribute on the mailing list there, or in the GitHub repository for 29 this draft: https://github.com/cbor-wg/CBORbis 31 The charter for the CBOR Working Group says that the WG will update 32 RFC 7049 to fix verified errata. Security issues and clarifications 33 may be addressed, but changes to this document will ensure backward 34 compatibility for popular deployed codebases. This document will be 35 targeted at becoming an Internet Standard. 37 Status of This Memo 39 This Internet-Draft is submitted in full conformance with the 40 provisions of BCP 78 and BCP 79. 42 Internet-Drafts are working documents of the Internet Engineering 43 Task Force (IETF). Note that other groups may also distribute 44 working documents as Internet-Drafts. The list of current Internet- 45 Drafts is at https://datatracker.ietf.org/drafts/current/. 47 Internet-Drafts are draft documents valid for a maximum of six months 48 and may be updated, replaced, or obsoleted by other documents at any 49 time. It is inappropriate to use Internet-Drafts as reference 50 material or to cite them other than as "work in progress." 52 This Internet-Draft will expire on May 8, 2020. 54 Copyright Notice 56 Copyright (c) 2019 IETF Trust and the persons identified as the 57 document authors. All rights reserved. 59 This document is subject to BCP 78 and the IETF Trust's Legal 60 Provisions Relating to IETF Documents 61 (https://trustee.ietf.org/license-info) in effect on the date of 62 publication of this document. Please review these documents 63 carefully, as they describe your rights and restrictions with respect 64 to this document. Code Components extracted from this document must 65 include Simplified BSD License text as described in Section 4.e of 66 the Trust Legal Provisions and are provided without warranty as 67 described in the Simplified BSD License. 69 Table of Contents 71 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4 72 1.1. Objectives . . . . . . . . . . . . . . . . . . . . . . . 4 73 1.2. Terminology . . . . . . . . . . . . . . . . . . . . . . . 6 74 2. CBOR Data Models . . . . . . . . . . . . . . . . . . . . . . 7 75 2.1. Extended Generic Data Models . . . . . . . . . . . . . . 8 76 2.2. Specific Data Models . . . . . . . . . . . . . . . . . . 9 77 3. Specification of the CBOR Encoding . . . . . . . . . . . . . 9 78 3.1. Major Types . . . . . . . . . . . . . . . . . . . . . . . 11 79 3.2. Indefinite Lengths for Some Major Types . . . . . . . . . 13 80 3.2.1. The "break" Stop Code . . . . . . . . . . . . . . . . 13 81 3.2.2. Indefinite-Length Arrays and Maps . . . . . . . . . . 14 82 3.2.3. Indefinite-Length Byte Strings and Text Strings . . . 16 83 3.3. Floating-Point Numbers and Values with No Content . . . . 16 84 3.4. Tagging of Items . . . . . . . . . . . . . . . . . . . . 18 85 3.4.1. Date and Time . . . . . . . . . . . . . . . . . . . . 21 86 3.4.2. Standard Date/Time String . . . . . . . . . . . . . . 21 87 3.4.3. Epoch-based Date/Time . . . . . . . . . . . . . . . . 21 88 3.4.4. Bignums . . . . . . . . . . . . . . . . . . . . . . . 22 89 3.4.5. Decimal Fractions and Bigfloats . . . . . . . . . . . 22 90 3.4.6. Content Hints . . . . . . . . . . . . . . . . . . . . 24 91 3.4.6.1. Encoded CBOR Data Item . . . . . . . . . . . . . 24 92 3.4.6.2. Expected Later Encoding for CBOR-to-JSON 93 Converters . . . . . . . . . . . . . . . . . . . 24 94 3.4.6.3. Encoded Text . . . . . . . . . . . . . . . . . . 25 95 3.4.7. Self-Described CBOR . . . . . . . . . . . . . . . . . 26 96 4. Serialization Considerations . . . . . . . . . . . . . . . . 26 97 4.1. Preferred Serialization . . . . . . . . . . . . . . . . . 26 98 4.2. Deterministically Encoded CBOR . . . . . . . . . . . . . 27 99 4.2.1. Core Deterministic Encoding Requirements . . . . . . 28 100 4.2.2. Additional Deterministic Encoding Considerations . . 29 101 4.2.3. Length-first map key ordering . . . . . . . . . . . . 30 102 5. Creating CBOR-Based Protocols . . . . . . . . . . . . . . . . 31 103 5.1. CBOR in Streaming Applications . . . . . . . . . . . . . 32 104 5.2. Generic Encoders and Decoders . . . . . . . . . . . . . . 32 105 5.3. Validity of Items . . . . . . . . . . . . . . . . . . . . 33 106 5.3.1. Basic validity . . . . . . . . . . . . . . . . . . . 33 107 5.3.2. Tag validity . . . . . . . . . . . . . . . . . . . . 34 108 5.4. Validity and Evolution . . . . . . . . . . . . . . . . . 34 109 5.5. Numbers . . . . . . . . . . . . . . . . . . . . . . . . . 35 110 5.6. Specifying Keys for Maps . . . . . . . . . . . . . . . . 36 111 5.6.1. Equivalence of Keys . . . . . . . . . . . . . . . . . 37 112 5.7. Undefined Values . . . . . . . . . . . . . . . . . . . . 38 113 6. Converting Data between CBOR and JSON . . . . . . . . . . . . 38 114 6.1. Converting from CBOR to JSON . . . . . . . . . . . . . . 38 115 6.2. Converting from JSON to CBOR . . . . . . . . . . . . . . 39 116 7. Future Evolution of CBOR . . . . . . . . . . . . . . . . . . 40 117 7.1. Extension Points . . . . . . . . . . . . . . . . . . . . 41 118 7.2. Curating the Additional Information Space . . . . . . . . 42 119 8. Diagnostic Notation . . . . . . . . . . . . . . . . . . . . . 42 120 8.1. Encoding Indicators . . . . . . . . . . . . . . . . . . . 43 121 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 44 122 9.1. Simple Values Registry . . . . . . . . . . . . . . . . . 44 123 9.2. Tags Registry . . . . . . . . . . . . . . . . . . . . . . 44 124 9.3. Media Type ("MIME Type") . . . . . . . . . . . . . . . . 45 125 9.4. CoAP Content-Format . . . . . . . . . . . . . . . . . . . 46 126 9.5. The +cbor Structured Syntax Suffix Registration . . . . . 46 127 10. Security Considerations . . . . . . . . . . . . . . . . . . . 47 128 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 49 129 11.1. Normative References . . . . . . . . . . . . . . . . . . 49 130 11.2. Informative References . . . . . . . . . . . . . . . . . 50 131 Appendix A. Examples . . . . . . . . . . . . . . . . . . . . . . 53 132 Appendix B. Jump Table . . . . . . . . . . . . . . . . . . . . . 57 133 Appendix C. Pseudocode . . . . . . . . . . . . . . . . . . . . . 60 134 Appendix D. Half-Precision . . . . . . . . . . . . . . . . . . . 62 135 Appendix E. Comparison of Other Binary Formats to CBOR's Design 136 Objectives . . . . . . . . . . . . . . . . . . . . . 63 137 E.1. ASN.1 DER, BER, and PER . . . . . . . . . . . . . . . . . 64 138 E.2. MessagePack . . . . . . . . . . . . . . . . . . . . . . . 64 139 E.3. BSON . . . . . . . . . . . . . . . . . . . . . . . . . . 65 140 E.4. MSDTP: RFC 713 . . . . . . . . . . . . . . . . . . . . . 65 141 E.5. Conciseness on the Wire . . . . . . . . . . . . . . . . . 65 142 Appendix F. Changes from RFC 7049 . . . . . . . . . . . . . . . 66 143 Appendix G. Well-formedness errors and examples . . . . . . . . 66 144 G.1. Examples for CBOR data items that are not well-formed . . 67 146 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . 69 147 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 70 149 1. Introduction 151 There are hundreds of standardized formats for binary representation 152 of structured data (also known as binary serialization formats). Of 153 those, some are for specific domains of information, while others are 154 generalized for arbitrary data. In the IETF, probably the best-known 155 formats in the latter category are ASN.1's BER and DER [ASN.1]. 157 The format defined here follows some specific design goals that are 158 not well met by current formats. The underlying data model is an 159 extended version of the JSON data model [RFC8259]. It is important 160 to note that this is not a proposal that the grammar in RFC 8259 be 161 extended in general, since doing so would cause a significant 162 backwards incompatibility with already deployed JSON documents. 163 Instead, this document simply defines its own data model that starts 164 from JSON. 166 Appendix E lists some existing binary formats and discusses how well 167 they do or do not fit the design objectives of the Concise Binary 168 Object Representation (CBOR). 170 This document is a revised edition of [RFC7049], with editorial 171 improvements, added detail, and fixed errata. This revision formally 172 obsoletes RFC 7049, while keeping full compatibility of the 173 interchange format from RFC 7049. It does not create a new version 174 of the format. 176 1.1. Objectives 178 The objectives of CBOR, roughly in decreasing order of importance, 179 are: 181 1. The representation must be able to unambiguously encode most 182 common data formats used in Internet standards. 184 * It must represent a reasonable set of basic data types and 185 structures using binary encoding. "Reasonable" here is 186 largely influenced by the capabilities of JSON, with the major 187 addition of binary byte strings. The structures supported are 188 limited to arrays and trees; loops and lattice-style graphs 189 are not supported. 191 * There is no requirement that all data formats be uniquely 192 encoded; that is, it is acceptable that the number "7" might 193 be encoded in multiple different ways. 195 2. The code for an encoder or decoder must be able to be compact in 196 order to support systems with very limited memory, processor 197 power, and instruction sets. 199 * An encoder and a decoder need to be implementable in a very 200 small amount of code (for example, in class 1 constrained 201 nodes as defined in [RFC7228]). 203 * The format should use contemporary machine representations of 204 data (for example, not requiring binary-to-decimal 205 conversion). 207 3. Data must be able to be decoded without a schema description. 209 * Similar to JSON, encoded data should be self-describing so 210 that a generic decoder can be written. 212 4. The serialization must be reasonably compact, but data 213 compactness is secondary to code compactness for the encoder and 214 decoder. 216 * "Reasonable" here is bounded by JSON as an upper bound in 217 size, and by implementation complexity maintaining a lower 218 bound. Using either general compression schemes or extensive 219 bit-fiddling violates the complexity goals. 221 5. The format must be applicable to both constrained nodes and high- 222 volume applications. 224 * This means it must be reasonably frugal in CPU usage for both 225 encoding and decoding. This is relevant both for constrained 226 nodes and for potential usage in applications with a very high 227 volume of data. 229 6. The format must support all JSON data types for conversion to and 230 from JSON. 232 * It must support a reasonable level of conversion as long as 233 the data represented is within the capabilities of JSON. It 234 must be possible to define a unidirectional mapping towards 235 JSON for all types of data. 237 7. The format must be extensible, and the extended data must be 238 decodable by earlier decoders. 240 * The format is designed for decades of use. 242 * The format must support a form of extensibility that allows 243 fallback so that a decoder that does not understand an 244 extension can still decode the message. 246 * The format must be able to be extended in the future by later 247 IETF standards. 249 1.2. Terminology 251 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 252 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 253 "OPTIONAL" in this document are to be interpreted as described in 254 BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all 255 capitals, as shown here. 257 The term "byte" is used in its now-customary sense as a synonym for 258 "octet". All multi-byte values are encoded in network byte order 259 (that is, most significant byte first, also known as "big-endian"). 261 This specification makes use of the following terminology: 263 Data item: A single piece of CBOR data. The structure of a data 264 item may contain zero, one, or more nested data items. The term 265 is used both for the data item in representation format and for 266 the abstract idea that can be derived from that by a decoder; the 267 former can be addressed specifically by using "encoded data item". 269 Decoder: A process that decodes a well-formed CBOR data item and 270 makes it available to an application. Formally speaking, a 271 decoder contains a parser to break up the input using the syntax 272 rules of CBOR, as well as a semantic processor to prepare the data 273 in a form suitable to the application. 275 Encoder: A process that generates the representation format of a 276 CBOR data item from application information. 278 Data Stream: A sequence of zero or more data items, not further 279 assembled into a larger containing data item. The independent 280 data items that make up a data stream are sometimes also referred 281 to as "top-level data items". 283 Well-formed: A data item that follows the syntactic structure of 284 CBOR. A well-formed data item uses the initial bytes and the byte 285 strings and/or data items that are implied by their values as 286 defined in CBOR and does not include following extraneous data. 287 CBOR decoders by definition only return contents from well-formed 288 data items. 290 Valid: A data item that is well-formed and also follows the semantic 291 restrictions that apply to CBOR data items. 293 Expected: Besides its normal English meaning, the term "expected" is 294 used to describe requirements beyond CBOR validity that an 295 application has on its input data. Well-formed (processable at 296 all), valid (checked by a validity-checking generic decoder), and 297 expected (checked by the application) form a hierarchy of layers 298 of acceptability. 300 Stream decoder: A process that decodes a data stream and makes each 301 of the data items in the sequence available to an application as 302 they are received. 304 Where bit arithmetic or data types are explained, this document uses 305 the notation familiar from the programming language C, except that 306 "**" denotes exponentiation. Similar to the "0x" notation for 307 hexadecimal numbers, numbers in binary notation are prefixed with 308 "0b". Underscores can be added to such a number solely for 309 readability, so 0b00100001 (0x21) might be written 0b001_00001 to 310 emphasize the desired interpretation of the bits in the byte; in this 311 case, it is split into three bits and five bits. Encoded CBOR data 312 items are sometimes given in the "0x" or "0b" notation; these values 313 are first interpreted as numbers as in C and are then interpreted as 314 byte strings in network byte order, including any leading zero bytes 315 expressed in the notation. 317 2. CBOR Data Models 319 CBOR is explicit about its generic data model, which defines the set 320 of all data items that can be represented in CBOR. Its basic generic 321 data model is extensible by the registration of simple type values 322 and tags. Applications can then subset the resulting extended 323 generic data model to build their specific data models. 325 Within environments that can represent the data items in the generic 326 data model, generic CBOR encoders and decoders can be implemented 327 (which usually involves defining additional implementation data types 328 for those data items that do not already have a natural 329 representation in the environment). The ability to provide generic 330 encoders and decoders is an explicit design goal of CBOR; however 331 many applications will provide their own application-specific 332 encoders and/or decoders. 334 In the basic (un-extended) generic data model, a data item is one of: 336 o an integer in the range -2**64..2**64-1 inclusive 337 o a simple value, identified by a number between 0 and 255, but 338 distinct from that number 340 o a floating-point value, distinct from an integer, out of the set 341 representable by IEEE 754 binary64 (including non-finites) 342 [IEEE754] 344 o a sequence of zero or more bytes ("byte string") 346 o a sequence of zero or more Unicode code points ("text string") 348 o a sequence of zero or more data items ("array") 350 o a mapping (mathematical function) from zero or more data items 351 ("keys") each to a data item ("values"), ("map") 353 o a tagged data item ("tag"), comprising a tag number (an integer in 354 the range 0..2**64-1) and a tagged value (a data item) 356 Note that integer and floating-point values are distinct in this 357 model, even if they have the same numeric value. 359 Also note that serialization variants, such as number of bytes of the 360 encoded floating value, or the choice of one of the ways in which an 361 integer, the length of a text or byte string, the number of elements 362 in an array or pairs in a map, or a tag number, (collectively "the 363 argument", see Section 3) can be encoded, are not visible at the 364 generic data model level. 366 2.1. Extended Generic Data Models 368 This basic generic data model comes pre-extended by the registration 369 of a number of simple values and tag numbers right in this document, 370 such as: 372 o "false", "true", "null", and "undefined" (simple values identified 373 by 20..23) 375 o integer and floating-point values with a larger range and 376 precision than the above (tag numbers 2 to 5) 378 o application data types such as a point in time or an RFC 3339 379 date/time string (tag numbers 1, 0) 381 Further elements of the extended generic data model can be (and have 382 been) defined via the IANA registries created for CBOR. Even if such 383 an extension is unknown to a generic encoder or decoder, data items 384 using that extension can be passed to or from the application by 385 representing them at the interface to the application within the 386 basic generic data model, i.e., as generic values of a simple type or 387 generic tags. 389 In other words, the basic generic data model is stable as defined in 390 this document, while the extended generic data model expands by the 391 registration of new simple values or tag numbers, but never shrinks. 393 While there is a strong expectation that generic encoders and 394 decoders can represent "false", "true", and "null" ("undefined" is 395 intentionally omitted) in the form appropriate for their programming 396 environment, implementation of the data model extensions created by 397 tags is truly optional and a matter of implementation quality. 399 2.2. Specific Data Models 401 The specific data model for a CBOR-based protocol usually subsets the 402 extended generic data model and assigns application semantics to the 403 data items within this subset and its components. When documenting 404 such specific data models, where it is desired to specify the types 405 of data items, it is preferred to identify the types by the names 406 they have in the generic data model ("negative integer", "array") 407 instead of by referring to aspects of their CBOR representation 408 ("major type 1", "major type 4"). 410 Specific data models can also specify what values (including values 411 of different types) are equivalent for the purposes of map keys and 412 encoder freedom. For example, in the generic data model, a valid map 413 MAY have both "0" and "0.0" as keys, and an encoder MUST NOT encode 414 "0.0" as an integer (major type 0, Section 3.1). However, if a 415 specific data model declares that floating-point and integer 416 representations of integral values are equivalent, using both map 417 keys "0" and "0.0" in a single map would be considered duplicates and 418 so invalid, and an encoder could encode integral-valued floats as 419 integers or vice versa, perhaps to save encoded bytes. 421 3. Specification of the CBOR Encoding 423 A CBOR data item (Section 2) is encoded to or decoded from a byte 424 string carrying a well-formed encoded data item as described in this 425 section. The encoding is summarized in Table 6. An encoder MUST 426 produce only well-formed encoded data items. A decoder MUST NOT 427 return a decoded data item when it encounters input that is not a 428 well-formed encoded CBOR data item (this does not detract from the 429 usefulness of diagnostic and recovery tools that might make available 430 some information from a damaged encoded CBOR data item). 432 The initial byte of each encoded data item contains both information 433 about the major type (the high-order 3 bits, described in 434 Section 3.1) and additional information (the low-order 5 bits). With 435 a few exceptions, the additional information's value describes how to 436 load an unsigned integer "argument": 438 Less than 24: The argument's value is the value of the additional 439 information. 441 24, 25, 26, or 27: The argument's value is held in the following 1, 442 2, 4, or 8 bytes, respectively, in network byte order. For major 443 type 7 and additional information value 25, 26, 27, these bytes 444 are not used as an integer argument, but as a floating-point value 445 (see Section 3.3). 447 28, 29, 30: These values are reserved for future additions to the 448 CBOR format. In the present version of CBOR, the encoded item is 449 not well-formed. 451 31: No argument value is derived. If the major type is 0, 1, or 6, 452 the encoded item is not well-formed. For major types 2 to 5, the 453 item's length is indefinite, and for major type 7, the byte does 454 not consitute a data item at all but terminates an indefinite 455 length item; both are described in Section 3.2. 457 The initial byte and any additional bytes consumed to construct the 458 argument are collectively referred to as the "head" of the data item. 460 The meaning of this argument depends on the major type. For example, 461 in major type 0, the argument is the value of the data item itself 462 (and in major type 1 the value of the data item is computed from the 463 argument); in major type 2 and 3 it gives the length of the string 464 data in bytes that follows; and in major types 4 and 5 it is used to 465 determine the number of data items enclosed. 467 If the encoded sequence of bytes ends before the end of a data item, 468 that item is not well-formed. If the encoded sequence of bytes still 469 has bytes remaining after the outermost encoded item is decoded, that 470 encoding is not a single well-formed CBOR item; depending on the 471 application, the decoder may either treat the encoding as not well- 472 formed or just identify the start of the remaining bytes to the 473 application. 475 A CBOR decoder implementation can be based on a jump table with all 476 256 defined values for the initial byte (Table 6). A decoder in a 477 constrained implementation can instead use the structure of the 478 initial byte and following bytes for more compact code (see 479 Appendix C for a rough impression of how this could look). 481 3.1. Major Types 483 The following lists the major types and the additional information 484 and other bytes associated with the type. 486 Major type 0: an integer in the range 0..2**64-1 inclusive. The 487 value of the encoded item is the argument itself. For example, 488 the integer 10 is denoted as the one byte 0b000_01010 (major type 489 0, additional information 10). The integer 500 would be 490 0b000_11001 (major type 0, additional information 25) followed by 491 the two bytes 0x01f4, which is 500 in decimal. 493 Major type 1: a negative integer in the range -2**64..-1 inclusive. 494 The value of the item is -1 minus the argument. For example, the 495 integer -500 would be 0b001_11001 (major type 1, additional 496 information 25) followed by the two bytes 0x01f3, which is 499 in 497 decimal. 499 Major type 2: a byte string. The number of bytes in the string is 500 equal to the argument. For example, a byte string whose length is 501 5 would have an initial byte of 0b010_00101 (major type 2, 502 additional information 5 for the length), followed by 5 bytes of 503 binary content. A byte string whose length is 500 would have 3 504 initial bytes of 0b010_11001 (major type 2, additional information 505 25 to indicate a two-byte length) followed by the two bytes 0x01f4 506 for a length of 500, followed by 500 bytes of binary content. 508 Major type 3: a text string (Section 2), encoded as UTF-8 509 ([RFC3629]). The number of bytes in the string is equal to the 510 argument. A string containing an invalid UTF-8 sequence is well- 511 formed but invalid. This type is provided for systems that need 512 to interpret or display human-readable text, and allows the 513 differentiation between unstructured bytes and text that has a 514 specified repertoire and encoding. In contrast to formats such as 515 JSON, the Unicode characters in this type are never escaped. 516 Thus, a newline character (U+000A) is always represented in a 517 string as the byte 0x0a, and never as the bytes 0x5c6e (the 518 characters "\" and "n") or as 0x5c7530303061 (the characters "\", 519 "u", "0", "0", "0", and "a"). 521 Major type 4: an array of data items. Arrays are also called lists, 522 sequences, or tuples. The argument is the number of data items in 523 the array. Items in an array do not need to all be of the same 524 type. For example, an array that contains 10 items of any type 525 would have an initial byte of 0b100_01010 (major type of 4, 526 additional information of 10 for the length) followed by the 10 527 remaining items. 529 Major type 5: a map of pairs of data items. Maps are also called 530 tables, dictionaries, hashes, or objects (in JSON). A map is 531 comprised of pairs of data items, each pair consisting of a key 532 that is immediately followed by a value. The argument is the 533 number of _pairs_ of data items in the map. For example, a map 534 that contains 9 pairs would have an initial byte of 0b101_01001 535 (major type of 5, additional information of 9 for the number of 536 pairs) followed by the 18 remaining items. The first item is the 537 first key, the second item is the first value, the third item is 538 the second key, and so on. Because items in a map come in pairs, 539 their total number is always even: A map that contains an odd 540 number of items (no value data present after the last key data 541 item) is not well-formed. A map that has duplicate keys may be 542 well-formed, but it is not valid, and thus it causes indeterminate 543 decoding; see also Section 5.6. 545 Major type 6: a tagged data item ("tag") whose tag number is the 546 argument and whose enclosed data item ("tag content") is the 547 single encoded data item that follows the head. See Section 3.4. 549 Major type 7: floating-point numbers and simple values, as well as 550 the "break" stop code. See Section 3.3. 552 These eight major types lead to a simple table showing which of the 553 256 possible values for the initial byte of a data item are used 554 (Table 6). 556 In major types 6 and 7, many of the possible values are reserved for 557 future specification. See Section 9 for more information on these 558 values. 560 Table 1 summarizes the major types defined by CBOR, ignoring the next 561 section for now. The number N in this table stands for the argument, 562 mt for the major type. 564 +----+-----------------------+---------------------------------+ 565 | mt | Meaning | Content | 566 +----+-----------------------+---------------------------------+ 567 | 0 | unsigned integer N | - | 568 | | | | 569 | 1 | negative integer -1-N | - | 570 | | | | 571 | 2 | byte string | N bytes | 572 | | | | 573 | 3 | text string | N bytes (UTF-8 text) | 574 | | | | 575 | 4 | array | N data items (elements) | 576 | | | | 577 | 5 | map | 2N data items (key/value pairs) | 578 | | | | 579 | 6 | tag of number N | 1 data item | 580 | | | | 581 | 7 | simple/float | - | 582 +----+-----------------------+---------------------------------+ 584 Table 1: Overview over CBOR major types (definite length encoded) 586 3.2. Indefinite Lengths for Some Major Types 588 Four CBOR items (arrays, maps, byte strings, and text strings) can be 589 encoded with an indefinite length using additional information value 590 31. This is useful if the encoding of the item needs to begin before 591 the number of items inside the array or map, or the total length of 592 the string, is known. (The application of this is often referred to 593 as "streaming" within a data item.) 595 Indefinite-length arrays and maps are dealt with differently than 596 indefinite-length byte strings and text strings. 598 3.2.1. The "break" Stop Code 600 The "break" stop code is encoded with major type 7 and additional 601 information value 31 (0b111_11111). It is not itself a data item: it 602 is just a syntactic feature to close an indefinite-length item. 604 If the "break" stop code appears anywhere where a data item is 605 expected, other than directly inside an indefinite-length string, 606 array, or map -- for example directly inside a definite-length array 607 or map -- the enclosing item is not well-formed. 609 3.2.2. Indefinite-Length Arrays and Maps 611 Indefinite-length arrays and maps are represented using their major 612 type with the additional information value of 31, followed by an 613 arbitrary-length sequence of zero or more items for an array or key/ 614 value pairs for a map, followed by the "break" stop code 615 (Section 3.2.1). In other words, indefinite-length arrays and maps 616 look identical to other arrays and maps except for beginning with the 617 additional information value of 31 and ending with the "break" stop 618 code. 620 If the break stop code appears after a key in a map, in place of that 621 key's value, the map is not well-formed. 623 There is no restriction against nesting indefinite-length array or 624 map items. A "break" only terminates a single item, so nested 625 indefinite-length items need exactly as many "break" stop codes as 626 there are type bytes starting an indefinite-length item. 628 For example, assume an encoder wants to represent the abstract array 629 [1, [2, 3], [4, 5]]. The definite-length encoding would be 630 0x8301820203820405: 632 83 -- Array of length 3 633 01 -- 1 634 82 -- Array of length 2 635 02 -- 2 636 03 -- 3 637 82 -- Array of length 2 638 04 -- 4 639 05 -- 5 641 Indefinite-length encoding could be applied independently to each of 642 the three arrays encoded in this data item, as required, leading to 643 representations such as: 645 0x9f018202039f0405ffff 646 9F -- Start indefinite-length array 647 01 -- 1 648 82 -- Array of length 2 649 02 -- 2 650 03 -- 3 651 9F -- Start indefinite-length array 652 04 -- 4 653 05 -- 5 654 FF -- "break" (inner array) 655 FF -- "break" (outer array) 657 0x9f01820203820405ff 658 9F -- Start indefinite-length array 659 01 -- 1 660 82 -- Array of length 2 661 02 -- 2 662 03 -- 3 663 82 -- Array of length 2 664 04 -- 4 665 05 -- 5 666 FF -- "break" 668 0x83018202039f0405ff 669 83 -- Array of length 3 670 01 -- 1 671 82 -- Array of length 2 672 02 -- 2 673 03 -- 3 674 9F -- Start indefinite-length array 675 04 -- 4 676 05 -- 5 677 FF -- "break" 679 0x83019f0203ff820405 680 83 -- Array of length 3 681 01 -- 1 682 9F -- Start indefinite-length array 683 02 -- 2 684 03 -- 3 685 FF -- "break" 686 82 -- Array of length 2 687 04 -- 4 688 05 -- 5 690 An example of an indefinite-length map (that happens to have two key/ 691 value pairs) might be: 693 0xbf6346756ef563416d7421ff 694 BF -- Start indefinite-length map 695 63 -- First key, UTF-8 string length 3 696 46756e -- "Fun" 697 F5 -- First value, true 698 63 -- Second key, UTF-8 string length 3 699 416d74 -- "Amt" 700 21 -- Second value, -2 701 FF -- "break" 703 3.2.3. Indefinite-Length Byte Strings and Text Strings 705 Indefinite-length strings are represented by a byte containing the 706 major type and additional information value of 31, followed by a 707 series of zero or more byte or text strings ("chunks") that have 708 definite lengths, followed by the "break" stop code (Section 3.2.1). 709 The data item represented by the indefinite-length string is the 710 concatenation of the chunks (i.e., the empty byte or text string, 711 respectively, if no chunk is present). 713 If any item between the indefinite-length string indicator 714 (0b010_11111 or 0b011_11111) and the "break" stop code is not a 715 definite-length string item of the same major type, the string is not 716 well-formed. 718 If any definite-length text string inside an indefinite-length text 719 string is invalid, the indefinite-length text string is invalid. 720 Note that this implies that the bytes of a single UTF-8 character 721 cannot be spread between chunks: a new chunk can only be started at a 722 character boundary. 724 For example, assume the sequence: 726 0b010_11111 0b010_00100 0xaabbccdd 0b010_00011 0xeeff99 0b111_11111 728 5F -- Start indefinite-length byte string 729 44 -- Byte string of length 4 730 aabbccdd -- Bytes content 731 43 -- Byte string of length 3 732 eeff99 -- Bytes content 733 FF -- "break" 735 After decoding, this results in a single byte string with seven 736 bytes: 0xaabbccddeeff99. 738 3.3. Floating-Point Numbers and Values with No Content 740 Major type 7 is for two types of data: floating-point numbers and 741 "simple values" that do not need any content. Each value of the 742 5-bit additional information in the initial byte has its own separate 743 meaning, as defined in Table 2. Like the major types for integers, 744 items of this major type do not carry content data; all the 745 information is in the initial bytes. 747 +------------+------------------------------------------------------+ 748 | 5-Bit | Semantics | 749 | Value | | 750 +------------+------------------------------------------------------+ 751 | 0..23 | Simple value (value 0..23) | 752 | | | 753 | 24 | Simple value (value 32..255 in following byte) | 754 | | | 755 | 25 | IEEE 754 Half-Precision Float (16 bits follow) | 756 | | | 757 | 26 | IEEE 754 Single-Precision Float (32 bits follow) | 758 | | | 759 | 27 | IEEE 754 Double-Precision Float (64 bits follow) | 760 | | | 761 | 28-30 | Reserved, not well-formed in the present document | 762 | | | 763 | 31 | "break" stop code for indefinite-length items | 764 | | (Section 3.2.1) | 765 +------------+------------------------------------------------------+ 767 Table 2: Values for Additional Information in Major Type 7 769 As with all other major types, the 5-bit value 24 signifies a single- 770 byte extension: it is followed by an additional byte to represent the 771 simple value. (To minimize confusion, only the values 32 to 255 are 772 used.) This maintains the structure of the initial bytes: as for the 773 other major types, the length of these always depends on the 774 additional information in the first byte. Table 3 lists the values 775 assigned and available for simple types. 777 +---------+-----------------+ 778 | Value | Semantics | 779 +---------+-----------------+ 780 | 0..19 | (Unassigned) | 781 | | | 782 | 20 | False | 783 | | | 784 | 21 | True | 785 | | | 786 | 22 | Null | 787 | | | 788 | 23 | Undefined value | 789 | | | 790 | 24..31 | (Reserved) | 791 | | | 792 | 32..255 | (Unassigned) | 793 +---------+-----------------+ 795 Table 3: Simple Values 797 An encoder MUST NOT issue two-byte sequences that start with 0xf8 798 (major type = 7, additional information = 24) and continue with a 799 byte less than 0x20 (32 decimal). Such sequences are not well- 800 formed. (This implies that an encoder cannot encode false, true, 801 null, or undefined in two-byte sequences, only the one-byte variants 802 of these are well-formed.) 804 The 5-bit values of 25, 26, and 27 are for 16-bit, 32-bit, and 64-bit 805 IEEE 754 binary floating-point values [IEEE754]. These floating- 806 point values are encoded in the additional bytes of the appropriate 807 size. (See Appendix D for some information about 16-bit floating 808 point.) 810 3.4. Tagging of Items 812 In CBOR, a data item can be enclosed by a tag to give it additional 813 semantics while retaining its structure. The tag is major type 6, 814 and represents an unsigned integer as indicated by the tag's argument 815 (Section 3); the (sole) enclosed data item is carried as content 816 data. If a tag requires structured data, this structure is encoded 817 into the nested data item. The definition of a tag number usually 818 restricts what kinds of nested data item or items are valid for tags 819 using this tag number. 821 For example, assume that a byte string of length 12 is marked with a 822 tag of number 2 to indicate it is a positive bignum (Section 3.4.4). 823 This would be marked as 0b110_00010 (major type 6, additional 824 information 2 for the tag number) followed by 0b010_01100 (major type 825 2, additional information of 12 for the length) followed by the 12 826 bytes of the bignum. 828 Decoders do not need to understand tags of every tag number, and tags 829 may be of little value in applications where the implementation 830 creating a particular CBOR data item and the implementation decoding 831 that stream know the semantic meaning of each item in the data flow. 832 Their primary purpose in this specification is to define common data 833 types such as dates. A secondary purpose is to provide conversion 834 hints when it is foreseen that the CBOR data item needs to be 835 translated into a different format, requiring hints about the content 836 of items. Understanding the semantics of tags is optional for a 837 decoder; it can just jump over the initial bytes of the tag (that 838 encode the tag number) and interpret the tag content itself, 839 presenting both tag number and tag content to the application. 841 A tag applies semantics to the data item it encloses. Thus, if tag A 842 encloses tag B, which encloses data item C, tag A applies to the 843 result of applying tag B on data item C. That is, a tag is a data 844 item consisting of a tag number and an enclosed value. The content 845 of the tag (the enclosed data item) is the data item (the value) that 846 is being tagged. 848 IANA maintains a registry of tag numbers as described in Section 9.2. 849 Table 4 provides a list of tag numbers that were defined in 850 [RFC7049], with definitions in the rest of this section. Note that 851 many other tag numbers have been defined since the publication of 852 [RFC7049]; see the registry described at Section 9.2 for the complete 853 list. 855 +----------+----------+---------------------------------------------+ 856 | Tag | Data | Semantics | 857 | Number | Item | | 858 +----------+----------+---------------------------------------------+ 859 | 0 | text | Standard date/time string; see | 860 | | string | Section 3.4.2 | 861 | | | | 862 | 1 | multiple | Epoch-based date/time; see Section 3.4.3 | 863 | | | | 864 | 2 | byte | Positive bignum; see Section 3.4.4 | 865 | | string | | 866 | | | | 867 | 3 | byte | Negative bignum; see Section 3.4.4 | 868 | | string | | 869 | | | | 870 | 4 | array | Decimal fraction; see Section 3.4.5 | 871 | | | | 872 | 5 | array | Bigfloat; see Section 3.4.5 | 873 | | | | 874 | 21 | multiple | Expected conversion to base64url encoding; | 875 | | | see Section 3.4.6.2 | 876 | | | | 877 | 22 | multiple | Expected conversion to base64 encoding; see | 878 | | | Section 3.4.6.2 | 879 | | | | 880 | 23 | multiple | Expected conversion to base16 encoding; see | 881 | | | Section 3.4.6.2 | 882 | | | | 883 | 24 | byte | Encoded CBOR data item; see Section 3.4.6.1 | 884 | | string | | 885 | | | | 886 | 32 | text | URI; see Section 3.4.6.3 | 887 | | string | | 888 | | | | 889 | 33 | text | base64url; see Section 3.4.6.3 | 890 | | string | | 891 | | | | 892 | 34 | text | base64; see Section 3.4.6.3 | 893 | | string | | 894 | | | | 895 | 35 | text | Regular expression; see Section 3.4.6.3 | 896 | | string | | 897 | | | | 898 | 36 | text | MIME message; see Section 3.4.6.3 | 899 | | string | | 900 | | | | 901 | 55799 | multiple | Self-described CBOR; see Section 3.4.7 | 902 +----------+----------+---------------------------------------------+ 904 Table 4: Tag numbers defined in RFC 7049 906 Conceptually, tags are interpreted in the generic data model, not at 907 (de-)serialization time. A small number of tags (specifically, tag 908 number 25 and tag number 29) have been registered with semantics that 909 do require processing at (de-)serialization time: The decoder needs 910 to be aware and the encoder needs to be under control of the exact 911 sequence in which data items are encoded into the CBOR data stream. 912 This means these tags cannot be implemented on top of every generic 913 CBOR encoder/decoder (which might not reflect the serialization order 914 for entries in a map at the data model level and vice versa); their 915 implementation therefore typically needs to be integrated into the 916 generic encoder/decoder. The definition of new tags with this 917 property is NOT RECOMMENDED. 919 3.4.1. Date and Time 921 Protocols using tag numbers 0 and 1 extend the generic data model 922 (Section 2) with data items representing points in time. 924 3.4.2. Standard Date/Time String 926 Tag number 0 contains a text string in the standard format described 927 by the "date-time" production in [RFC3339], as refined by Section 3.3 928 of [RFC4287], representing the point in time described there. A 929 nested item of another type or that doesn't match the [RFC4287] 930 format is invalid. 932 3.4.3. Epoch-based Date/Time 934 Tag number 1 contains a numerical value counting the number of 935 seconds from 1970-01-01T00:00Z in UTC time to the represented point 936 in civil time. 938 The enclosed item MUST be an unsigned or negative integer (major 939 types 0 and 1), or a floating-point number (major type 7 with 940 additional information 25, 26, or 27). Other contained types are 941 invalid. 943 Non-negative values (major type 0 and non-negative floating-point 944 numbers) stand for time values on or after 1970-01-01T00:00Z UTC and 945 are interpreted according to POSIX [TIME_T]. (POSIX time is also 946 known as UNIX Epoch time. Note that leap seconds are handled 947 specially by POSIX time and this results in a 1 second discontinuity 948 several times per decade.) Note that applications that require the 949 expression of times beyond early 2106 cannot leave out support of 950 64-bit integers for the enclosed value. 952 Negative values (major type 1 and negative floating-point numbers) 953 are interpreted as determined by the application requirements as 954 there is no universal standard for UTC count-of-seconds time before 955 1970-01-01T00:00Z (this is particularly true for points in time that 956 precede discontinuities in national calendars). The same applies to 957 non-finite values. 959 To indicate fractional seconds, floating-point values can be used 960 within tag number 1 instead of integer values. Note that this 961 generally requires binary64 support, as binary16 and binary32 provide 962 non-zero fractions of seconds only for a short period of time around 963 early 1970. An application that requires tag number 1 support may 964 restrict the enclosed value to be an integer (or a floating-point 965 value) only. 967 3.4.4. Bignums 969 Protocols using tag numbers 2 and 3 extend the generic data model 970 (Section 2) with "bignums" representing arbitrarily sized integers. 971 In the generic data model, bignum values are not equal to integers 972 from the basic data model, but specific data models can define that 973 equivalence, and preferred encoding never makes use of bignums that 974 also can be expressed as basic integers (see below). 976 Bignums are encoded as a byte string data item, which is interpreted 977 as an unsigned integer n in network byte order. Contained items of 978 other types are invalid. For tag number 2, the value of the bignum 979 is n. For tag number 3, the value of the bignum is -1 - n. The 980 preferred encoding of the byte string is to leave out any leading 981 zeroes (note that this means the preferred encoding for n = 0 is the 982 empty byte string, but see below). Decoders that understand these 983 tags MUST be able to decode bignums that do have leading zeroes. The 984 preferred encoding of an integer that can be represented using major 985 type 0 or 1 is to encode it this way instead of as a bignum (which 986 means that the empty string never occurs in a bignum when using 987 preferred encoding). Note that this means the non-preferred choice 988 of a bignum representation instead of a basic integer for encoding a 989 number is not intended to have application semantics (just as the 990 choice of a longer basic integer representation than needed, such as 991 0x1800 for 0x00 does not). 993 For example, the number 18446744073709551616 (2**64) is represented 994 as 0b110_00010 (major type 6, tag number 2), followed by 0b010_01001 995 (major type 2, length 9), followed by 0x010000000000000000 (one byte 996 0x01 and eight bytes 0x00). In hexadecimal: 998 C2 -- Tag 2 999 49 -- Byte string of length 9 1000 010000000000000000 -- Bytes content 1002 3.4.5. Decimal Fractions and Bigfloats 1004 Protocols using tag number 4 extend the generic data model with data 1005 items representing arbitrary-length decimal fractions of the form 1006 m*(10**e). Protocols using tag number 5 extend the generic data 1007 model with data items representing arbitrary-length binary fractions 1008 of the form m*(2**e). As with bignums, values of different types are 1009 not equal in the generic data model. 1011 Decimal fractions combine an integer mantissa with a base-10 scaling 1012 factor. They are most useful if an application needs the exact 1013 representation of a decimal fraction such as 1.1 because there is no 1014 exact representation for many decimal fractions in binary floating 1015 point. 1017 Bigfloats combine an integer mantissa with a base-2 scaling factor. 1018 They are binary floating-point values that can exceed the range or 1019 the precision of the three IEEE 754 formats supported by CBOR 1020 (Section 3.3). Bigfloats may also be used by constrained 1021 applications that need some basic binary floating-point capability 1022 without the need for supporting IEEE 754. 1024 A decimal fraction or a bigfloat is represented as a tagged array 1025 that contains exactly two integer numbers: an exponent e and a 1026 mantissa m. Decimal fractions (tag number 4) use base-10 exponents; 1027 the value of a decimal fraction data item is m*(10**e). Bigfloats 1028 (tag number 5) use base-2 exponents; the value of a bigfloat data 1029 item is m*(2**e). The exponent e MUST be represented in an integer 1030 of major type 0 or 1, while the mantissa also can be a bignum 1031 (Section 3.4.4). Contained items with other structures are invalid. 1033 An example of a decimal fraction is that the number 273.15 could be 1034 represented as 0b110_00100 (major type of 6 for the tag, additional 1035 information of 4 for the number of tag), followed by 0b100_00010 1036 (major type of 4 for the array, additional information of 2 for the 1037 length of the array), followed by 0b001_00001 (major type of 1 for 1038 the first integer, additional information of 1 for the value of -2), 1039 followed by 0b000_11001 (major type of 0 for the second integer, 1040 additional information of 25 for a two-byte value), followed by 1041 0b0110101010110011 (27315 in two bytes). In hexadecimal: 1043 C4 -- Tag 4 1044 82 -- Array of length 2 1045 21 -- -2 1046 19 6ab3 -- 27315 1048 An example of a bigfloat is that the number 1.5 could be represented 1049 as 0b110_00101 (major type of 6 for the tag, additional information 1050 of 5 for the number of tag), followed by 0b100_00010 (major type of 4 1051 for the array, additional information of 2 for the length of the 1052 array), followed by 0b001_00000 (major type of 1 for the first 1053 integer, additional information of 0 for the value of -1), followed 1054 by 0b000_00011 (major type of 0 for the second integer, additional 1055 information of 3 for the value of 3). In hexadecimal: 1057 C5 -- Tag 5 1058 82 -- Array of length 2 1059 20 -- -1 1060 03 -- 3 1062 Decimal fractions and bigfloats provide no representation of 1063 Infinity, -Infinity, or NaN; if these are needed in place of a 1064 decimal fraction or bigfloat, the IEEE 754 half-precision 1065 representations from Section 3.3 can be used. For constrained 1066 applications, where there is a choice between representing a specific 1067 number as an integer and as a decimal fraction or bigfloat (such as 1068 when the exponent is small and non-negative), there is a quality-of- 1069 implementation expectation that the integer representation is used 1070 directly. 1072 3.4.6. Content Hints 1074 The tags in this section are for content hints that might be used by 1075 generic CBOR processors. These content hints do not extend the 1076 generic data model. 1078 3.4.6.1. Encoded CBOR Data Item 1080 Sometimes it is beneficial to carry an embedded CBOR data item that 1081 is not meant to be decoded immediately at the time the enclosing data 1082 item is being decoded. Tag number 24 (CBOR data item) can be used to 1083 tag the embedded byte string as a data item encoded in CBOR format. 1084 Contained items that aren't byte strings are invalid. A contained 1085 byte string is valid if it encodes a well-formed CBOR item; validity 1086 checking of the decoded CBOR item is not required for tag validity 1087 (but could be offered by a generic decoder as a special option). 1089 3.4.6.2. Expected Later Encoding for CBOR-to-JSON Converters 1091 Tags number 21 to 23 indicate that a byte string might require a 1092 specific encoding when interoperating with a text-based 1093 representation. These tags are useful when an encoder knows that the 1094 byte string data it is writing is likely to be later converted to a 1095 particular JSON-based usage. That usage specifies that some strings 1096 are encoded as base64, base64url, and so on. The encoder uses byte 1097 strings instead of doing the encoding itself to reduce the message 1098 size, to reduce the code size of the encoder, or both. The encoder 1099 does not know whether or not the converter will be generic, and 1100 therefore wants to say what it believes is the proper way to convert 1101 binary strings to JSON. 1103 The data item tagged can be a byte string or any other data item. In 1104 the latter case, the tag applies to all of the byte string data items 1105 contained in the data item, except for those contained in a nested 1106 data item tagged with an expected conversion. 1108 These three tag numbers suggest conversions to three of the base data 1109 encodings defined in [RFC4648]. For base64url encoding (tag number 1110 21), padding is not used (see Section 3.2 of RFC 4648); that is, all 1111 trailing equals signs ("=") are removed from the encoded string. For 1112 base64 encoding (tag number 22), padding is used as defined in RFC 1113 4648. For both base64url and base64, padding bits are set to zero 1114 (see Section 3.5 of RFC 4648), and encoding is performed without the 1115 inclusion of any line breaks, whitespace, or other additional 1116 characters. Note that, for all three tag numbers, the encoding of 1117 the empty byte string is the empty text string. 1119 3.4.6.3. Encoded Text 1121 Some text strings hold data that have formats widely used on the 1122 Internet, and sometimes those formats can be validated and presented 1123 to the application in appropriate form by the decoder. There are 1124 tags for some of these formats. As with tag numbers 21 to 23, if 1125 these tags are applied to an item other than a text string, they 1126 apply to all text string data items it contains. 1128 o Tag number 32 is for URIs, as defined in [RFC3986]. If the text 1129 string doesn't match the "URI-reference" production, the string is 1130 invalid. 1132 o Tag numbers 33 and 34 are for base64url- and base64-encoded text 1133 strings, as defined in [RFC4648]. If any of: 1135 * the encoded text string contains non-alphabet characters or 1136 only 1 character in the last block of 4, or 1138 * the padding bits in a 2- or 3-character block are not 0, or 1140 * the base64 encoding has the wrong number of padding characters, 1141 or 1143 * the base64url encoding has padding characters, 1145 the string is invalid. 1147 o Tag number 35 is for regular expressions that are roughly in Perl 1148 Compatible Regular Expressions (PCRE/PCRE2) form [PCRE] or a 1149 version of the JavaScript regular expression syntax [ECMA262]. 1150 (Note that more specific identification may be necessary if the 1151 actual version of the specification underlying the regular 1152 expression, or more than just the text of the regular expression 1153 itself, need to be conveyed.) Any contained string value is 1154 valid. 1156 o Tag number 36 is for MIME messages (including all headers), as 1157 defined in [RFC2045]. A text string that isn't a valid MIME 1158 message is invalid. (For this tag, validity checking may be 1159 particularly onerous for a generic decoder and might therefore not 1160 be offered. Note that many MIME messages are general binary data 1161 and can therefore not be represented in a text string; 1162 [IANA.cbor-tags] lists a registration for tag number 257 that is 1163 similar to tag number 36 but is used with an enclosed byte 1164 string.) 1166 Note that tag numbers 33 and 34 differ from 21 and 22 in that the 1167 data is transported in base-encoded form for the former and in raw 1168 byte string form for the latter. 1170 3.4.7. Self-Described CBOR 1172 In many applications, it will be clear from the context that CBOR is 1173 being employed for encoding a data item. For instance, a specific 1174 protocol might specify the use of CBOR, or a media type is indicated 1175 that specifies its use. However, there may be applications where 1176 such context information is not available, such as when CBOR data is 1177 stored in a file that does not have disambiguating metadata. Here, 1178 it may help to have some distinguishing characteristics for the data 1179 itself. 1181 Tag number 55799 is defined for this purpose. It does not impart any 1182 special semantics on the data item that it encloses; that is, the 1183 semantics of a data item enclosed in tag number 55799 is exactly 1184 identical to the semantics of the data item itself. 1186 The serialization of this tag's head is 0xd9d9f7, which does not 1187 appear to be in use as a distinguishing mark for any frequently used 1188 file types. In particular, 0xd9d9f7 is not a valid start of a 1189 Unicode text in any Unicode encoding if it is followed by a valid 1190 CBOR data item. 1192 For instance, a decoder might be able to decode both CBOR and JSON. 1193 Such a decoder would need to mechanically distinguish the two 1194 formats. An easy way for an encoder to help the decoder would be to 1195 tag the entire CBOR item with tag number 55799, the serialization of 1196 which will never be found at the beginning of a JSON text. 1198 4. Serialization Considerations 1200 4.1. Preferred Serialization 1202 For some values at the data model level, CBOR provides multiple 1203 serializations. For many applications, it is desirable that an 1204 encoder always chooses a preferred serialization (preferred 1205 encoding); however, the present specification does not put the burden 1206 of enforcing this preference on either encoder or decoder. 1208 Some constrained decoders may be limited in their ability to decode 1209 non-preferred serializations: For example, if only integers below 1210 1_000_000_000 are expected in an application, the decoder may leave 1211 out the code that would be needed to decode 64-bit arguments in 1212 integers. An encoder that always uses preferred serialization 1213 ("preferred encoder") interoperates with this decoder for the numbers 1214 that can occur in this application. More generally speaking, it 1215 therefore can be said that a preferred encoder is more universally 1216 interoperable (and also less wasteful) than one that, say, always 1217 uses 64-bit integers. 1219 Similarly, a constrained encoder may be limited in the variety of 1220 representation variants it supports in such a way that it does not 1221 emit preferred serializations ("variant encoder"): Say, it could be 1222 designed to always use the 32-bit variant for an integer that it 1223 encodes even if a short representation is available (again, assuming 1224 that there is no application need for integers that can only be 1225 represented with the 64-bit variant). A decoder that does not rely 1226 on only ever receiving preferred serializations ("variation-tolerant 1227 decoder") can there be said to be more universally interoperable (it 1228 might very well optimize for the case of receiving preferred 1229 serializations, though). Full implementations of CBOR decoders are 1230 by definition variation-tolerant; the distinction is only relevant if 1231 a constrained implementation of a CBOR decoder meets a variant 1232 encoder. 1234 The preferred serialization always uses the shortest form of 1235 representing the argument (Section 3)); it also uses the shortest 1236 floating-point encoding that preserves the value being encoded (see 1237 Section 5.5). Definite length encoding is preferred whenever the 1238 length is known at the time the serialization of the item starts. 1240 4.2. Deterministically Encoded CBOR 1242 Some protocols may want encoders to only emit CBOR in a particular 1243 deterministic format; those protocols might also have the decoders 1244 check that their input is in that deterministic format. Those 1245 protocols are free to define what they mean by a "deterministic 1246 format" and what encoders and decoders are expected to do. This 1247 section defines a set of restrictions that can serve as the base of 1248 such a deterministic format. 1250 4.2.1. Core Deterministic Encoding Requirements 1252 A CBOR encoding satisfies the "core deterministic encoding 1253 requirements" if it satisfies the following restrictions: 1255 o Preferred serialization MUST be used. In particular, this means 1256 that arguments (see Section 3) for integers, lengths in major 1257 types 2 through 5, and tags MUST be as short as possible, for 1258 instance: 1260 * 0 to 23 and -1 to -24 MUST be expressed in the same byte as the 1261 major type; 1263 * 24 to 255 and -25 to -256 MUST be expressed only with an 1264 additional uint8_t; 1266 * 256 to 65535 and -257 to -65536 MUST be expressed only with an 1267 additional uint16_t; 1269 * 65536 to 4294967295 and -65537 to -4294967296 MUST be expressed 1270 only with an additional uint32_t. 1272 Floating point values also MUST use the shortest form that 1273 preserves the value, e.g. 1.5 is encoded as 0xf93e00 and 1000000.5 1274 as 0xfa49742408. 1276 o Indefinite-length items MUST NOT appear. They can be encoded as 1277 definite-length items instead. 1279 o The keys in every map MUST be sorted in the bytewise lexicographic 1280 order of their deterministic encodings. For example, the 1281 following keys are sorted correctly: 1283 1. 10, encoded as 0x0a. 1285 2. 100, encoded as 0x1864. 1287 3. -1, encoded as 0x20. 1289 4. "z", encoded as 0x617a. 1291 5. "aa", encoded as 0x626161. 1293 6. [100], encoded as 0x811864. 1295 7. [-1], encoded as 0x8120. 1297 8. false, encoded as 0xf4. 1299 4.2.2. Additional Deterministic Encoding Considerations 1301 If a protocol allows for IEEE floats, then additional deterministic 1302 encoding rules might need to be added. One example rule might be to 1303 have all floats start as a 64-bit float, then do a test conversion to 1304 a 32-bit float; if the result is the same numeric value, use the 1305 shorter value and repeat the process with a test conversion to a 1306 16-bit float. (This rule selects 16-bit float for positive and 1307 negative Infinity as well.) Although IEEE floats can represent both 1308 positive and negative zero as distinct values, the application might 1309 not distinguish these and might decide to represent all zero values 1310 with a positive sign, disallowing negative zero. 1312 CBOR tags present additional considerations for deterministic 1313 encoding. If a CBOR-based protocol were to provide the same 1314 semantics for the presence and absence of a specific tag (e.g., by 1315 allowing both tag 1 data items and raw numbers in a date/time 1316 position, treating the latter as if they were tagged), the 1317 deterministic format would not allow them. In a protocol that 1318 requires tags in certain places to obtain specific semantics, the tag 1319 needs to appear in the deterministic format as well. 1321 Protocols that include floating, big integer, or other complex values 1322 need to define extra requirements on their deterministic encodings. 1323 For example: 1325 o If a protocol includes a field that can express floating-point 1326 values (Section 3.3), the protocol's deterministic encoding needs 1327 to specify whether the integer 1.0 is encoded as 0x01, 0xf93c00, 1328 0xfa3f800000, or 0xfb3ff0000000000000. Three sensible rules for 1329 this are: 1331 1. Encode integral values that fit in 64 bits as values from 1332 major types 0 and 1, and other values as the smallest of 16-, 1333 32-, or 64-bit floating point that accurately represents the 1334 value, 1336 2. Encode all values as the smallest of 16-, 32-, or 64-bit 1337 floating point that accurately represents the value, even for 1338 integral values, or 1340 3. Encode all values as 64-bit floating point. 1342 Rule 1 straddles the boundaries between integers and floating 1343 point values, and Rule 3 does not use preferred encoding, so Rule 1344 2 may be a good choice in many cases. 1346 If NaN is an allowed value and there is no intent to support NaN 1347 payloads or signaling NaNs, the protocol needs to pick a single 1348 representation, for example 0xf97e00. If that simple choice is 1349 not possible, specific attention will be needed for NaN handling. 1351 Subnormal numbers (nonzero numbers with the lowest possible 1352 exponent of a given IEEE 754 number format) may be flushed to zero 1353 outputs or be treated as zero inputs in some floating point 1354 implementations. A protocol's deterministic encoding may want to 1355 exclude them from interchange, interchanging zero instead. 1357 o If a protocol includes a field that can express integers with an 1358 absolute value of 2^64 or larger using tag numbers 2 or 3 1359 (Section 3.4.4), the protocol's deterministic encoding needs to 1360 specify whether small integers are expressed using the tag or 1361 major types 0 and 1. 1363 o A protocol might give encoders the choice of representing a URL as 1364 either a text string or, using Section 3.4.6.3, tag number 32 1365 containing a text string. This protocol's deterministic encoding 1366 needs to either require that the tag is present or require that 1367 it's absent, not allow either one. 1369 4.2.3. Length-first map key ordering 1371 The core deterministic encoding requirements sort map keys in a 1372 different order from the one suggested by Section 3.9 of [RFC7049] 1373 (called "Canonical CBOR" there). Protocols that need to be 1374 compatible with [RFC7049]'s order can instead be specified in terms 1375 of this specification's "length-first core deterministic encoding 1376 requirements": 1378 A CBOR encoding satisfies the "length-first core deterministic 1379 encoding requirements" if it satisfies the core deterministic 1380 encoding requirements except that the keys in every map MUST be 1381 sorted such that: 1383 1. If two keys have different lengths, the shorter one sorts 1384 earlier; 1386 2. If two keys have the same length, the one with the lower value in 1387 (byte-wise) lexical order sorts earlier. 1389 For example, under the length-first core deterministic encoding 1390 requirements, the following keys are sorted correctly: 1392 1. 10, encoded as 0x0a. 1394 2. -1, encoded as 0x20. 1396 3. false, encoded as 0xf4. 1398 4. 100, encoded as 0x1864. 1400 5. "z", encoded as 0x617a. 1402 6. [-1], encoded as 0x8120. 1404 7. "aa", encoded as 0x626161. 1406 8. [100], encoded as 0x811864. 1408 (Although [RFC7049] used the term "Canonical CBOR" for its form of 1409 requirements on deterministic encoding, this document avoids this 1410 term because "canonicalization" is often associated with specific 1411 uses of deterministic encoding only. The terms are essentially 1412 interchangeable, however, and the set of core requirements in this 1413 document could also be called "Canonical CBOR", while the length- 1414 first-ordered version of that could be called "Old Canonical CBOR".) 1416 5. Creating CBOR-Based Protocols 1418 Data formats such as CBOR are often used in environments where there 1419 is no format negotiation. A specific design goal of CBOR is to not 1420 need any included or assumed schema: a decoder can take a CBOR item 1421 and decode it with no other knowledge. 1423 Of course, in real-world implementations, the encoder and the decoder 1424 will have a shared view of what should be in a CBOR data item. For 1425 example, an agreed-to format might be "the item is an array whose 1426 first value is a UTF-8 string, second value is an integer, and 1427 subsequent values are zero or more floating-point numbers" or "the 1428 item is a map that has byte strings for keys and contains at least 1429 one pair whose key is 0xab01". 1431 CBOR-based protocols MUST specify how their decoders handle invalid 1432 and other unexpected data. CBOR-based protocols MAY specify that 1433 they treat arbitrary valid data as unexpected. Encoders for CBOR- 1434 based protocols MUST produce only valid items, that is, the protocol 1435 cannot be designed to make use of invalid items. An encoder can be 1436 capable of encoding as many or as few types of values as is required 1437 by the protocol in which it is used; a decoder can be capable of 1438 understanding as many or as few types of values as is required by the 1439 protocols in which it is used. This lack of restrictions allows CBOR 1440 to be used in extremely constrained environments. 1442 This section discusses some considerations in creating CBOR-based 1443 protocols. With few exceptions, it is advisory only and explicitly 1444 excludes any language from BCP 14 other than words that could be 1445 interpreted as "MAY" in the sense of BCP 14. The exceptions aim at 1446 facilitating interoperability of CBOR-based protocols while making 1447 use of a wide variety of both generic and application-specific 1448 encoders and decoders. 1450 5.1. CBOR in Streaming Applications 1452 In a streaming application, a data stream may be composed of a 1453 sequence of CBOR data items concatenated back-to-back. In such an 1454 environment, the decoder immediately begins decoding a new data item 1455 if data is found after the end of a previous data item. 1457 Not all of the bytes making up a data item may be immediately 1458 available to the decoder; some decoders will buffer additional data 1459 until a complete data item can be presented to the application. 1460 Other decoders can present partial information about a top-level data 1461 item to an application, such as the nested data items that could 1462 already be decoded, or even parts of a byte string that hasn't 1463 completely arrived yet. 1465 Note that some applications and protocols will not want to use 1466 indefinite-length encoding. Using indefinite-length encoding allows 1467 an encoder to not need to marshal all the data for counting, but it 1468 requires a decoder to allocate increasing amounts of memory while 1469 waiting for the end of the item. This might be fine for some 1470 applications but not others. 1472 5.2. Generic Encoders and Decoders 1474 A generic CBOR decoder can decode all well-formed CBOR data and 1475 present them to an application. See Appendix C. 1477 Even though CBOR attempts to minimize these cases, not all well- 1478 formed CBOR data is valid: for example, the encoded text string 1479 "0x62c0ae" does not contain valid UTF-8 and so is not a valid CBOR 1480 item. Also, specific tags may make semantic constraints that may be 1481 violated, such as a bignum tag enclosing another tag, or an instance 1482 of tag number 0 containing a byte string or a text string with 1483 contents that do not match [RFC3339]'s "date-time" production. There 1484 is no requirement that generic encoders and decoders make unnatural 1485 choices for their application interface to enable the processing of 1486 invalid data. Generic encoders and decoders are expected to forward 1487 simple values and tags even if their specific codepoints are not 1488 registered at the time the encoder/decoder is written (Section 5.4). 1490 Generic decoders provide ways to present well-formed CBOR values, 1491 both valid and invalid, to an application. The diagnostic notation 1492 (Section 8) may be used to present well-formed CBOR values to humans. 1494 Generic encoders provide an application interface that allows the 1495 application to specify any well-formed value, including simple values 1496 and tags unknown to the encoder. 1498 5.3. Validity of Items 1500 A well-formed but invalid CBOR data item presents a problem with 1501 interpreting the data encoded in it in the CBOR data model. A CBOR- 1502 based protocol could be specified in several layers, in which the 1503 lower layers don't process the semantics of some of the CBOR data 1504 they forward. These layers can't notice any validity errors in data 1505 they don't process and MUST forward that data as-is. The first layer 1506 that does process the semantics of an invalid CBOR item MUST take one 1507 of two choices: 1509 1. Replace the problematic item with an error marker and continue 1510 with the next item, or 1512 2. Issue an error and stop processing altogether. 1514 A CBOR-based protocol MUST specify which of these options its 1515 decoders take, for each kind of invalid item they might encounter. 1517 Such problems might occur at the basic validity level of CBOR or in 1518 the context of tags (tag validity). 1520 5.3.1. Basic validity 1522 Two kinds of validity errors can occur in the basic generic data 1523 model: 1525 Duplicate keys in a map: Generic decoders (Section 5.2) make data 1526 available to applications using the native CBOR data model. That 1527 data model includes maps (key-value mappings with unique keys), 1528 not multimaps (key-value mappings where multiple entries can have 1529 the same key). Thus, a generic decoder that gets a CBOR map item 1530 that has duplicate keys will decode to a map with only one 1531 instance of that key, or it might stop processing altogether. On 1532 the other hand, a "streaming decoder" may not even be able to 1533 notice (Section 5.6). 1535 Invalid UTF-8 string: A decoder might or might not want to verify 1536 that the sequence of bytes in a UTF-8 string (major type 3) is 1537 actually valid UTF-8 and react appropriately. 1539 5.3.2. Tag validity 1541 Two additional kinds of validity errors are introduced by adding tags 1542 to the basic generic data model: 1544 Inadmissible type for tag content: Tags (Section 3.4) specify what 1545 type of data item is supposed to be enclosed by the tag; for 1546 example, the tags for positive or negative bignums are supposed to 1547 be put on byte strings. A decoder that decodes the tagged data 1548 item into a native representation (a native big integer in this 1549 example) is expected to check the type of the data item being 1550 tagged. Even decoders that don't have such native representations 1551 available in their environment may perform the check on those tags 1552 known to them and react appropriately. 1554 Inadmissible value for tag content: The type of data item may be 1555 admissible for a tag's content, but the specific value may not be; 1556 e.g., a value of "yesterday" is not acceptable for the content of 1557 tag 0, even though it properly is a text string. A decoder that 1558 normally ingests such tags into equivalent platform types might 1559 present this tag to the application in a similar way to how it 1560 would present a tag with an unknown tag number (Section 5.4). 1562 5.4. Validity and Evolution 1564 A decoder with validity checking will expend the effort to reliably 1565 detect data items with validity errors. For example, such a decoder 1566 needs to have an API that reports an error (and does not return data) 1567 for a CBOR data item that contains any of the validity errors listed 1568 in the previous subsection. 1570 The set of tags defined in the tag registry (Section 9.2), as well as 1571 the set of simple values defined in the simple values registry 1572 (Section 9.1), can grow at any time beyond the set understood by a 1573 generic decoder. A validity-checking decoder can do one of two 1574 things when it encounters such a case that it does not recognize: 1576 o It can report an error (and not return data). Note that this 1577 error is not a validity error per se. This kind of error is more 1578 likely to be raised by a decoder that would be performing validity 1579 checking if this were a known case. 1581 o It can emit the unknown item (type, value, and, for tags, the 1582 decoded tagged data item) to the application calling the decoder, 1583 with an indication that the decoder did not recognize that tag 1584 number or simple value. 1586 The latter approach, which is also appropriate for decoders that do 1587 not support validity checking, provides forward compatibility with 1588 newly registered tags and simple values without the requirement to 1589 update the encoder at the same time as the calling application. (For 1590 this, the API for the decoder needs to have a way to mark unknown 1591 items so that the calling application can handle them in a manner 1592 appropriate for the program.) 1594 Since some of the processing needed for validity checking may have an 1595 appreciable cost (in particular with duplicate detection for maps), 1596 support of validity checking is not a requirement placed on all CBOR 1597 decoders. 1599 Some encoders will rely on their applications to provide input data 1600 in such a way that valid CBOR results from the encoder. A generic 1601 encoder also may want to provide a validity-checking mode where it 1602 reliably limits its output to valid CBOR, independent of whether or 1603 not its application is indeed providing API-conformant data. 1605 5.5. Numbers 1607 CBOR-based protocols should take into account that different language 1608 environments pose different restrictions on the range and precision 1609 of numbers that are representable. For example, the JavaScript 1610 number system treats all numbers as floating point, which may result 1611 in silent loss of precision in decoding integers with more than 53 1612 significant bits. A protocol that uses numbers should define its 1613 expectations on the handling of non-trivial numbers in decoders and 1614 receiving applications. 1616 A CBOR-based protocol that includes floating-point numbers can 1617 restrict which of the three formats (half-precision, single- 1618 precision, and double-precision) are to be supported. For an 1619 integer-only application, a protocol may want to completely exclude 1620 the use of floating-point values. 1622 A CBOR-based protocol designed for compactness may want to exclude 1623 specific integer encodings that are longer than necessary for the 1624 application, such as to save the need to implement 64-bit integers. 1625 There is an expectation that encoders will use the most compact 1626 integer representation that can represent a given value. However, a 1627 compact application should accept values that use a longer-than- 1628 needed encoding (such as encoding "0" as 0b000_11001 followed by two 1629 bytes of 0x00) as long as the application can decode an integer of 1630 the given size. 1632 The preferred encoding for a floating-point value is the shortest 1633 floating-point encoding that preserves its value, e.g., 0xf94580 for 1634 the number 5.5, and 0xfa45ad9c00 for the number 5555.5, unless the 1635 CBOR-based protocol specifically excludes the use of the shorter 1636 floating-point encodings. For NaN values, a shorter encoding is 1637 preferred if zero-padding the shorter significand towards the right 1638 reconstitutes the original NaN value (for many applications, the 1639 single NaN encoding 0xf97e00 will suffice). 1641 5.6. Specifying Keys for Maps 1643 The encoding and decoding applications need to agree on what types of 1644 keys are going to be used in maps. In applications that need to 1645 interwork with JSON-based applications, keys probably should be 1646 limited to UTF-8 strings only; otherwise, there has to be a specified 1647 mapping from the other CBOR types to Unicode characters, and this 1648 often leads to implementation errors. In applications where keys are 1649 numeric in nature and numeric ordering of keys is important to the 1650 application, directly using the numbers for the keys is useful. 1652 If multiple types of keys are to be used, consideration should be 1653 given to how these types would be represented in the specific 1654 programming environments that are to be used. For example, in 1655 JavaScript Maps [ECMA262], a key of integer 1 cannot be distinguished 1656 from a key of floating-point 1.0. This means that, if integer keys 1657 are used, the protocol needs to avoid use of floating-point keys the 1658 values of which happen to be integer numbers in the same map. 1660 Decoders that deliver data items nested within a CBOR data item 1661 immediately on decoding them ("streaming decoders") often do not keep 1662 the state that is necessary to ascertain uniqueness of a key in a 1663 map. Similarly, an encoder that can start encoding data items before 1664 the enclosing data item is completely available ("streaming encoder") 1665 may want to reduce its overhead significantly by relying on its data 1666 source to maintain uniqueness. 1668 A CBOR-based protocol MUST define what to do when a receiving 1669 application does see multiple identical keys in a map. The resulting 1670 rule in the protocol MUST respect the CBOR data model: it cannot 1671 prescribe a specific handling of the entries with the identical keys, 1672 except that it might have a rule that having identical keys in a map 1673 indicates a malformed map and that the decoder has to stop with an 1674 error. Duplicate keys are also prohibited by CBOR decoders that 1675 enforce validity (Section 5.4). 1677 The CBOR data model for maps does not allow ascribing semantics to 1678 the order of the key/value pairs in the map representation. Thus, a 1679 CBOR-based protocol MUST NOT specify that changing the key/value pair 1680 order in a map would change the semantics, except to specify that 1681 some, orders are disallowed, for example where they would not meet 1682 the requirements of a deterministic encoding (Section 4.2). (Any 1683 secondary effects of map ordering such as on timing, cache usage, and 1684 other potential side channels are not considered part of the 1685 semantics but may be enough reason on its own for a protocol to 1686 require a deterministic encoding format.) 1688 Applications for constrained devices that have maps where a small 1689 number of frequently used keys can be identified should consider 1690 using small integers as keys; for instance, a set of 24 or fewer 1691 frequent keys can be encoded in a single byte as unsigned integers, 1692 up to 48 if negative integers are also used. Less frequently 1693 occurring keys can then use integers with longer encodings. 1695 5.6.1. Equivalence of Keys 1697 The specific data model applying to a CBOR data item is used to 1698 determine whether keys occurring in maps are duplicates or distinct. 1700 At the generic data model level, numerically equivalent integer and 1701 floating-point values are distinct from each other, as they are from 1702 the various big numbers (Tags 2 to 5). Similarly, text strings are 1703 distinct from byte strings, even if composed of the same bytes. A 1704 tagged value is distinct from an untagged value or from a value 1705 tagged with a different tag number. 1707 Within each of these groups, numeric values are distinct unless they 1708 are numerically equal (specifically, -0.0 is equal to 0.0); for the 1709 purpose of map key equivalence, NaN (not a number) values are 1710 equivalent if they have the same significand after zero-extending 1711 both significands at the right to 64 bits. 1713 (Byte and text) strings are compared byte by byte, arrays element by 1714 element, and are equal if they have the same number of bytes/elements 1715 and the same values at the same positions. Two maps are equal if 1716 they have the same set of pairs regardless of their order; pairs are 1717 equal if both the key and value are equal. 1719 Tagged values are equal if both the tag number and the enclosed item 1720 are equal. (Note that a generic decoder that provides processing for 1721 a specific tag may not be able to distinguish some semantically 1722 equivalent values, e.g. if leading zeroes occur in the content of tag 1723 2/3 (Section 3.4.4).) Simple values are equal if they simply have 1724 the same value. Nothing else is equal in the generic data model, a 1725 simple value 2 is not equivalent to an integer 2 and an array is 1726 never equivalent to a map. 1728 As discussed in Section 2.2, specific data models can make values 1729 equivalent for the purpose of comparing map keys that are distinct in 1730 the generic data model. Note that this implies that a generic 1731 decoder may deliver a decoded map to an application that needs to be 1732 checked for duplicate map keys by that application (alternatively, 1733 the decoder may provide a programming interface to perform this 1734 service for the application). Specific data models cannot 1735 distinguish values for map keys that are equal for this purpose at 1736 the generic data model level. 1738 5.7. Undefined Values 1740 In some CBOR-based protocols, the simple value (Section 3.3) of 1741 Undefined might be used by an encoder as a substitute for a data item 1742 with an encoding problem, in order to allow the rest of the enclosing 1743 data items to be encoded without harm. 1745 6. Converting Data between CBOR and JSON 1747 This section gives non-normative advice about converting between CBOR 1748 and JSON. Implementations of converters are free to use whichever 1749 advice here they want. 1751 It is worth noting that a JSON text is a sequence of characters, not 1752 an encoded sequence of bytes, while a CBOR data item consists of 1753 bytes, not characters. 1755 6.1. Converting from CBOR to JSON 1757 Most of the types in CBOR have direct analogs in JSON. However, some 1758 do not, and someone implementing a CBOR-to-JSON converter has to 1759 consider what to do in those cases. The following non-normative 1760 advice deals with these by converting them to a single substitute 1761 value, such as a JSON null. 1763 o An integer (major type 0 or 1) becomes a JSON number. 1765 o A byte string (major type 2) that is not embedded in a tag that 1766 specifies a proposed encoding is encoded in base64url without 1767 padding and becomes a JSON string. 1769 o A UTF-8 string (major type 3) becomes a JSON string. Note that 1770 JSON requires escaping certain characters ([RFC8259], Section 7): 1771 quotation mark (U+0022), reverse solidus (U+005C), and the "C0 1772 control characters" (U+0000 through U+001F). All other characters 1773 are copied unchanged into the JSON UTF-8 string. 1775 o An array (major type 4) becomes a JSON array. 1777 o A map (major type 5) becomes a JSON object. This is possible 1778 directly only if all keys are UTF-8 strings. A converter might 1779 also convert other keys into UTF-8 strings (such as by converting 1780 integers into strings containing their decimal representation); 1781 however, doing so introduces a danger of key collision. Note also 1782 that, if tags on UTF-8 strings are ignored as proposed below, this 1783 will cause a key collision if the tags are different but the 1784 strings are the same. 1786 o False (major type 7, additional information 20) becomes a JSON 1787 false. 1789 o True (major type 7, additional information 21) becomes a JSON 1790 true. 1792 o Null (major type 7, additional information 22) becomes a JSON 1793 null. 1795 o A floating-point value (major type 7, additional information 25 1796 through 27) becomes a JSON number if it is finite (that is, it can 1797 be represented in a JSON number); if the value is non-finite (NaN, 1798 or positive or negative Infinity), it is represented by the 1799 substitute value. 1801 o Any other simple value (major type 7, any additional information 1802 value not yet discussed) is represented by the substitute value. 1804 o A bignum (major type 6, tag number 2 or 3) is represented by 1805 encoding its byte string in base64url without padding and becomes 1806 a JSON string. For tag number 3 (negative bignum), a "~" (ASCII 1807 tilde) is inserted before the base-encoded value. (The conversion 1808 to a binary blob instead of a number is to prevent a likely 1809 numeric overflow for the JSON decoder.) 1811 o A byte string with an encoding hint (major type 6, tag number 21 1812 through 23) is encoded as described and becomes a JSON string. 1814 o For all other tags (major type 6, any other tag number), the 1815 enclosed CBOR item is represented as a JSON value; the tag number 1816 is ignored. 1818 o Indefinite-length items are made definite before conversion. 1820 6.2. Converting from JSON to CBOR 1822 All JSON values, once decoded, directly map into one or more CBOR 1823 values. As with any kind of CBOR generation, decisions have to be 1824 made with respect to number representation. In a suggested 1825 conversion: 1827 o JSON numbers without fractional parts (integer numbers) are 1828 represented as integers (major types 0 and 1, possibly major type 1829 6 tag number 2 and 3), choosing the shortest form; integers longer 1830 than an implementation-defined threshold may instead be 1831 represented as floating-point values. The default range that is 1832 represented as integer is -2**53+1..2**53-1 (fully exploiting the 1833 range for exact integers in the binary64 representation often used 1834 for decoding JSON [RFC7493]), implementations may choose 1835 -2**32..2**32-1 or -2**64..2**64-1 (fully using the integer ranges 1836 available in CBOR with uint32_t or uint64_t, respectively) or even 1837 -2**31..2**31-1 or -2**63..2**63-1 (using popular ranges for two's 1838 complement signed integers). (If the JSON was generated from a 1839 JavaScript implementation, its precision is already limited to 53 1840 bits maximum.) 1842 o Numbers with fractional parts are represented as floating-point 1843 values, performing the decimal-to-binary conversion based on the 1844 precision provided by IEEE 754 binary64. Then, when encoding in 1845 CBOR, the preferred serialization uses the shortest floating-point 1846 representation exactly representing this conversion result; for 1847 instance, 1.5 is represented in a 16-bit floating-point value (not 1848 all implementations will be capable of efficiently finding the 1849 minimum form, though). Instead of using the default binary64 1850 precision, there may be an implementation-defined limit to the 1851 precision of the conversion that will affect the precision of the 1852 represented values. Decimal representation should only be used on 1853 the CBOR side if that is specified in a protocol. 1855 CBOR has been designed to generally provide a more compact encoding 1856 than JSON. One implementation strategy that might come to mind is to 1857 perform a JSON-to-CBOR encoding in place in a single buffer. This 1858 strategy would need to carefully consider a number of pathological 1859 cases, such as that some strings represented with no or very few 1860 escapes and longer (or much longer) than 255 bytes may expand when 1861 encoded as UTF-8 strings in CBOR. Similarly, a few of the binary 1862 floating-point representations might cause expansion from some short 1863 decimal representations (1.1, 1e9) in JSON. This may be hard to get 1864 right, and any ensuing vulnerabilities may be exploited by an 1865 attacker. 1867 7. Future Evolution of CBOR 1869 Successful protocols evolve over time. New ideas appear, 1870 implementation platforms improve, related protocols are developed and 1871 evolve, and new requirements from applications and protocols are 1872 added. Facilitating protocol evolution is therefore an important 1873 design consideration for any protocol development. 1875 For protocols that will use CBOR, CBOR provides some useful 1876 mechanisms to facilitate their evolution. Best practices for this 1877 are well known, particularly from JSON format development of JSON- 1878 based protocols. Therefore, such best practices are outside the 1879 scope of this specification. 1881 However, facilitating the evolution of CBOR itself is very well 1882 within its scope. CBOR is designed to both provide a stable basis 1883 for development of CBOR-based protocols and to be able to evolve. 1884 Since a successful protocol may live for decades, CBOR needs to be 1885 designed for decades of use and evolution. This section provides 1886 some guidance for the evolution of CBOR. It is necessarily more 1887 subjective than other parts of this document. It is also necessarily 1888 incomplete, lest it turn into a textbook on protocol development. 1890 7.1. Extension Points 1892 In a protocol design, opportunities for evolution are often included 1893 in the form of extension points. For example, there may be a 1894 codepoint space that is not fully allocated from the outset, and the 1895 protocol is designed to tolerate and embrace implementations that 1896 start using more codepoints than initially allocated. 1898 Sizing the codepoint space may be difficult because the range 1899 required may be hard to predict. An attempt should be made to make 1900 the codepoint space large enough so that it can slowly be filled over 1901 the intended lifetime of the protocol. 1903 CBOR has three major extension points: 1905 o the "simple" space (values in major type 7). Of the 24 efficient 1906 (and 224 slightly less efficient) values, only a small number have 1907 been allocated. Implementations receiving an unknown simple data 1908 item may be able to process it as such, given that the structure 1909 of the value is indeed simple. The IANA registry in Section 9.1 1910 is the appropriate way to address the extensibility of this 1911 codepoint space. 1913 o the "tag" space (values in major type 6). Again, only a small 1914 part of the codepoint space has been allocated, and the space is 1915 abundant (although the early numbers are more efficient than the 1916 later ones). Implementations receiving an unknown tag number can 1917 choose to simply ignore it or to process it as an unknown tag 1918 number wrapping the enclosed data item. The IANA registry in 1919 Section 9.2 is the appropriate way to address the extensibility of 1920 this codepoint space. 1922 o the "additional information" space. An implementation receiving 1923 an unknown additional information value has no way to continue 1924 decoding, so allocating codepoints to this space is a major step. 1925 There are also very few codepoints left. 1927 7.2. Curating the Additional Information Space 1929 The human mind is sometimes drawn to filling in little perceived gaps 1930 to make something neat. We expect the remaining gaps in the 1931 codepoint space for the additional information values to be an 1932 attractor for new ideas, just because they are there. 1934 The present specification does not manage the additional information 1935 codepoint space by an IANA registry. Instead, allocations out of 1936 this space can only be done by updating this specification. 1938 For an additional information value of n >= 24, the size of the 1939 additional data typically is 2**(n-24) bytes. Therefore, additional 1940 information values 28 and 29 should be viewed as candidates for 1941 128-bit and 256-bit quantities, in case a need arises to add them to 1942 the protocol. Additional information value 30 is then the only 1943 additional information value available for general allocation, and 1944 there should be a very good reason for allocating it before assigning 1945 it through an update of this protocol. 1947 8. Diagnostic Notation 1949 CBOR is a binary interchange format. To facilitate documentation and 1950 debugging, and in particular to facilitate communication between 1951 entities cooperating in debugging, this section defines a simple 1952 human-readable diagnostic notation. All actual interchange always 1953 happens in the binary format. 1955 Note that this truly is a diagnostic format; it is not meant to be 1956 parsed. Therefore, no formal definition (as in ABNF) is given in 1957 this document. (Implementers looking for a text-based format for 1958 representing CBOR data items in configuration files may also want to 1959 consider YAML [YAML].) 1961 The diagnostic notation is loosely based on JSON as it is defined in 1962 RFC 8259, extending it where needed. 1964 The notation borrows the JSON syntax for numbers (integer and 1965 floating point), True (>true<), False (>false<), Null (>null<), UTF-8 1966 strings, arrays, and maps (maps are called objects in JSON; the 1967 diagnostic notation extends JSON here by allowing any data item in 1968 the key position). Undefined is written >undefined< as in 1969 JavaScript. The non-finite floating-point numbers Infinity, 1970 -Infinity, and NaN are written exactly as in this sentence (this is 1971 also a way they can be written in JavaScript, although JSON does not 1972 allow them). A tag is written as an integer number for the tag 1973 number, followed by the tag content in parentheses; for instance, an 1974 RFC 3339 (ISO 8601) date could be notated as: 1976 0("2013-03-21T20:04:00Z") 1978 or the equivalent relative time as 1980 1(1363896240) 1982 Byte strings are notated in one of the base encodings, without 1983 padding, enclosed in single quotes, prefixed by >h< for base16, >b32< 1984 for base32, >h32< for base32hex, >b64< for base64 or base64url (the 1985 actual encodings do not overlap, so the string remains unambiguous). 1986 For example, the byte string 0x12345678 could be written h'12345678', 1987 b32'CI2FM6A', or b64'EjRWeA'. 1989 Unassigned simple values are given as "simple()" with the appropriate 1990 integer in the parentheses. For example, "simple(42)" indicates 1991 major type 7, value 42. 1993 8.1. Encoding Indicators 1995 Sometimes it is useful to indicate in the diagnostic notation which 1996 of several alternative representations were actually used; for 1997 example, a data item written >1.5< by a diagnostic decoder might have 1998 been encoded as a half-, single-, or double-precision float. 2000 The convention for encoding indicators is that anything starting with 2001 an underscore and all following characters that are alphanumeric or 2002 underscore, is an encoding indicator, and can be ignored by anyone 2003 not interested in this information. Encoding indicators are always 2004 optional. 2006 A single underscore can be written after the opening brace of a map 2007 or the opening bracket of an array to indicate that the data item was 2008 represented in indefinite-length format. For example, [_ 1, 2] 2009 contains an indicator that an indefinite-length representation was 2010 used to represent the data item [1, 2]. 2012 An underscore followed by a decimal digit n indicates that the 2013 preceding item (or, for arrays and maps, the item starting with the 2014 preceding bracket or brace) was encoded with an additional 2015 information value of 24+n. For example, 1.5_1 is a half-precision 2016 floating-point number, while 1.5_3 is encoded as double precision. 2017 This encoding indicator is not shown in Appendix A. (Note that the 2018 encoding indicator "_" is thus an abbreviation of the full form "_7", 2019 which is not used.) 2021 As a special case, byte and text strings of indefinite length can be 2022 notated in the form (_ h'0123', h'4567') and (_ "foo", "bar"). 2024 9. IANA Considerations 2026 IANA has created two registries for new CBOR values. The registries 2027 are separate, that is, not under an umbrella registry, and follow the 2028 rules in [RFC8126]. IANA has also assigned a new MIME media type and 2029 an associated Constrained Application Protocol (CoAP) Content-Format 2030 entry. 2032 9.1. Simple Values Registry 2034 IANA has created the "Concise Binary Object Representation (CBOR) 2035 Simple Values" registry at [IANA.cbor-simple-values]. The initial 2036 values are shown in Table 3. 2038 New entries in the range 0 to 19 are assigned by Standards Action. 2039 It is suggested that these Standards Actions allocate values starting 2040 with the number 16 in order to reserve the lower numbers for 2041 contiguous blocks (if any). 2043 New entries in the range 32 to 255 are assigned by Specification 2044 Required. 2046 9.2. Tags Registry 2048 IANA has created the "Concise Binary Object Representation (CBOR) 2049 Tags" registry at [IANA.cbor-tags]. The tags that were defined in 2050 [RFC7049] are described in detail in Section 3.4, but other tags have 2051 already been defined. 2053 New entries in the range 0 to 23 are assigned by Standards Action. 2054 New entries in the range 24 to 255 are assigned by Specification 2055 Required. New entries in the range 256 to 18446744073709551615 are 2056 assigned by First Come First Served. The template for registration 2057 requests is: 2059 o Data item 2061 o Semantics (short form) 2062 In addition, First Come First Served requests should include: 2064 o Point of contact 2066 o Description of semantics (URL) - This description is optional; the 2067 URL can point to something like an Internet-Draft or a web page. 2069 9.3. Media Type ("MIME Type") 2071 The Internet media type [RFC6838] for a single encoded CBOR data item 2072 is application/cbor. 2074 Type name: application 2076 Subtype name: cbor 2078 Required parameters: n/a 2080 Optional parameters: n/a 2082 Encoding considerations: binary 2084 Security considerations: See Section 10 of this document 2086 Interoperability considerations: n/a 2088 Published specification: This document 2090 Applications that use this media type: None yet, but it is expected 2091 that this format will be deployed in protocols and applications. 2093 Additional information: 2094 Magic number(s): n/a 2095 File extension(s): .cbor 2096 Macintosh file type code(s): n/a 2098 Person & email address to contact for further information: 2099 Carsten Bormann 2100 cabo@tzi.org 2102 Intended usage: COMMON 2104 Restrictions on usage: none 2106 Author: 2107 Carsten Bormann 2109 Change controller: 2110 The IESG 2112 9.4. CoAP Content-Format 2114 Media Type: application/cbor 2116 Encoding: - 2118 Id: 60 2120 Reference: [RFCthis] 2122 9.5. The +cbor Structured Syntax Suffix Registration 2124 Name: Concise Binary Object Representation (CBOR) 2126 +suffix: +cbor 2128 References: [RFCthis] 2130 Encoding Considerations: CBOR is a binary format. 2132 Interoperability Considerations: n/a 2133 Fragment Identifier Considerations: 2134 The syntax and semantics of fragment identifiers specified for 2135 +cbor SHOULD be as specified for "application/cbor". (At 2136 publication of this document, there is no fragment identification 2137 syntax defined for "application/cbor".) 2139 The syntax and semantics for fragment identifiers for a specific 2140 "xxx/yyy+cbor" SHOULD be processed as follows: 2142 For cases defined in +cbor, where the fragment identifier resolves 2143 per the +cbor rules, then process as specified in +cbor. 2145 For cases defined in +cbor, where the fragment identifier does 2146 not resolve per the +cbor rules, then process as specified in 2147 "xxx/yyy+cbor". 2149 For cases not defined in +cbor, then process as specified in 2150 "xxx/yyy+cbor". 2152 Security Considerations: See Section 10 of this document 2154 Contact: 2155 Apps Area Working Group (apps-discuss@ietf.org) 2157 Author/Change Controller: 2158 The Apps Area Working Group. 2159 The IESG has change control over this registration. 2161 10. Security Considerations 2163 A network-facing application can exhibit vulnerabilities in its 2164 processing logic for incoming data. Complex parsers are well known 2165 as a likely source of such vulnerabilities, such as the ability to 2166 remotely crash a node, or even remotely execute arbitrary code on it. 2167 CBOR attempts to narrow the opportunities for introducing such 2168 vulnerabilities by reducing parser complexity, by giving the entire 2169 range of encodable values a meaning where possible. 2171 Because CBOR decoders are often used as a first step in processing 2172 unvalidated input, they need to be fully prepared for all types of 2173 hostile input that may be designed to corrupt, overrun, or achieve 2174 control of the system decoding the CBOR data item. A CBOR decoder 2175 needs to assume that all input may be hostile even if it has been 2176 checked by a firewall, has come over a secure channel such as TLS, is 2177 encrypted or signed, or has come from some other source that is 2178 presumed trusted. 2180 Hostile input may be constructed to overrun buffers, overflow or 2181 underflow integer arithmetic, or cause other decoding disruption. 2182 CBOR data items might have lengths or sizes that are intentionally 2183 extremely large or too short. Resource exhaustion attacks might 2184 attempt to lure a decoder into allocating very big data items 2185 (strings, arrays, maps, or even arbitrary precision numbers) or 2186 exhaust the stack depth by setting up deeply nested items. Decoders 2187 need to have appropriate resource management to mitigate these 2188 attacks. (Items for which very large sizes are given can also 2189 attempt to exploit integer overflow vulnerabilities.) 2191 A CBOR decoder, by definition, only accepts well-formed CBOR; this is 2192 the first step to its robustness. Input that is not well-formed CBOR 2193 causes no further processing from the point where the lack of well- 2194 formedness was detected. If possible, any data decoded up to this 2195 point should have no impact on the application using the CBOR 2196 decoder. 2198 In addition to ascertaining well-formedness, a CBOR decoder might 2199 also perform validity checks on the CBOR data. Alternatively, it can 2200 leave those checks to the application using the decoder. This choice 2201 needs to be clearly documented in the decoder. Beyond the validity 2202 at the CBOR level, an application also needs to ascertain that the 2203 input is in alignment with the application protocol that is 2204 serialized in CBOR. 2206 The input check itself may consume resources. This is usually linear 2207 in the size of the input, which means that an attacker has to spend 2208 resources that are commensurate to the resources spent by the 2209 defender on input validation. Processing for arbitrary-precision 2210 numbers may exceed linear effort. Also, some hash-table 2211 implementations that are used by decoders to build in-memory 2212 representations of maps can be attacked to spend quadratic effort, 2213 unless a secret key is employed (see Section 7 of [SIPHASH]). Such 2214 superlinear efforts can be employed by an attacker to exhaust 2215 resources at or before the input validator; they therefore need to be 2216 avoided in a CBOR decoder implementation. Note that tag number 2217 definitions and their implementations can add security considerations 2218 of this kind; this should then be discussed in the security 2219 considerations of the tag number definition. 2221 CBOR encoders do not receive input directly from the network and are 2222 thus not directly attackable in the same way as CBOR decoders. 2223 However, CBOR encoders often have an API that takes input from 2224 another level in the implementation and can be attacked through that 2225 API. The design and implementation of that API should assume the 2226 behavior of its caller may be based on hostile input or on coding 2227 mistakes. It should check inputs for buffer overruns, overflow and 2228 underflow of integer arithmetic, and other such errors that are aimed 2229 to disrupt the encoder. 2231 Protocols should be defined in such a way that potential multiple 2232 interpretations are reliably reduced to a single interpretation. For 2233 example, an attacker could make use of invalid input such as 2234 duplicate keys in maps, or exploit different precision in processing 2235 numbers to make one application base its decisions on a different 2236 interpretation than the one that will be used by a second 2237 application. To facilitate consistent interpretation, encoder and 2238 decoder implementations should provide a validity checking mode of 2239 operation (Section 5.4). Note, however, that a generic decoder 2240 cannot know about all requirements that an application poses on its 2241 input data; it is therefore not relieving the application from 2242 performing its own input checking. Also, since the set of defined 2243 tag numbers evolves, the application may employ a tag number that is 2244 not yet supported for validity checking by the generic decoder it 2245 uses. Generic decoders therefore need to provide documentation which 2246 tag numbers they support and what validity checking they can provide 2247 for each of them as well as for basic CBOR validity (UTF-8 checking, 2248 duplicate map key checking). 2250 11. References 2252 11.1. Normative References 2254 [ECMA262] Ecma International, "ECMAScript 2018 Language 2255 Specification", ECMA Standard ECMA-262, 9th Edition, June 2256 2018, . 2260 [IEEE754] IEEE, "IEEE Standard for Floating-Point Arithmetic", IEEE 2261 Std 754-2008. 2263 [RFC2045] Freed, N. and N. Borenstein, "Multipurpose Internet Mail 2264 Extensions (MIME) Part One: Format of Internet Message 2265 Bodies", RFC 2045, DOI 10.17487/RFC2045, November 1996, 2266 . 2268 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 2269 Requirement Levels", BCP 14, RFC 2119, 2270 DOI 10.17487/RFC2119, March 1997, 2271 . 2273 [RFC3339] Klyne, G. and C. Newman, "Date and Time on the Internet: 2274 Timestamps", RFC 3339, DOI 10.17487/RFC3339, July 2002, 2275 . 2277 [RFC3629] Yergeau, F., "UTF-8, a transformation format of ISO 2278 10646", STD 63, RFC 3629, DOI 10.17487/RFC3629, November 2279 2003, . 2281 [RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform 2282 Resource Identifier (URI): Generic Syntax", STD 66, 2283 RFC 3986, DOI 10.17487/RFC3986, January 2005, 2284 . 2286 [RFC4287] Nottingham, M., Ed. and R. Sayre, Ed., "The Atom 2287 Syndication Format", RFC 4287, DOI 10.17487/RFC4287, 2288 December 2005, . 2290 [RFC4648] Josefsson, S., "The Base16, Base32, and Base64 Data 2291 Encodings", RFC 4648, DOI 10.17487/RFC4648, October 2006, 2292 . 2294 [RFC8126] Cotton, M., Leiba, B., and T. Narten, "Guidelines for 2295 Writing an IANA Considerations Section in RFCs", BCP 26, 2296 RFC 8126, DOI 10.17487/RFC8126, June 2017, 2297 . 2299 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2300 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 2301 May 2017, . 2303 [TIME_T] The Open Group Base Specifications, "Vol. 1: Base 2304 Definitions, Issue 7", Section 4.15 'Seconds Since the 2305 Epoch', IEEE Std 1003.1, 2013 Edition, 2013, 2306 . 2309 11.2. Informative References 2311 [ASN.1] International Telecommunication Union, "Information 2312 Technology -- ASN.1 encoding rules: Specification of Basic 2313 Encoding Rules (BER), Canonical Encoding Rules (CER) and 2314 Distinguished Encoding Rules (DER)", ITU-T Recommendation 2315 X.690, 1994. 2317 [BSON] Various, "BSON - Binary JSON", 2013, 2318 . 2320 [I-D.ietf-cbor-sequence] 2321 Bormann, C., "Concise Binary Object Representation (CBOR) 2322 Sequences", draft-ietf-cbor-sequence-02 (work in 2323 progress), September 2019. 2325 [IANA.cbor-simple-values] 2326 IANA, "Concise Binary Object Representation (CBOR) Simple 2327 Values", 2328 . 2330 [IANA.cbor-tags] 2331 IANA, "Concise Binary Object Representation (CBOR) Tags", 2332 . 2334 [MessagePack] 2335 Furuhashi, S., "MessagePack", 2013, . 2337 [PCRE] Ho, A., "PCRE - Perl Compatible Regular Expressions", 2338 2018, . 2340 [RFC0713] Haverty, J., "MSDTP-Message Services Data Transmission 2341 Protocol", RFC 713, DOI 10.17487/RFC0713, April 1976, 2342 . 2344 [RFC6838] Freed, N., Klensin, J., and T. Hansen, "Media Type 2345 Specifications and Registration Procedures", BCP 13, 2346 RFC 6838, DOI 10.17487/RFC6838, January 2013, 2347 . 2349 [RFC7049] Bormann, C. and P. Hoffman, "Concise Binary Object 2350 Representation (CBOR)", RFC 7049, DOI 10.17487/RFC7049, 2351 October 2013, . 2353 [RFC7228] Bormann, C., Ersue, M., and A. Keranen, "Terminology for 2354 Constrained-Node Networks", RFC 7228, 2355 DOI 10.17487/RFC7228, May 2014, 2356 . 2358 [RFC7493] Bray, T., Ed., "The I-JSON Message Format", RFC 7493, 2359 DOI 10.17487/RFC7493, March 2015, 2360 . 2362 [RFC8259] Bray, T., Ed., "The JavaScript Object Notation (JSON) Data 2363 Interchange Format", STD 90, RFC 8259, 2364 DOI 10.17487/RFC8259, December 2017, 2365 . 2367 [RFC8618] Dickinson, J., Hague, J., Dickinson, S., Manderson, T., 2368 and J. Bond, "Compacted-DNS (C-DNS): A Format for DNS 2369 Packet Capture", RFC 8618, DOI 10.17487/RFC8618, September 2370 2019, . 2372 [SIPHASH] Aumasson, J. and D. Bernstein, "SipHash: A Fast Short- 2373 Input PRF", Lecture Notes in Computer Science pp. 489-508, 2374 DOI 10.1007/978-3-642-34931-7_28, 2012. 2376 [YAML] Ben-Kiki, O., Evans, C., and I. Net, "YAML Ain't Markup 2377 Language (YAML[TM]) Version 1.2", 3rd Edition, October 2378 2009, . 2380 Appendix A. Examples 2382 The following table provides some CBOR-encoded values in hexadecimal 2383 (right column), together with diagnostic notation for these values 2384 (left column). Note that the string "\u00fc" is one form of 2385 diagnostic notation for a UTF-8 string containing the single Unicode 2386 character U+00FC, LATIN SMALL LETTER U WITH DIAERESIS (u umlaut). 2387 Similarly, "\u6c34" is a UTF-8 string in diagnostic notation with a 2388 single character U+6C34 (CJK UNIFIED IDEOGRAPH-6C34, often 2389 representing "water"), and "\ud800\udd51" is a UTF-8 string in 2390 diagnostic notation with a single character U+10151 (GREEK ACROPHONIC 2391 ATTIC FIFTY STATERS). (Note that all these single-character strings 2392 could also be represented in native UTF-8 in diagnostic notation, 2393 just not in an ASCII-only specification like the present one.) In 2394 the diagnostic notation provided for bignums, their intended numeric 2395 value is shown as a decimal number (such as 18446744073709551616) 2396 instead of showing a tagged byte string (such as 2397 2(h'010000000000000000')). 2399 +------------------------------+------------------------------------+ 2400 | Diagnostic | Encoded | 2401 +------------------------------+------------------------------------+ 2402 | 0 | 0x00 | 2403 | | | 2404 | 1 | 0x01 | 2405 | | | 2406 | 10 | 0x0a | 2407 | | | 2408 | 23 | 0x17 | 2409 | | | 2410 | 24 | 0x1818 | 2411 | | | 2412 | 25 | 0x1819 | 2413 | | | 2414 | 100 | 0x1864 | 2415 | | | 2416 | 1000 | 0x1903e8 | 2417 | | | 2418 | 1000000 | 0x1a000f4240 | 2419 | | | 2420 | 1000000000000 | 0x1b000000e8d4a51000 | 2421 | | | 2422 | 18446744073709551615 | 0x1bffffffffffffffff | 2423 | | | 2424 | 18446744073709551616 | 0xc249010000000000000000 | 2425 | | | 2426 | -18446744073709551616 | 0x3bffffffffffffffff | 2427 | | | 2428 | -18446744073709551617 | 0xc349010000000000000000 | 2429 | | | 2430 | -1 | 0x20 | 2431 | | | 2432 | -10 | 0x29 | 2433 | | | 2434 | -100 | 0x3863 | 2435 | | | 2436 | -1000 | 0x3903e7 | 2437 | | | 2438 | 0.0 | 0xf90000 | 2439 | | | 2440 | -0.0 | 0xf98000 | 2441 | | | 2442 | 1.0 | 0xf93c00 | 2443 | | | 2444 | 1.1 | 0xfb3ff199999999999a | 2445 | | | 2446 | 1.5 | 0xf93e00 | 2447 | | | 2448 | 65504.0 | 0xf97bff | 2449 | | | 2450 | 100000.0 | 0xfa47c35000 | 2451 | | | 2452 | 3.4028234663852886e+38 | 0xfa7f7fffff | 2453 | | | 2454 | 1.0e+300 | 0xfb7e37e43c8800759c | 2455 | | | 2456 | 5.960464477539063e-8 | 0xf90001 | 2457 | | | 2458 | 0.00006103515625 | 0xf90400 | 2459 | | | 2460 | -4.0 | 0xf9c400 | 2461 | | | 2462 | -4.1 | 0xfbc010666666666666 | 2463 | | | 2464 | Infinity | 0xf97c00 | 2465 | | | 2466 | NaN | 0xf97e00 | 2467 | | | 2468 | -Infinity | 0xf9fc00 | 2469 | | | 2470 | Infinity | 0xfa7f800000 | 2471 | | | 2472 | NaN | 0xfa7fc00000 | 2473 | | | 2474 | -Infinity | 0xfaff800000 | 2475 | | | 2476 | Infinity | 0xfb7ff0000000000000 | 2477 | | | 2478 | NaN | 0xfb7ff8000000000000 | 2479 | | | 2480 | -Infinity | 0xfbfff0000000000000 | 2481 | | | 2482 | false | 0xf4 | 2483 | | | 2484 | true | 0xf5 | 2485 | | | 2486 | null | 0xf6 | 2487 | | | 2488 | undefined | 0xf7 | 2489 | | | 2490 | simple(16) | 0xf0 | 2491 | | | 2492 | simple(255) | 0xf8ff | 2493 | | | 2494 | 0("2013-03-21T20:04:00Z") | 0xc074323031332d30332d32315432303a | 2495 | | 30343a30305a | 2496 | | | 2497 | 1(1363896240) | 0xc11a514b67b0 | 2498 | | | 2499 | 1(1363896240.5) | 0xc1fb41d452d9ec200000 | 2500 | | | 2501 | 23(h'01020304') | 0xd74401020304 | 2502 | | | 2503 | 24(h'6449455446') | 0xd818456449455446 | 2504 | | | 2505 | 32("http://www.example.com") | 0xd82076687474703a2f2f7777772e6578 | 2506 | | 616d706c652e636f6d | 2507 | | | 2508 | h'' | 0x40 | 2509 | | | 2510 | h'01020304' | 0x4401020304 | 2511 | | | 2512 | "" | 0x60 | 2513 | | | 2514 | "a" | 0x6161 | 2515 | | | 2516 | "IETF" | 0x6449455446 | 2517 | | | 2518 | "\"\\" | 0x62225c | 2519 | | | 2520 | "\u00fc" | 0x62c3bc | 2521 | | | 2522 | "\u6c34" | 0x63e6b0b4 | 2523 | | | 2524 | "\ud800\udd51" | 0x64f0908591 | 2525 | | | 2526 | [] | 0x80 | 2527 | | | 2528 | [1, 2, 3] | 0x83010203 | 2529 | | | 2530 | [1, [2, 3], [4, 5]] | 0x8301820203820405 | 2531 | | | 2532 | [1, 2, 3, 4, 5, 6, 7, 8, 9, | 0x98190102030405060708090a0b0c0d0e | 2533 | 10, 11, 12, 13, 14, 15, 16, | 0f101112131415161718181819 | 2534 | 17, 18, 19, 20, 21, 22, 23, | | 2535 | 24, 25] | | 2536 | | | 2537 | {} | 0xa0 | 2538 | | | 2539 | {1: 2, 3: 4} | 0xa201020304 | 2540 | | | 2541 | {"a": 1, "b": [2, 3]} | 0xa26161016162820203 | 2542 | | | 2543 | ["a", {"b": "c"}] | 0x826161a161626163 | 2544 | | | 2545 | {"a": "A", "b": "B", "c": | 0xa5616161416162614261636143616461 | 2546 | "C", "d": "D", "e": "E"} | 4461656145 | 2547 | | | 2548 | (_ h'0102', h'030405') | 0x5f42010243030405ff | 2549 | | | 2550 | (_ "strea", "ming") | 0x7f657374726561646d696e67ff | 2551 | | | 2552 | [_ ] | 0x9fff | 2553 | | | 2554 | [_ 1, [2, 3], [_ 4, 5]] | 0x9f018202039f0405ffff | 2555 | | | 2556 | [_ 1, [2, 3], [4, 5]] | 0x9f01820203820405ff | 2557 | | | 2558 | [1, [2, 3], [_ 4, 5]] | 0x83018202039f0405ff | 2559 | | | 2560 | [1, [_ 2, 3], [4, 5]] | 0x83019f0203ff820405 | 2561 | | | 2562 | [_ 1, 2, 3, 4, 5, 6, 7, 8, | 0x9f0102030405060708090a0b0c0d0e0f | 2563 | 9, 10, 11, 12, 13, 14, 15, | 101112131415161718181819ff | 2564 | 16, 17, 18, 19, 20, 21, 22, | | 2565 | 23, 24, 25] | | 2566 | | | 2567 | {_ "a": 1, "b": [_ 2, 3]} | 0xbf61610161629f0203ffff | 2568 | | | 2569 | ["a", {_ "b": "c"}] | 0x826161bf61626163ff | 2570 | | | 2571 | {_ "Fun": true, "Amt": -2} | 0xbf6346756ef563416d7421ff | 2572 +------------------------------+------------------------------------+ 2574 Table 5: Examples of Encoded CBOR Data Items 2576 Appendix B. Jump Table 2578 For brevity, this jump table does not show initial bytes that are 2579 reserved for future extension. It also only shows a selection of the 2580 initial bytes that can be used for optional features. (All unsigned 2581 integers are in network byte order.) 2583 +------------+------------------------------------------------------+ 2584 | Byte | Structure/Semantics | 2585 +------------+------------------------------------------------------+ 2586 | 0x00..0x17 | Integer 0x00..0x17 (0..23) | 2587 | | | 2588 | 0x18 | Unsigned integer (one-byte uint8_t follows) | 2589 | | | 2590 | 0x19 | Unsigned integer (two-byte uint16_t follows) | 2591 | | | 2592 | 0x1a | Unsigned integer (four-byte uint32_t follows) | 2593 | | | 2594 | 0x1b | Unsigned integer (eight-byte uint64_t follows) | 2595 | | | 2596 | 0x20..0x37 | Negative integer -1-0x00..-1-0x17 (-1..-24) | 2597 | | | 2598 | 0x38 | Negative integer -1-n (one-byte uint8_t for n | 2599 | | follows) | 2600 | | | 2601 | 0x39 | Negative integer -1-n (two-byte uint16_t for n | 2602 | | follows) | 2603 | | | 2604 | 0x3a | Negative integer -1-n (four-byte uint32_t for n | 2605 | | follows) | 2606 | | | 2607 | 0x3b | Negative integer -1-n (eight-byte uint64_t for n | 2608 | | follows) | 2609 | | | 2610 | 0x40..0x57 | byte string (0x00..0x17 bytes follow) | 2611 | | | 2612 | 0x58 | byte string (one-byte uint8_t for n, and then n | 2613 | | bytes follow) | 2614 | | | 2615 | 0x59 | byte string (two-byte uint16_t for n, and then n | 2616 | | bytes follow) | 2617 | | | 2618 | 0x5a | byte string (four-byte uint32_t for n, and then n | 2619 | | bytes follow) | 2620 | | | 2621 | 0x5b | byte string (eight-byte uint64_t for n, and then n | 2622 | | bytes follow) | 2623 | | | 2624 | 0x5f | byte string, byte strings follow, terminated by | 2625 | | "break" | 2626 | | | 2627 | 0x60..0x77 | UTF-8 string (0x00..0x17 bytes follow) | 2628 | | | 2629 | 0x78 | UTF-8 string (one-byte uint8_t for n, and then n | 2630 | | bytes follow) | 2631 | | | 2632 | 0x79 | UTF-8 string (two-byte uint16_t for n, and then n | 2633 | | bytes follow) | 2634 | | | 2635 | 0x7a | UTF-8 string (four-byte uint32_t for n, and then n | 2636 | | bytes follow) | 2637 | | | 2638 | 0x7b | UTF-8 string (eight-byte uint64_t for n, and then n | 2639 | | bytes follow) | 2640 | | | 2641 | 0x7f | UTF-8 string, UTF-8 strings follow, terminated by | 2642 | | "break" | 2643 | | | 2644 | 0x80..0x97 | array (0x00..0x17 data items follow) | 2645 | | | 2646 | 0x98 | array (one-byte uint8_t for n, and then n data items | 2647 | | follow) | 2648 | | | 2649 | 0x99 | array (two-byte uint16_t for n, and then n data | 2650 | | items follow) | 2651 | | | 2652 | 0x9a | array (four-byte uint32_t for n, and then n data | 2653 | | items follow) | 2654 | | | 2655 | 0x9b | array (eight-byte uint64_t for n, and then n data | 2656 | | items follow) | 2657 | | | 2658 | 0x9f | array, data items follow, terminated by "break" | 2659 | | | 2660 | 0xa0..0xb7 | map (0x00..0x17 pairs of data items follow) | 2661 | | | 2662 | 0xb8 | map (one-byte uint8_t for n, and then n pairs of | 2663 | | data items follow) | 2664 | | | 2665 | 0xb9 | map (two-byte uint16_t for n, and then n pairs of | 2666 | | data items follow) | 2667 | | | 2668 | 0xba | map (four-byte uint32_t for n, and then n pairs of | 2669 | | data items follow) | 2670 | | | 2671 | 0xbb | map (eight-byte uint64_t for n, and then n pairs of | 2672 | | data items follow) | 2673 | | | 2674 | 0xbf | map, pairs of data items follow, terminated by | 2675 | | "break" | 2676 | | | 2677 | 0xc0 | Text-based date/time (data item follows; see | 2678 | | Section 3.4.2) | 2679 | | | 2680 | 0xc1 | Epoch-based date/time (data item follows; see | 2681 | | Section 3.4.3) | 2682 | | | 2683 | 0xc2 | Positive bignum (data item "byte string" follows) | 2684 | | | 2685 | 0xc3 | Negative bignum (data item "byte string" follows) | 2686 | | | 2687 | 0xc4 | Decimal Fraction (data item "array" follows; see | 2688 | | Section 3.4.5) | 2689 | | | 2690 | 0xc5 | Bigfloat (data item "array" follows; see | 2691 | | Section 3.4.5) | 2692 | | | 2693 | 0xc6..0xd4 | (tag) | 2694 | | | 2695 | 0xd5..0xd7 | Expected Conversion (data item follows; see | 2696 | | Section 3.4.6.2) | 2697 | | | 2698 | 0xd8..0xdb | (more tags, 1/2/4/8 bytes and then a data item | 2699 | | follow) | 2700 | | | 2701 | 0xe0..0xf3 | (simple value) | 2702 | | | 2703 | 0xf4 | False | 2704 | | | 2705 | 0xf5 | True | 2706 | | | 2707 | 0xf6 | Null | 2708 | | | 2709 | 0xf7 | Undefined | 2710 | | | 2711 | 0xf8 | (simple value, one byte follows) | 2712 | | | 2713 | 0xf9 | Half-Precision Float (two-byte IEEE 754) | 2714 | | | 2715 | 0xfa | Single-Precision Float (four-byte IEEE 754) | 2716 | | | 2717 | 0xfb | Double-Precision Float (eight-byte IEEE 754) | 2718 | | | 2719 | 0xff | "break" stop code | 2720 +------------+------------------------------------------------------+ 2722 Table 6: Jump Table for Initial Byte 2724 Appendix C. Pseudocode 2726 The well-formedness of a CBOR item can be checked by the pseudocode 2727 in Figure 1. The data is well-formed if and only if: 2729 o the pseudocode does not "fail"; 2731 o after execution of the pseudocode, no bytes are left in the input 2732 (except in streaming applications) 2734 The pseudocode has the following prerequisites: 2736 o take(n) reads n bytes from the input data and returns them as a 2737 byte string. If n bytes are no longer available, take(n) fails. 2739 o uint() converts a byte string into an unsigned integer by 2740 interpreting the byte string in network byte order. 2742 o Arithmetic works as in C. 2744 o All variables are unsigned integers of sufficient range. 2746 Note that "well_formed" returns the major type for well-formed 2747 definite length items, but 0 for an indefinite length item (or -1 for 2748 a break stop code, only if "breakable" is set). This is used in 2749 "well_formed_indefinite" to ascertain that indefinite length strings 2750 only contain definite length strings as chunks. 2752 well_formed (breakable = false) { 2753 // process initial bytes 2754 ib = uint(take(1)); 2755 mt = ib >> 5; 2756 val = ai = ib & 0x1f; 2757 switch (ai) { 2758 case 24: val = uint(take(1)); break; 2759 case 25: val = uint(take(2)); break; 2760 case 26: val = uint(take(4)); break; 2761 case 27: val = uint(take(8)); break; 2762 case 28: case 29: case 30: fail(); 2763 case 31: 2764 return well_formed_indefinite(mt, breakable); 2765 } 2766 // process content 2767 switch (mt) { 2768 // case 0, 1, 7 do not have content; just use val 2769 case 2: case 3: take(val); break; // bytes/UTF-8 2770 case 4: for (i = 0; i < val; i++) well_formed(); break; 2771 case 5: for (i = 0; i < val*2; i++) well_formed(); break; 2772 case 6: well_formed(); break; // 1 embedded data item 2773 case 7: if (ai == 24 && val < 32) fail(); // bad simple 2774 } 2775 return mt; // finite data item 2776 } 2778 well_formed_indefinite(mt, breakable) { 2779 switch (mt) { 2780 case 2: case 3: 2781 while ((it = well_formed(true)) != -1) 2782 if (it != mt) // need finite-length chunk 2783 fail(); // of same type 2784 break; 2785 case 4: while (well_formed(true) != -1); break; 2786 case 5: while (well_formed(true) != -1) well_formed(); break; 2787 case 7: 2788 if (breakable) 2789 return -1; // signal break out 2790 else fail(); // no enclosing indefinite 2791 default: fail(); // wrong mt 2792 } 2793 return 0; // no break out 2794 } 2796 Figure 1: Pseudocode for Well-Formedness Check 2798 Note that the remaining complexity of a complete CBOR decoder is 2799 about presenting data that has been decoded to the application in an 2800 appropriate form. 2802 Major types 0 and 1 are designed in such a way that they can be 2803 encoded in C from a signed integer without actually doing an if-then- 2804 else for positive/negative (Figure 2). This uses the fact that 2805 (-1-n), the transformation for major type 1, is the same as ~n 2806 (bitwise complement) in C unsigned arithmetic; ~n can then be 2807 expressed as (-1)^n for the negative case, while 0^n leaves n 2808 unchanged for non-negative. The sign of a number can be converted to 2809 -1 for negative and 0 for non-negative (0 or positive) by arithmetic- 2810 shifting the number by one bit less than the bit length of the number 2811 (for example, by 63 for 64-bit numbers). 2813 void encode_sint(int64_t n) { 2814 uint64t ui = n >> 63; // extend sign to whole length 2815 mt = ui & 0x20; // extract major type 2816 ui ^= n; // complement negatives 2817 if (ui < 24) 2818 *p++ = mt + ui; 2819 else if (ui < 256) { 2820 *p++ = mt + 24; 2821 *p++ = ui; 2822 } else 2823 ... 2825 Figure 2: Pseudocode for Encoding a Signed Integer 2827 Appendix D. Half-Precision 2829 As half-precision floating-point numbers were only added to IEEE 754 2830 in 2008 [IEEE754], today's programming platforms often still only 2831 have limited support for them. It is very easy to include at least 2832 decoding support for them even without such support. An example of a 2833 small decoder for half-precision floating-point numbers in the C 2834 language is shown in Figure 3. A similar program for Python is in 2835 Figure 4; this code assumes that the 2-byte value has already been 2836 decoded as an (unsigned short) integer in network byte order (as 2837 would be done by the pseudocode in Appendix C). 2839 #include 2841 double decode_half(unsigned char *halfp) { 2842 int half = (halfp[0] << 8) + halfp[1]; 2843 int exp = (half >> 10) & 0x1f; 2844 int mant = half & 0x3ff; 2845 double val; 2846 if (exp == 0) val = ldexp(mant, -24); 2847 else if (exp != 31) val = ldexp(mant + 1024, exp - 25); 2848 else val = mant == 0 ? INFINITY : NAN; 2849 return half & 0x8000 ? -val : val; 2850 } 2852 Figure 3: C Code for a Half-Precision Decoder 2854 import struct 2855 from math import ldexp 2857 def decode_single(single): 2858 return struct.unpack("!f", struct.pack("!I", single))[0] 2860 def decode_half(half): 2861 valu = (half & 0x7fff) << 13 | (half & 0x8000) << 16 2862 if ((half & 0x7c00) != 0x7c00): 2863 return ldexp(decode_single(valu), 112) 2864 return decode_single(valu | 0x7f800000) 2866 Figure 4: Python Code for a Half-Precision Decoder 2868 Appendix E. Comparison of Other Binary Formats to CBOR's Design 2869 Objectives 2871 The proposal for CBOR follows a history of binary formats that is as 2872 long as the history of computers themselves. Different formats have 2873 had different objectives. In most cases, the objectives of the 2874 format were never stated, although they can sometimes be implied by 2875 the context where the format was first used. Some formats were meant 2876 to be universally usable, although history has proven that no binary 2877 format meets the needs of all protocols and applications. 2879 CBOR differs from many of these formats due to it starting with a set 2880 of objectives and attempting to meet just those. This section 2881 compares a few of the dozens of formats with CBOR's objectives in 2882 order to help the reader decide if they want to use CBOR or a 2883 different format for a particular protocol or application. 2885 Note that the discussion here is not meant to be a criticism of any 2886 format: to the best of our knowledge, no format before CBOR was meant 2887 to cover CBOR's objectives in the priority we have assigned them. A 2888 brief recap of the objectives from Section 1.1 is: 2890 1. unambiguous encoding of most common data formats from Internet 2891 standards 2893 2. code compactness for encoder or decoder 2895 3. no schema description needed 2897 4. reasonably compact serialization 2899 5. applicability to constrained and unconstrained applications 2901 6. good JSON conversion 2903 7. extensibility 2905 A discussion of CBOR and other formats with respect to a different 2906 set of design objectives is provided in Section 5 and Appendix C of 2907 [RFC8618]. 2909 E.1. ASN.1 DER, BER, and PER 2911 [ASN.1] has many serializations. In the IETF, DER and BER are the 2912 most common. The serialized output is not particularly compact for 2913 many items, and the code needed to decode numeric items can be 2914 complex on a constrained device. 2916 Few (if any) IETF protocols have adopted one of the several variants 2917 of Packed Encoding Rules (PER). There could be many reasons for 2918 this, but one that is commonly stated is that PER makes use of the 2919 schema even for parsing the surface structure of the data stream, 2920 requiring significant tool support. There are different versions of 2921 the ASN.1 schema language in use, which has also hampered adoption. 2923 E.2. MessagePack 2925 [MessagePack] is a concise, widely implemented counted binary 2926 serialization format, similar in many properties to CBOR, although 2927 somewhat less regular. While the data model can be used to represent 2928 JSON data, MessagePack has also been used in many remote procedure 2929 call (RPC) applications and for long-term storage of data. 2931 MessagePack has been essentially stable since it was first published 2932 around 2011; it has not yet had a transition. The evolution of 2933 MessagePack is impeded by an imperative to maintain complete 2934 backwards compatibility with existing stored data, while only few 2935 bytecodes are still available for extension. Repeated requests over 2936 the years from the MessagePack user community to separate out binary 2937 and text strings in the encoding recently have led to an extension 2938 proposal that would leave MessagePack's "raw" data ambiguous between 2939 its usages for binary and text data. The extension mechanism for 2940 MessagePack remains unclear. 2942 E.3. BSON 2944 [BSON] is a data format that was developed for the storage of JSON- 2945 like maps (JSON objects) in the MongoDB database. Its major 2946 distinguishing feature is the capability for in-place update, which 2947 prevents a compact representation. BSON uses a counted 2948 representation except for map keys, which are null-byte terminated. 2949 While BSON can be used for the representation of JSON-like objects on 2950 the wire, its specification is dominated by the requirements of the 2951 database application and has become somewhat baroque. The status of 2952 how BSON extensions will be implemented remains unclear. 2954 E.4. MSDTP: RFC 713 2956 Message Services Data Transmission (MSDTP) is a very early example of 2957 a compact message format; it is described in [RFC0713], written in 2958 1976. It is included here for its historical value, not because it 2959 was ever widely used. 2961 E.5. Conciseness on the Wire 2963 While CBOR's design objective of code compactness for encoders and 2964 decoders is a higher priority than its objective of conciseness on 2965 the wire, many people focus on the wire size. Table 7 shows some 2966 encoding examples for the simple nested array [1, [2, 3]]; where some 2967 form of indefinite-length encoding is supported by the encoding, 2968 [_ 1, [2, 3]] (indefinite length on the outer array) is also shown. 2970 +-------------+--------------------------+--------------------------+ 2971 | Format | [1, [2, 3]] | [_ 1, [2, 3]] | 2972 +-------------+--------------------------+--------------------------+ 2973 | RFC 713 | c2 05 81 c2 02 82 83 | | 2974 | | | | 2975 | ASN.1 BER | 30 0b 02 01 01 30 06 02 | 30 80 02 01 01 30 06 02 | 2976 | | 01 02 02 01 03 | 01 02 02 01 03 00 00 | 2977 | | | | 2978 | MessagePack | 92 01 92 02 03 | | 2979 | | | | 2980 | BSON | 22 00 00 00 10 30 00 01 | | 2981 | | 00 00 00 04 31 00 13 00 | | 2982 | | 00 00 10 30 00 02 00 00 | | 2983 | | 00 10 31 00 03 00 00 00 | | 2984 | | 00 00 | | 2985 | | | | 2986 | CBOR | 82 01 82 02 03 | 9f 01 82 02 03 ff | 2987 +-------------+--------------------------+--------------------------+ 2989 Table 7: Examples for Different Levels of Conciseness 2991 Appendix F. Changes from RFC 7049 2993 The following is a list of known changes from RFC 7049. This list is 2994 non-authoritative. It is meant to help reviewers see the significant 2995 differences. 2997 o Updated reference for [RFC4627] to [RFC8259] in many places 2999 o Updated reference for [CNN-TERMS] to [RFC7228] 3001 o Added a comment to the last example in Section 2.2.1 (added 3002 "Second value") 3004 o Fixed a bug in the example in Section 2.4.2 ("29" -> "49") 3006 o Fixed a bug in the last paragraph of Section 3.6 ("0b000_11101" -> 3007 "0b000_11001") 3009 Appendix G. Well-formedness errors and examples 3011 There are three basic kinds of well-formedness errors that can occur 3012 in decoding a CBOR data item: 3014 o Too much data: There are input bytes left that were not consumed. 3015 This is only an error if the application assumed that the input 3016 bytes would span exexactly one data item. Where the application 3017 uses the self-delimiting nature of CBOR encoding to permit 3018 additional data after the data item, as is for example done in 3019 CBOR sequences [I-D.ietf-cbor-sequence], the CBOR decoder can 3020 simply indicate what part of the input has not been consumed. 3022 o Too little data: The input data available would need additional 3023 bytes added at their end for a complete CBOR data item. This may 3024 indicate the input is truncated; it is also a common error when 3025 trying to decode random data as CBOR. For some applications 3026 however, this may not be actually be an error, as the application 3027 may not be certain it has all the data yet and can obtain or wait 3028 for additional input bytes. Some of these applications may have 3029 an upper limit for how much additional data can show up; here the 3030 decoder may be able to indicate that the encoded CBOR data item 3031 cannot be completed within this limit. 3033 o Syntax error: The input data are not consistent with the 3034 requirements of the CBOR encoding, and this cannot be remedied by 3035 adding (or removing) data at the end. 3037 In Appendix C, errors of the first kind are addressed in the first 3038 paragraph/bullet list (requiring "no bytes are left"), and errors of 3039 the second kind are addressed in the second paragraph/bullet list 3040 (failing "if n bytes are no longer available"). Errors of the third 3041 kind are identified in the pseudocode by specific instances of 3042 calling fail(), in order: 3044 o a reserved value is used for additional information (28, 29, 30) 3046 o major type 7, additional information 24, value < 32 (incorrect or 3047 incorrectly encoded simple type) 3049 o incorrect substructure of indefinite length byte/text string (may 3050 only contain definite length strings of the same major type) 3052 o break stop code (mt=7, ai=31) occurs in a value position of a map 3053 or except at a position directly in an indefinite length item 3054 where also another enclosed data item could occur 3056 o additional information 31 used with major type 0, 1, or 6 3058 G.1. Examples for CBOR data items that are not well-formed 3060 This subsection shows a few examples for CBOR data items that are not 3061 well-formed. Each example is a sequence of bytes each shown in 3062 hexadecimal; multiple examples in a list are separated by commas. 3064 Examples for well-formedness error kind 1 (too much data) can easily 3065 be formed by adding data to a well-formed encoded CBOR data item. 3067 Similarly, examples for well-formedness error kind 2 (too little 3068 data) can be formed by truncating a well-formed encoded CBOR data 3069 item. In test suites, it may be beneficial to specifically test with 3070 incomplete data items that would require large amounts of addition to 3071 be completed (for instance by starting the encoding of a string of a 3072 very large size). 3074 A premature end of the input can occur in a head or within the 3075 enclosed data, which may be bare strings or enclosed data items that 3076 are either counted or should have been ended by a break stop code. 3078 o End of input in a head: 18, 19, 1a, 1b, 19 01, 1a 01 02, 1b 01 02 3079 03 04 05 06 07, 38, 58, 78, 98, 9a 01 ff 00, b8, d8, f8, f9 00, fa 3080 00 00, fb 00 00 00 3082 o Definite length strings with short data: 41, 61, 5a ff ff ff ff 3083 00, 5b ff ff ff ff ff ff ff ff 01 02 03, 7a ff ff ff ff 00, 7b 7f 3084 ff ff ff ff ff ff ff 01 02 03 3086 o Definite length maps and arrays not closed with enough items: 81, 3087 81 81 81 81 81 81 81 81 81, 82 00, a1, a2 01 02, a1 00, a2 00 00 3088 00 3090 o Indefinite length strings not closed by a break stop code: 5f 41 3091 00, 7f 61 00 3093 o Indefinite length maps and arrays not closed by a break stop code: 3094 9f, 9f 01 02, bf, bf 01 02 01 02, 81 9f, 9f 80 00, 9f 9f 9f 9f 9f 3095 ff ff ff ff, 9f 81 9f 81 9f 9f ff ff ff 3097 A few examples for the five subkinds of well-formedness error kind 3 3098 (syntax error) are shown below. 3100 Subkind 1: 3102 o Reserved additional information values: 1c, 1d, 1e, 3c, 3d, 3e, 3103 5c, 5d, 5e, 7c, 7d, 7e, 9c, 9d, 9e, bc, bd, be, dc, dd, de, fc, 3104 fd, fe, 3106 Subkind 2: 3108 o Reserved two-byte encodings of simple types: f8 00, f8 01, f8 18, 3109 f8 1f 3111 Subkind 3: 3113 o Indefinite length string chunks not of the correct type: 5f 00 ff, 3114 5f 21 ff, 5f 61 00 ff, 5f 80 ff, 5f a0 ff, 5f c0 00 ff, 5f e0 ff, 3115 7f 41 00 ff 3117 o Indefinite length string chunks not definite length: 5f 5f 41 00 3118 ff ff, 7f 7f 61 00 ff ff 3120 Subkind 4: 3122 o Break occurring on its own outside of an indefinite length item: 3123 ff 3125 o Break occurring in a definite length array or map or a tag: 81 ff, 3126 82 00 ff, a1 ff, a1 ff 00, a1 00 ff, a2 00 00 ff, 9f 81 ff, 9f 82 3127 9f 81 9f 9f ff ff ff ff 3129 o Break in indefinite length map would lead to odd number of items 3130 (break in a value position): bf 00 ff, bf 00 00 00 ff 3132 Subkind 5: 3134 o Major type 0, 1, 6 with additional information 31: 1f, 3f, df 3136 Acknowledgements 3138 CBOR was inspired by MessagePack. MessagePack was developed and 3139 promoted by Sadayuki Furuhashi ("frsyuki"). This reference to 3140 MessagePack is solely for attribution; CBOR is not intended as a 3141 version of or replacement for MessagePack, as it has different design 3142 goals and requirements. 3144 The need for functionality beyond the original MessagePack 3145 Specification became obvious to many people at about the same time 3146 around the year 2012. BinaryPack is a minor derivation of 3147 MessagePack that was developed by Eric Zhang for the binaryjs 3148 project. A similar, but different, extension was made by Tim Caswell 3149 for his msgpack-js and msgpack-js-browser projects. Many people have 3150 contributed to the discussion about extending MessagePack to separate 3151 text string representation from byte string representation. 3153 The encoding of the additional information in CBOR was inspired by 3154 the encoding of length information designed by Klaus Hartke for CoAP. 3156 This document also incorporates suggestions made by many people, 3157 notably Dan Frost, James Manger, Jeffrey Yaskin, Joe Hildebrand, 3158 Keith Moore, Laurence Lundblade, Matthew Lepinski, Michael 3159 Richardson, Nico Williams, Peter Occil, Phillip Hallam-Baker, Ray 3160 Polk, Tim Bray, Tony Finch, Tony Hansen, and Yaron Sheffer. 3162 Authors' Addresses 3164 Carsten Bormann 3165 Universitaet Bremen TZI 3166 Postfach 330440 3167 D-28359 Bremen 3168 Germany 3170 Phone: +49-421-218-63921 3171 EMail: cabo@tzi.org 3173 Paul Hoffman 3174 ICANN 3176 EMail: paul.hoffman@icann.org