CBOR Configuration

CBOR Configuration Universität Bremen TZI

Postfach 330440 Bremen D-28359 Germany +49-421-218-63921 cabo@tzi.org

Applications and Real-Time CBOR (Concise Binary Object Representation Maintenance and Extensions) Ecosystem Serialization Subsetting Normalization Interoperability This document discusses configuration of CBOR processors. Using this information as a basis, it provides WGLC feedback on draft-ietf-cbor-serialization-08. About This Document Status information for this document may be found at . Discussion of this document takes place on the CBOR (Concise Binary Object Representation Maintenance and Extensions) Working Group mailing list (), which is archived at . Subscribe at . Source for this draft and an issue tracker can be found at .

Introduction This document discusses configuration of CBOR processors, more specifically, of libraries and other implementations that handle generation and ingestion of encoded CBOR data items for an application. Such a configuration will influence a number of individual services, including serialization (while meeting specific serialization constraints), data model subsetting, and normalization services (). Of these, only serialization/deserialization is a core service of such a library, but that is often combined with other services for convenience and ease of implementation.

For convenience and easier reference to CBOR concepts, this document includes working-group last call comments on .

Status: Not Ready
- The document is difficult to appraise as it doesn’t reveal what it actually is trying to do and how. Reasons for a need for fundamental changes are not identified. Lack of terminological clarity makes it hard to predict how the document will be (mis)understood, and thus more generally the outcome of publishing the document.

The introduction contains this statement:

As a result, CBOR libraries and protocol implementations commonly support only the serialization forms required for their intended use cases. This behavior is expected and aligns with CBOR’s design goals.

(This sentence uses the word "support", which doesn't reveal whether this is about decoder support for consuming the specific serialization forms, or about encoder support for meeting arbitrary partial decoder implementations' requirements. Imprecision of this kind makes it hard to supply straightforward fixes.) It is not at all clear what the normative intent of this statement is. This paints a picture of perpetuating a forest of barely interoperable implementations with a thicket of restrictions. Instead, the reality is that generic CBOR implementations exist, and that it is much easier to build CBOR applications based on these. The normative intent of any contribution to the CBOR ecosystem must be to aid making this reality more universal, instead of encouraging a world of dominating interoperability concerns. (Partial implementations have their place, see below, but should not be the usual situation an application developer is by default confronted with.) Since another revision of is likely to be needed, this review does not yet attempt detailed checking of the examples.

Conventions and Definitions For brevity, this revision of this document uses the definitions of RFC 8949 and of . Please do consult these definitions while reading the present document. Additional Terms:

Service:: A potential function of a CBOR processor implementation. This includes services that are combined with the core services provided by that implementation, for convenience and ease of implementation.
Partial implementation enabler (PIE):: An agreement between producers and consumers of data items used in a CBOR-based protocol that allows for partial implementations on the consumer side.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Configuration A CBOR application often makes use of a CBOR library (more generally speaking, CBOR implementation) to perform certain services. There may be a need to communicate preferences or instructions from the CBOR application to the CBOR library on how to perform the services; this document uses the term "configuration" in this specific sense. Some parameters of a configuration mechanism provided may be entirely local, such as memory allocation parameters. Some may be unilateral, i.e., they influence the generated CBOR, but the consuming application does not need to agree to these influences. Some may be interoperability constraints, i.e., certain parameters need to be known on both the producing and the consuming side.

Serialization constraints restrict the serialization variants available for CBOR data items.
- This may be motivated by enabling a partial implementation on the consumer side which has to deal with fewer variants.
- In the area of serialization, the CBOR application may want to make use of deterministic serialization, which needs to be explicitly configured on the producer side and usually (depending on application requirements) also on the consumer side.
Data model subsetting may be a service requested by an application that does not need to handle certain CBOR data items. For instance, a CBOR application may not need to handle floating point values. This can enable compiling the library used with special flags to save the code space that would be needed (partial implementation enabler). Data model subsetting on the consumer side typically is an interoperability constraint (certain values can no longer be interchanged).
Normalization services may be provided by a CBOR library as a service that offloads some application code into the library, e.g., if the application desires to have all text strings sent in a Unicode Normalization Form (such as NFC). In certain cases, normalization services may be performed at the consuming side before ingested CBOR is handed to the application; this needs to be handled with caution as the application no longer sees what was actually interchanged.

Partial Implementations CBOR has been designed to be useful in constrained environments. It therefore embraces partial implementations, i.e., implementations that do not provide access to all the features of the CBOR representation format. Partial implementations often have interoperability considerations: They can only be used for CBOR-based protocols that do not exercise features that have been left out (for instance, a constrained implementation that does not provide floating point values cannot be used for high-resolution timestamps based on CBOR Tag 1). To a certain extent, CBOR-based protocols can accommodate partial implementations: A protocol that uses Tag 1 could restrict the tag content to an integer, enabling the use of partial implementations without floating point support. While CBOR does embrace partial implementations in this way, this does not mean a receiving CBOR protocol implementation is free to leave out features and expect the CBOR generator to magically know that this will be the case and implement workarounds. Instead, the use of a partial implementation needs to be enabled; the present document speaks about partial implementation enablers. While there is an infinite number of implementation omissions that could be made in a CBOR implementation, certain omissions are widely employed and therefore may be worth to be explicitly called out, assigning them a name. This facilitates using these names in the definition of a CBOR-based protocol and in the configuration interface of a CBOR generator. Some of these are about serialization configuration, see ; this leads to a small number of easy-to-implement and widely considered useful partial implementation enablers such as "definite-length only" (DLO) or "no integer overlap" (NIO). Some of these are about data model subsetting (e.g., no floating point), which actually have implications on the application data model and are therefore much more numerous, but also don't need library support (can be left to the control of the application).

Serialization Configuration

Example: JSON For a given value, there are often a large number of JSON texts that could be used to serialize it. For instance:

The number format provides significant serialization variation, e.g., (in the interpretation) the number 42 can be serialized in an infinite number of ways, as 42, 42.0, 42.00, 42.0000000 (adding trailing insignificant zeros), 42e0, 4.2e1, .42e2, 420e-1 (using exponents in various ways), 42e00, 4.2e01, 4.2e+1, 4.2e+01 (adding sign characters and/or insignificant zeros to an exponent), etc.
Each character in a text string may provide a choice between directly including that character in the JSON text and using a \uNNNN or \uHHHH\uLLLL escape; the / character can also be escaped as \/.
The serialization of containers (arrays and JSON objects (maps)) provide for the insertion of blank space between elements/entries. Blank space can also be added outside (before or after) a serialized JSON value to make a JSON text.

JSON Implementations: Serialization Controls There are probably a few hundred "serious", open-source general-purpose JSON implementations. These rarely provide controls that choose specific serialization variants, beyond those that relate to human consumption of the serialized JSON text (i.e., pretty printing). With luck, there may be a control for the text string escaping choosing between direct representation of all Unicode characters where JSON allows this vs. an "ASCII-only" mode escaping non-ASCII characters.

JSON Deterministic Serialization A specification for deterministic serialization of JSON data exists . This is usually not implemented as a serialization option by general JSON implementations, but by specially made JCS implementations (of which there probably are a couple of dozen "serious", open-source ones in existence). For example, the implementation of JCS number formatting is generally considered highly error-prone.

Example: CBOR For a given CBOR data item, there are often multiple encoded CBOR data items ("encodings") that can be used to serialize it. Deterministic serializations reduce this number to exactly one variant. In contrast to JSON, CBOR explicitly embraces partial implementations of the CBOR data format. A partial implementation at a producer may involve data model subsetting in the CBOR library; unless that data model subsetting is part of the application data model, this may mean limitations to the expressiveness that the producer can make use of. As it is part of the library configuration it hands down, the producer application will be aware of this subsetting and can implement strategies for dealing with this subsetting (e.g., the application might round tag 1 timestamp values to the nearest integer if the additional resolution provided by floating point values is not needed). If data model subsetting is to enable the consumer to use a partial implementation, the producer needs to be aware of this; this then becomes an interoperability constraint. This document speaks about a Partial implementation enabler (PIE): An agreement between producers and consumers of data items as defined in a CBOR-based protocol that allows for partial implementations on the consumer side. A PIE may place additional onus on the producer side, including the application (e.g., converting all timestamps to integer in order to avoid the interchange of floating point numbers), and/or reduced onus (no need to encode floating point numbers if they are not used in the data model).

CBOR Implementations: Serialization Controls Handling interoperability in an ecosystem that embraces partial implementations requires more discipline.

Deterministic Serialization for CBOR Deterministic serialization is generally easy to achieve in CBOR and in many cases can be based on a generic implementation (encoder/decoder). There are several different (completed and draft) specifications for deterministic serialization:

A common format specified in Section of RFC 8949 .
A legacy specification in Section of RFC 8949 , compatible to a previous specification (called "Canonical CBOR" there). (Common and legacy deterministic serialization differ in the ordering of maps.)
, a "bespoke serialization of CBOR intended for use with the special tag 42 in various end-to-end protocols that came out of the IPFS community".
, also based on Section of RFC 8949 , but mixing some unrelated data model subsetting into the configuration.

Map sorting obviously requires additional effort in an encoder. Map sorting can help taking an onus off the consumer: Being able to process the map entries in any order can take more code/be less efficient for the consumer than knowing what order they will arrive in. (The latter does require making the order available to the application processing.) (CoAP Option processing is an example for how knowing the order in which entries arrive can be quite beneficial for a constrained implementation.)

Data Model Subsetting The CBOR library may be made aware that certain parts of the CBOR generic data model are not used in the application data model of a CBOR application, possibly leading to a reusable subset of the generic data model. In this case, the producer may want to do a poor form of data model validation, and the consumer may do the same or simply reject input that doesn't respect the data model subset. The producer also may want to offer a normalization service (), possibly in support of data model subsetting. A typical example is a constrained implementation for a platform that doesn't provide a platform type for floating point numbers, or maybe only for float32 and not float64. In the latter case, a normalization service could round float64 numbers to a close float32 number. Whether that is acceptable for the application needs to be defined by the CBOR-based protocol. A disadvantage of performing data model subsetting in the CBOR library is that, unless there is detail information supplied by some data definition (e.g., an extension to CDDL), it applies equally whole-sale to the entire CBOR data item. Data model subsetting can be a partial implementation enabler (PIE); a bespoke CBOR consumer does not need to implement parts of the CBOR functionality that are not in the protocol's data model subset.

Normalization Normalization services may be a convenient addition to a CBOR library. For example, an application protocol that requires text strings to be in some Unicode normalization form (in many cases where this is needed, that will today be NFC) may want to flip a bit in the encoder configuration instead of doing the processing in the application implementation. Similarly, an application protocol that needs NaNs as a distinct data type but only one NaN value may want the encoder to normalize all NaN values to the chosen one. Normalization is visible at the application layer; users of CBOR libraries need to be aware of which normalization processes are enabled in the library. The CBOR-based protocol usually does not know (or care) whether the normalized values came from the application or from a normalization service in a library. As with data model subsetting, a disadvantage of performing normalization in the CBOR library is that, unless it is controlled by some data definition (e.g., an extension to CDDL), it applies whole-sale to the entire CBOR data item. On the other hand, normalization will typically not reach into the internals of CBOR data items, in which case embedded CBOR or homogeneous tagged arrays might not be covered.

Other WGLC Feedback

"New Tag Data Model Rule"

WGLC : attempts to reflect a valid concern that has been brought to light in the WG's discussion of the special role of CBOR tag 2/3. The section needs to be more specific in what way new tags may not "affect" existing data types of CBOR. (Does tag 25 "affect" strings? Some relevant discussion about serialization aspects also in .) The observation identified here is that tag 2/3 can be used not only to serialize data items not in the basic generic data model (i.e., outside the space already covered by mt0/1), but can also be used as an alternative serialization of existing integer data items that also can be expressed as mt0/1. This can be called the "integer overlap". Supporting the integer overlap in CBOR was a deliberate decision, enabling e.g. a more efficient representation of 40-bit integers; more generally it is part of the strategy of CBOR to work well with no longer than 64-bit arguments (i.e., "ai" values from 28 to 30 are not used). It does have consequences on how a generic decoder needs to work. (This makes decoding tag 2/3 a requirement for generic decoders, even if the application data model does not contain integers outside mt0/1.) In short, while acceptable for the legacy case of integers, implementing such an overlap for a new tag would require more effort than the WG found acceptable for definitions of new tags. For certain applications, a "no integer overlap" PIE for the tag 2/3 case is useful. (No integer overlap is also implied by Preferred Serialization, see Section of RFC 8949 .) Beyond that, it is not so easy and will be time-consuming to formulate an exact rule for what a new tag is allowed or is not allowed to do; the important contribution here might be writing up the problem and giving the protocol designer (and later the Designated Expert) enough information to diagnose whether the concern would apply to a new tag.

CDDL Serialization Control Operator

WGLC : The new ".serial" CDDL control operator introduced in is only sketched. The CDDL processing model does not provide for controlling serialization. Functionality like this can be added to CDDL by extending its processing model, but generally requires more careful considerations. Questions such as how nested applications of the operators combine have already come up on the mailing list. The registry to be set up appears to try to limit the kinds of serialization constraints that can be expressed.

New controller values for new serializations are possible but are highly discouraged. Standards action is required to add them.

Given that there are several deterministic serializations in use, as well as a number of partial implementation enablers identified, this is not going to address a sufficient part of reality. The solution definitely should make it easy to do the right thing. It should not try to make it impossible or very hard to express not doing a greenfield design where a different deterministic encoding is already deployed.

NaNs (Appendix C)

The technical arguments are well known, they will not be repeated here. What is maybe less obvious is what role CBOR plays in this. There actually is no requirement that CBOR has to heal all of IEEE 754's shortcomings in this space before it can make floating point values available for interchange. CBOR's data model calls for serializations in binary64, and, for preferred serializations, in binary32 and binary16. For platforms that already support IEEE 754, it is usually trivial to obtain a binary64 that is ready for interchange. (Certain legacy platforms such as PA-RISC and pre-R6 MIPS may have an additional step, but that is well-known for those platforms.) The small piece of work a CBOR implementation has to do here (if it wants to support deterministic encoding) is narrowing these values to binary32 and binary16 where that is possible. defines exactly how this is to be done, for the purposes of CBOR. shows one way how to do this, which was example code sorely missing from previous documents about CBOR. Since most modern CPUs have instructions for doing this work (to which CBOR is compatible by design), there may be an even simpler way to do this. The WG may not have a lot of interest in financial or instrumentation applications of CBOR. This does not mean we should simply steamroll over them by trying to create a world where NaNs are actively ostracized.

Implementation aid (Appendix D)

Serialization Checking (Appendix H)

Nits

Consistent Notation for encoded data items The notation for encoded data items switches between a hex string preceded with a 0x, as in: or and a sequence of bytes as in (The habit of using the 0x prefix for notating encoded data items has been criticized for good reasons but seems hard to get rid of; this comment is just about consistently using one form.)
Confusing statements about COSE

It is also RECOMMENDED that CBOR libraries support deterministic serialization, as some protocols (for example, COSE) require it.

While this statement is technically true, what COSE needs is a rather limited form of producer support for COSE's deterministic encoding (which is defined on a data model subset that makes it compatible with both CDE and LDE), to enable creating signing input only. COSE does not use deterministic encoding for anything exchanged over the wire. The text looks like it is saying that, and it certainly will be misinterpreted and misquoted.

Terms Bignum, Big Integer

The various terms in use here are ambiguous whether they refer (1) to the numeric range or (2) to the specific encoding by either mt0/1 or tag 2/3. We probably should decide on two pairs of terms and unambiguously define them.

Terms Subset/Superset

Most of the configuration options discussed in the present document are about requiring subsetting the set of choices. Combining requirements can also be perceived as creating a superset of requirements. These interpretations with an opposing sense are probably best avoided unless accompanied by text that makes clear which kind subset/superset is being discussed. There are related sentences such as (5.1):

All of preferred-plus serialization defined in Section 4.1 MUST be used.

This probably does not mean that all of preferred-plus needs to be exercised, but that all requirements stated must be fulfilled. (The unease some of us feel about "plus" is another manifestation of this ambiguity.)

Term "Preserve"

A CBOR library is not required to preserve the order of keys when decoding a map,

The point here appears to be that there is no need for the CBOR application to be informed about the order of the entries in the map. Whether that would be trivial ("preserve") or hard to do (e.g., if the platform only has an unordered map) depends on the platform.

Security Considerations TODO configuration security

WGLC : The security considerations of only seem to be concerned with exploiting serialization variation as a covert channel (between the CBOR encoder and a third party?). This is somewhat surprising, as JSON and a lot of other IETF specifications have exactly the same considerations. More generally, if the document wants to focus on this subject, it should provide context on how this is a general problem of a wide selection of protocols, and probably reference research such as . One solution suggested in the text is "to require preferred-plus or deterministic serialization". A CBOR-based protocol that requires a deterministic serialization indeed reduces the attack surface. However, preferred-plus serialization should not be listed as a solution: it provides a false sense of security unless map ordering also is deterministic; only the agreement on actual deterministic encoding provides at least some protection in a checking decoder. (Data model subsetting can be combined with preferred-plus, limiting any maps encoded to zero or one entries, to achieve a specialized variant of deterministic encoding. It is not really clear why a somewhat risky combination of configuration settings are acceptable here when explicitly identifying the encoding as deterministic would be straightforward instead.)

IANA Considerations This document has no IANA actions.

References Normative References Key words for use in RFCs to Indicate Requirement Levels In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. Informative References CBOR Common Deterministic Encoding (CDE) Universität Bremen TZI CBOR (STD 94, RFC 8949) defines the concept of "Deterministically Encoded CBOR" in its Section 4.2, determining one specific way to encode each particular CBOR value. This definition is instantiated by "core requirements", providing some flexibility for application specific decisions; this makes it harder than necessary to offer Deterministic Encoding as a selectable feature of generic CBOR encoders. The present specification documents the Best Current Practice for CBOR _Common Deterministic Encoding_ (CDE), which can be shared by a large set of applications with potentially diverging detailed application requirements. The document also discusses the desire for partial implementations, which can be another reason for constraining CBOR encoders, and singles out the encoding constraint "definite-length-only" as a likely constraint to be used in application protocol and media type definitions. This specification updates RFC 8949 in that it provides clarifications and definitions of additional terms as well as more examples and explanatory text; it does not make technical changes to RFC 8949. // This revision -13 merges all active pull requests in preparation // for the 2025-cbor-17 interim on 2025-10-15. Concise Binary Object Representation (CBOR) The Concise Binary Object Representation (CBOR) is a data format whose design goals include the possibility of extremely small code size, fairly small message size, and extensibility without the need for version negotiation. These design goals make it different from earlier binary serializations such as ASN.1 and MessagePack. This document obsoletes RFC 7049, providing editorial improvements, new details, and errata fixes while keeping full compatibility with the interchange format of RFC 7049. It does not create a new version of the format. Concise Binary Object Representation (CBOR) The Concise Binary Object Representation (CBOR) is a data format whose design goals include the possibility of extremely small code size, fairly small message size, and extensibility without the need for version negotiation. These design goals make it different from earlier binary serializations such as ASN.1 and MessagePack. Obsoleted by , cited here for historical interest in legacy Section 3.9 "Canonical CBOR" The tag-42 profile of CBOR IPFS Foundation IPFS Foundation This document defines a bespoke serialization of CBOR intended for use with the special tag 42 in various end-to-end protocols that came out of the IPFS community. Much of its design dates to the first CBOR RFC and predates much of the terminology and the layered approach to determinism elaborated in later years. CBOR-42 can be used as an internet-scale serialization for JSON, and is optimized for objects that compose into a directed acyclical graph. Since CBOR-42 objects link to one another by hash-based identifiers tagged "42", deterministic encoding is mandated to verify dereferenced links and encode new ones. FIX ME: https://github.com/ietf-tools/bibxml-service/issues/458 The JavaScript Object Notation (JSON) Data Interchange Format JavaScript Object Notation (JSON) is a lightweight, text-based, language-independent data interchange format. It was derived from the ECMAScript Programming Language Standard. JSON defines a small set of formatting rules for the portable representation of structured data. This document removes inconsistencies with other specifications of JSON, repairs specification errors, and offers experience-based interoperability guidance. The I-JSON Message Format I-JSON (short for "Internet JSON") is a restricted profile of JSON designed to maximize interoperability and increase confidence that software can process it successfully with predictable results. JSON Canonicalization Scheme (JCS) Cryptographic operations like hashing and signing need the data to be expressed in an invariant format so that the operations are reliably repeatable. One way to address this is to create a canonical representation of the data. Canonicalization also permits data to be exchanged in its original form on the "wire" while cryptographic operations performed on the canonicalized counterpart of the data in the producer and consumer endpoints generate consistent results. This document describes the JSON Canonicalization Scheme (JCS). This specification defines how to create a canonical representation of JSON data by building on the strict serialization methods for JSON primitives defined by ECMAScript, constraining JSON data to the Internet JSON (I-JSON) subset, and by using deterministic property sorting. CBOR Serialization and Determinism Security Theory LLC This document defines two CBOR serializations: "preferred-plus serialization" and "deterministic serialization." It also introduces the term "general serialization" to name the complete set of all serializations defined in RFC 8949. Together, these three form a set of serializations that cover the majority of CBOR serialization use cases. These serializations are largely compatible with those widely implemented by the CBOR community. Notable CBOR Tags Universität Bremen TZI The Concise Binary Object Representation (CBOR, RFC 8949) is a data format whose design goals include the possibility of extremely small code size, fairly small message size, and extensibility without the need for version negotiation. In CBOR, one point of extensibility is the definition of CBOR tags. RFC 8949's original edition, RFC 7049, defined a basic set of 16 tags as well as a registry that can be used to contribute additional tag definitions [IANA.cbor-tags]. Since RFC 7049 was published, at the time of writing some 250 definitions of tags and ranges of tags have been added to that registry. The present document provides a roadmap to a large subset of these tag definitions. Where applicable, it points to an IETF standards or standard development document that specifies the tag. Where no such document exists, the intention is to collect specification information from the sources of the registrations. After some more development, the present document is intended to be useful as a reference document for the IANA registrations of the CBOR tags the definitions of which have been collected. You Can’t Do That on Protocols Anymore: Analysis of Covert Channels in IETF Standards National Research Council of Italy (CNR), Institute for Applied Mathematics and Information Technologies ”Enrico Magenes” (IMATI), Genova, Italy Institute of Computer Science, Warsaw University of Technology, Warsaw, Poland Institute of Electrical and Electronics Engineers (IEEE)

Acknowledgments TODO acknowledge.