Subject: [Anima-signaling] Fwd: Re: CBOR/JSON

From: Brian E Carpenter <[email-masked]>
Date: Fri, 29 May 2015 08:16:58 +1200

FYI
   Brian

-------- Forwarded Message --------
Subject: Re: CBOR/JSON
Date: Thu, 28 May 2015 15:31:24 +0300
From: Markus Stenberg <[email-masked]>
To: Brian E Carpenter <[email-masked]>
CC: Markus Stenberg <[email-masked]>

On 28.5.2015, at 0.39, Brian E Carpenter <[email-masked]> wrote:
> We're having a big debate over in the Anima signaling design team
> about whether we should switch from a binary TLV format to a
> JSON/CBOR format. Clearly there is a run-time penalty in that but
> it would bring flexibility and convenience for defining new autonomic
> objectives, and that is a very important aspect for us.
> 
> Mark Townsley told me that you considered this choice at the beginning
> of the HNCP/DNCP saga. Can you comment on why you settled on a binary
> TLV? I’d like to pass that on to the design team, if that's OK.

For implementor, binary TLVs are _much_ easier to parse/produce than JSON and they’re also more efficient on the wire.

This is assuming low-level language where you want to minimize the footprint; if you’re dealing with something that is ‘hipster
enabled’ and has built-in json parser[1], this statement is not true anymore.

CBOR hasn’t received so far runaway success so I cannot comment on it; it’s encoding is (again) less efficient and parsing
harder than binary blobs, but not as much as text.

Sticking binary data in json leads to fun questions like ‘do I encode this in hex or what?’

TBH, I am not sure there is any case why you would _want_ JSON in a low-level protocol; certainly higher up the stack when you
deal with mostly with strings (that have well-defined encoding), but dealing with mostly binary data as strings is bad idea and
pain to implement.

Cheers,

-Markus

[1] Historic version of hnet work actually _did_ use JSON inside OSPFv3 AC LSA TLVs, but that was only for my private extension
purposes. (I wrote that prototype in Lua so there were built-in json functions.)

---

From: Toerless Eckert <[email-masked]>
Date: Thu, 28 May 2015 13:58:30 -0700

Lets keep Markus on the thread replying to his arguments...

Markus:

Just a quick round of pro-CBOR arguments:

- I do not want to make TLV parsing/generation easier, i want
  to eliminate the need for a lot of coders having to redo it over and
  over:

- There is a wide range of code quality generated by different coders
  and under different commercial conditions (time cost pressure,
  care for safety/reliabilty). Consider the code quality
  generated by the average commercial situation compared to re-using
  a (eg: CBOR) libraries that are continuously re-vetted.

- According to Cabo, he has seen a TLV encodings that where
  less efficient than CBOR. Which is to me attestation of a lot of
  low level designs taking a lot of shortcuts and end up being less
  extensible mid-term.

- Personal data point: Extending IPFIX with additional hierarchical
  subdomains. The IETF process is hard enough, the coding effort in 
  implementations is even worse.

- Aka: In general i am sick and tired of having to backfit extensions
  into closed minded TLV designs, and there is no way to have the
  same degree of extensibility designed into a randmon new design as
  would be in CBOR/JSON. If you did, you'd just reinvent CBOR/JSON,
  and the whole exercise of doing your own TLV design is to get away
  with less work. Which then just comes with all these catch22. More
  generally: Hierarchical structure is much less baked into TLV designs
  as into structured data designs.

- Diagnostics and policy language in network devices really suffer
  from binary TLVs. Most likely TLV designs have no good idea of
  transitive transparency of unknown TLVs while this is a lot more
  commonly well defined when you start with XML/HTML/JSON, and once
  you have this extensibility but binary TLV you have no way on older
  devices to deal with that stuff. It's all just numeric. A mix of
  binary and for extensions strings like possible in CBOR looks like
  a very attractive option to investigate.

Cheers
    Toerless

On Fri, May 29, 2015 at 08:16:58AM +1200, Brian E Carpenter wrote:
> > TLV? I???d like to pass that on to the design team, if that's OK.
> 
> For implementor, binary TLVs are _much_ easier to parse/produce than JSON and they???re also more efficient on the wire.
> 
> This is assuming low-level language where you want to minimize the footprint; if you???re dealing with something that is ???hipster
> enabled??? and has built-in json parser[1], this statement is not true anymore.
> 
> CBOR hasn???t received so far runaway success so I cannot comment on it; it???s encoding is (again) less efficient and parsing
> Sticking binary data in json leads to fun questions like ???do I encode this in hex or what????
>

---

From: Markus Stenberg <[email-masked]>
Date: Fri, 29 May 2015 17:25:26 +0300

>  to eliminate the need for a lot of coders having to redo it over and

This is the main reason for using ASN.1 or YANG+x, not CBOR. You have multiple encodings, well-defined, and more efficient too than CBOR some of them, based on a single schema, and still extensible as needed :)

CBOR states its schemaless-ness is a feature, but it is actually a bug; you want to an extensible schema if you go towards something pre-defined here. Otherwise you wind up parsing/validating data structure subsets your parser produces _anyway_ to make sure they at least fullfill the criteria you want, by hand, before actually doing anything with it. 

There is a number of binary JSON encodings, and while CBOR is from IETF, I don’t see it in the wild in the things I do, so I do not particularly have strong opinion either way about it. Raw JSON I consider a bad idea if you need to transport binary data as it is still no longer readable etc, and number of parsers are not actually 100% binary-proof so hello hex string encoding.

>  a (eg: CBOR) libraries that are continuously re-vetted.

Considering the number of bugs I have seen in e.g. popular JSON parsers (json-c), I am not as optimistic as you are about their quality. But I am sure ’this time it is different’.

>  low level designs taking a lot of shortcuts and end up being less

Sure, you can define better or worse TLV encodings. CBOR is moderately efficient but not insanely so (I could define more efficient one, but there is a tradeoff between encoding efficiency and en/decode complexity obviously.)

>  as into structured data designs.

JSON is crap for this usecase, so please just leave it out.

If you want structured data, you want also schema so CBOR is not really helpful in my opinion.

>  a very attractive option to investigate.

See above. CBOR is just modern ASN.1 _encoding_. XML is again not so great if you need to transport lots of binary content (hello, hex blobs in XML and you suddenly have TLV-ish stuff inside XML binary blobs rapidly).

Anyway, I would recommend ASN.1 over CBOR personally. It is mature, proven technology with actually superior features in this use case. Another option would be YANG schema encoded in XML or JSON (or even CBOR, if someone defines a mapping for it), if you want to pursue this road. All of these options are extensible, and _provide validation for “free"_, and _well-defined schema to stick in your standard_.

Finally, all this said, my original recommendation still stands if you do not want schema (CBOR in and of itself I consider awkward because you define want schema in your document); I have had not so fun time with ASN.1, JSON, XML, and various binary JSON variants over the years and usually they have _not_ been easier than relatively simple binary formats you can define for particular application. 

Cheers,

-Markus

---

From: Markus Stenberg <[email-masked]>
Date: Fri, 29 May 2015 17:25:26 +0300

(resent just to list; Toerless had some issues whitelisting + approving the original one)

Cheers,

-Markus

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Fri, 29 May 2015 17:22:40 +0000

On 5/29/15, 8:25 AM, "Markus Stenberg" <[email-masked]> wrote:

>This is the main reason for using ASN.1 or YANG+x, not CBOR. 

Speaking as someone who made their bones doing an XML-based protocol (XMPP), I strongly urge against using XML for any new protocol work.  It's just barely still viable for doing document-based work (such as our new RFC format).  We now have at least one (JSON) and maybe two (CBOR) alternatives that are better at programability, interoperability, and security (JOSE/COSE vs. XML Dsig and friends).

My position on ASN.1 is well-documented.  If you can't parse all possible inputs without having the schema, your format is a non-starter going forward.

>You have multiple encodings, well-defined, and more efficient too than CBOR some of them, based on a single schema, and still extensible as needed :)

Multiple encodings are bad for interoperability.  I dare you to prove with data that either ASN.1 or XML is more efficient than CBOR in code size, execution speed, memory utilization, power utilization, or wire size.

Maybe there is some other axis on which one of them is better (perhaps "formal extensibility in a way that nobody understands" for XML) that you are referring to?

>CBOR states its schemaless-ness is a feature, but it is actually a bug; you want to an extensible schema if you go towards something pre-defined here. 

So that's basically impossible with DTDs for XML.  It's REALLY hard to do with XML Schema, and it's possible in an annoying way with RelaxNG.

ASN.1's schema extensibility works, but has had problems with extensions breaking the syntax parse, disallowing the most important feature of modern protocols: ignoring new features you do not understand.

>Otherwise you wind up parsing/validating data structure subsets your parser produces _anyway_ to make sure they at least fullfill the criteria you want, by hand, before actually doing anything with it.

You're going to have to do that no matter what schema protections you think you have.  One of the key things we learned with "guard" technology for XMPP was that the schemas were never 100% correct versus the implementations that would otherwise interoperate.  For example, the ordering of XML elements in XML Schema are significant, but the order is not actually interesting to most protocol-based uses of XML.  Doing true schema validation at every processing point adds massive amounts of CPU and latency overhead without adding any real-world value.

I have experience building these systems, deploying them at scale, and selling them to some of the most difficult customers in the world, so this is not just gut feel on my part.  It's hard lessons learned in an operational environment.

>There is a number of binary JSON encodings, and while CBOR is from IETF, I don’t see it in the wild in the things I do, so I do not particularly have strong opinion either way about it. 

Appendix E of RFC 7049 addresses the most popular of those.

>Raw JSON I consider a bad idea if you need to transport binary data as it is still no longer readable etc, and number of parsers are not actually 100% binary-proof so hello hex string encoding.

if by "a number of" you mean "all", then yes.  Doing binary in JSON is a major PITA.  That's one of the main benefits of CBOR.

>Considering the number of bugs I have seen in e.g. popular JSON parsers (json-c), I am not as optimistic as you are about their quality. But I am sure ’this time it is different’.

JSON is harder to parse than anyone thinks.  That's another benefit of CBOR.

Regardless, having a small number of parsers makes finding bugs more likely than having a different parser at each layer in the protocol stack.

>Sure, you can define better or worse TLV encodings. CBOR is moderately efficient but not insanely so (I could define more efficient one, but there is a tradeoff between encoding efficiency and en/de…

Agree.  CBOR was written to balance out a bunch of competing requirements, and I think (other than the chunked encoding stuff, which I would profile out of most protocols) the team did a pretty good job on that balance.

>JSON is crap for this usecase, so please just leave it out.

Do you mean here the ability to encapsulate binary, or something else?  If something else, I'd like to know a little more about what you're thinking.

>If you want structured data, you want also schema so CBOR is not really helpful in my opinion.

I don't understand this point.  CDDL (draft-greevenbosch-appsawg-cbor-cddl) is being worked on.  There's nothing about CBOR that prevents schema-fication.

>See above. CBOR is just modern ASN.1 _encoding_. 

The data is shaped slightly differently from ASN.1 as well, removing some of the more complex bits that are hard to implement, hard to interoperate with, etc.  Regardless, it's certainly more modern, based on a couple of things we've learned in the last decades.

>XML is again not so great if you need to transport lots of binary content (hello, hex blobs in XML and you suddenly have TLV-ish stuff inside XML binary blobs rapidly).

XML is horrible for protocols in almost every direction.

>Anyway, I would recommend ASN.1 over CBOR personally. It is mature, proven technology with actually superior features in this use case. 

I haven't heard one yet.  I do admit that it depends what developer audience you're hoping will implement your specifications.

>Another option would be YANG schema encoded in XML or JSON (or even CBOR, if someone defines a mapping for it),

I actually think a CBOR mapping from YANG is interesting work.  I did a quick peek at what that would take a couple of weeks ago, and it looks doable.  It would certainly be a nice architectural approach.

>if you want to pursue this road. All of these options are extensible, and _provide validation for “free"_, and _well-defined schema to stick in your standard_.
>

Again, I think it depends on the target developers somewhat.  Folks who have grown up not needing to write their own parsers, relying solely on JSON.parse and friends, are quite likely to throw up their hands and move on to the next thing rather than figure out a new format that doesn't yet have a library for their programming language.

-- 
Joe Hildebrand

---

From: Markus Stenberg <[email-masked]>
Date: Fri, 29 May 2015 21:53:23 +0300

The tone on the list sounds so friendly and all-knowing that this is my last comment, feel free to drop me from the comments. I did the original comments on my free time as a favor and clearly it was not worth the trouble, and you clearly also have all you need on the list already.

> Speaking as someone who made their bones doing an XML-based protocol (XMPP), I strongly urge against using XML for any new protocol work.  It's just barely still viable for doing document-based wor…

Agreed, XML is a bad idea.

> Multiple encodings are bad for interoperability.  I dare you to prove with data that either ASN.1 or XML is more efficient than CBOR in code size, execution speed, memory utilization, power utiliza…

Well, YANG can be encoded as XML, JSON (or potentially CBOR). Is it bad? No. Actual protocols using YANG choose which encoding to use. Same with ASN.1.

Only meaningful comparison is in terms of wire size - there are cases with some ASN.1 encodings (e.g. PER) but obviously you do not want to go there. (And I am not saying the signaling work should, either, but I would say it is better than _pure_ CBOR. Not (schema)+CBOR, though.)

The subjective measures you quote such as code size, execution speed, memory utilization, power utilization are all _for_ not using CBOR but instead custom C structs. e.g.

struct foo_struct __attribute__(packed) {
 .. things
}

if (buflen != sizeof(foo_struct)) .. error; else deal with foo_struct

.. or ASN.1 schema based generated code that can be actually more efficient due to being non-generic (but still extensible given correct encoding), but I do not have the interest to prove it to you particularly.

> ASN.1’s schema extensibility works, but has had problems with extensions breaking the syntax parse, disallowing the most important feature of modern protocols: ignoring new features you do not understand.

Depends on encoding; not with all of them. Been years, but I have seen e.g. X.509 work with bonus extensions not breaking existing things. Obviously with PER you have to know schema in advance. 

> You're going to have to do that no matter what schema protections you think you have.  One of the key things we learned with "guard" technology for XMPP was that the schemas were never 100% correct…

Lots of code that deals with JSON/XML stuff is like

if foo1 present ok1 else error
if foo2 present ok2 else error
..
repeat times N

given schema based validator+code generator you get those checks for free, and can just deal with ok1/ok2 processing given (optional) code generator + en-decode code.

> I have experience building these systems, deploying them at scale, and selling them to some of the most difficult customers in the world, so this is not just gut feel on my part.  It’s hard lessons learned in an operational environment.

I have been around the block too, thank you. probably 1e8+ shipped things also some under your own $MEGACORP brand. Perhaps your scale is 1e9+ so I bow to your superior wisdom.

> if by "a number of" you mean "all", then yes.  Doing binary in JSON is a major PITA.  That’s one of the main benefits of CBOR.

I have seen (and written) some that do it correctly. However, the point of using a human-readable format (JSON) to transport escaped binary is pointless and I guess we agree on that.

> I don't understand this point.  CDDL (draft-greevenbosch-appsawg-cbor-cddl) is being worked on.  There’s nothing about CBOR that prevents schema-fication.

CDDL is not CBOR. The point is, for reasonable standard work, you _want_ schema and encoding, I do not particularly care which. I guess there is no RFC option for that yet (CDDL or some hypothetical YANG+CBOR). Encoding in and of itself is not of significant value either to protocol specifier, or for implementator following the specification.

> I actually think a CBOR mapping from YANG is interesting work.  I did a quick peek at what that would take a couple of weeks ago, and it looks doable.  It would certainly be a nice architectural ap…

It is doable (as the schema semantics map to JSON/XML in the netconf land). 

Cheers,

-Markus

---

From: Toerless Eckert <[email-masked]>
Date: Fri, 29 May 2015 13:17:07 -0700

Markus,

For me this thread is very helpfull because i can't compete with your
or Joe's 1e<N large> coding experience, so i am trying to learn from both your insights.

  One background i ihave is my 1e<more than i cared> non-interop
  deployment experience with varius X.N00 applications back in the 90th.
  IMHO, ISO software back then was written on the principle of "Send/generate
  based on your liberal interpretation of humunguous specs and accept on input
  only what you can not find a good reason for to reject". And diagnostics
  was a pain. Every time i had to do it, it somehow was related to something ASN.1. 

So, the one property i'd like to have is an encoding where the parser does not
have to give up upon an unexpected element or mismatch of schemata but can proceed -
with the maybe just a (very often optional) part of the parse-tree missing/ignored. And
in my understanding (correct me if i am wrong), the best ways to achieve this
are simple self-descriptive structured encodings implemented via widely reused SDKs
(so to minimize actual paser errors in the code). Thats what i was hoping to get
from CBOR. And to the best of my understanding, it would be lot harder to get
with PER.

I would certainly like to better understand what the best options are to
leverage schematas to define, create&validate and parse the encodings of the structured
data.

Yang: sure, we can define the object model/schemata with Yang, but we should agree why:

-> Yang is often used to enable multiple encodings. I think we should do a good enough
   job upfront for anima that we do not need multiple encodings. As for me, whatever
   encoding comes up, even if i don't like it, i'd rather stick to just one encoding.

   Lets say anima decides on XML and wakes up later learning that doesn't fit IOT
   encoding, and needs to add a second encoding - that where yang would be very
   helpfull to ensure consistent definition across encoding. But i think avoiding
   this to happen is why we have this discussion here.

-> Yang is a common language for definition making mutual review easier.
   Thats i think the important, but only thing Yang gives us in Anima.
   .. what am i missing ?

I have seen enough efforts that simply tried to understand the constraints of Yang,
then defined everything first in their encoding, and later added a Yang definition.
This was mostly done in recent years when Yang toolchains where not always
available.

Which brings me to the create&validate/parse coding question: I don't know what
are really good & lightweight toolchain options that would make this attractive to
be schema based. If for example i look into encoding JSON, it seems its very
much depending also on your programming language. If you use JS, you pretty much
jave JSON as part of the language itself, so schema creation is 1:1 programming
language. And hopefully there would be an easy way to permit/check just the CBOR
compatible subset.

Parsing/Validation on receipt based on schema & SDK for it would be great. I also
thought that therre might be JSON schema definitions that ould be used for that
against the CBOR encoding... Yes/No ? And of coure those CBOR compatible JSON
schemas could be created from Yang once there are appropriate toolchains.

And the beauty could be that all of these schemata related aspects could be 
optoinal and be used when deemed beneficial by the coder - and not impacting
the parsability of the self-descriptive encoding..

Cheers
    Toerless

On Fri, May 29, 2015 at 05:25:26PM +0300, Markus Stenberg wrote:
> There is a number of binary JSON encodings, and while CBOR is from IETF, I don???t see it in the wild in the things I do, so I do not particularly have strong opinion either way about it. Raw JSON I consider a bad idea if you need to transport binary data as it is still no longer readable etc, and number of parsers are not actually 100% binary-proof so hello hex string encoding.
> 
> Considering the number of bugs I have seen in e.g. popular JSON parsers (json-c), I am not as optimistic as you are about their quality. But I am sure ???this time it is different???.
> 
> Anyway, I would recommend ASN.1 over CBOR personally. It is mature, proven technology with actually superior features in this use case. Another option would be YANG schema encoded in XML or JSON (or even CBOR, if someone defines a mapping for it), if you want to pursue this road. All of these options are extensible, and _provide validation for ???free"_, and _well-defined schema to stick in your standard_.
>

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 30 May 2015 08:20:05 +1200

On 30/05/2015 06:53, Markus Stenberg wrote:
> The tone on the list sounds so friendly and all-knowing that this is my last comment,

Markus, you were dropped into the middle of a fairly hot discussion so
I'm sorry about that.

Guys, let's stay polite...

> feel free to drop me from the comments. I did the original comments on my free time as a favor and clearly it was not worth the trouble, and you clearly also have all you need on the list already.

I'm not sure we do, quite. However, I do have to agree that XML is not the future.
That view is strongly influenced by having worked for IBM at the time when WSDL
was in development, and seeing up close and personal the horrific data on
memory and path length for parsing it. ASN.1 is probably better but really doesn't
help people brought upon object-oriented programming.

I absolutely expect some autonomic networking use cases to involve YANG
and I hope we are able to carry *any* format if we need to (that is clearly
supported in the existing binary TLV version of GDNP: the V can be anything).

Regardless, thanks for your input; we have to marshall the arguments, and
the final choice will be made by the WG not the design team.

Best,

    Brian