Modeling JSON Messages Using YANG

JavaScript Object Notation (JSON) has been a popular choice as the message encoding for many network protocols such as the Application-Layer Traffic Optimization (ALTO) protocol , the Content Delivery Networks Interconnection (CDNi) protocol , the Port Control Protocol (PCP) , etc. Meanwhile, there are broad interests in the networking community to use the YANG data modeling language to define data store and protocol messages, so that one can use YANG related tools such as the OpenDayLight Controller. Although YANG itself is XML based, there have been efforts to model JSON content using YANG This document explores the conditions under which the messages of a JSON based protocol can have a syntactically equivalent hence interoperable YANG model, and provides a conversion process from the JSON message to the YANG model. In particular, this document shows that any JSON protocol message with stand-alone non-object JSON values, certain JSON arrays of elements of mixed types, or non-keyword keys in key-value pairs cannot have a syntactically equivalent YANG model. It also applies these conditions to the ALTO and CDNi protocol messages as examples. For protocols that do not have syntactically equivalent YANG models, we also explore the possibility of a semantically equivalent YANG model.

Assume that we follow the specifications of JSON as specified in , the YANG modeling languate as specified in . and the JSON encoding of data modeled in YANG as specified in . A JSON based protocol message can have a syntactically equivalent YANG model if and only if: (1) the message encoding condition is met; (2) the uri condition is met.

The JSON message encoding condition is the following: The message MUST be a JSON object. The arrays MUST be either all objects, or all non-object non-array types, i.e. false, null, true, number, string; The JSON message MUST NOT contain a variable as a key or null as a value in a JSON object key-value pair; in other words, the keys in the JSON message must be an already defined constant keyword in the message format of the protocol.

Some of the keys (e.g. "PID1", "PID2", "PID3") in this JSON object are not pre-defined keywords, which violated Condition (3). Hence it cannot have a syntactically equivalent YANG Model.

This JSON object, as specified in Section 6.4.2 in , satisfies all three message encoding conditions. Hence it is possible to model it using YANG.

Some of the YANG related protocols might have URI constraints, e.g., RESTCONF. There might be also be future protocols that put on additional restraints. In addition, it is a restraint on the data (the uri in the JSON message), instead of on the data model. In case these constraints are in place, to resolve the issue, the uri in the JSON message could be conformed to constraint-compliant uri; alternatively, the server could set up a gateway translation for uri. Hence, for now we focus only on the message encoding condition and leave this condition open.

We prove the necessity of the message encoding condition by proving its contrapositive.

Although there are no strict rules to convert from JSON to XML, a stand-alone non-object JSON value would not have an XML tag, which fails because of the same reason for Condition (3).

YANG cannot model stand alone arrays as non of the YANG statements maps to an array according to . There are two statements which generate a name/array pair in YANG, the leaf-list statement and the list statement. For a leaf-list, the element type must be a YANG datatype (Section. 6 of ); for a list, the array elements must be JSON objects. Hence, there cannot be a mix.

If one of the keys in the key-value pair in the JSON document is not pre-defined, the corresponding XML tags will not be pre-defined keywords. Therefore, it would not possible to model it in YANG without using YANG’s anyxml statement (which allows arbitrary XML content). However, using the anyxml statement defeats the purpose of modeling the data as it allows arbitrary XML content, and will not help the subsequent parsing process.

The above three conditions are necessary to model a JSON message in YANG.

We prove this by providing a translation procedure from a JSON message that is compliant with the protocol we are trying to model, to a custom java class that can be used for Jackson data binding, then to a YANG model. Note that the middle step of translating to and from the custom parser class is not necessary, but it is useful. This also implies which data binding/JSON parser we use does not matter.

JSON data binding is the process of binding data structures (objects, arrays, etc.) in JSON to the appropriate data structures in the server (e.g. java classes, database tables, etc.) in parsing the JSON text. In order to process JSON messages in a meaningful manner, data binding is necessary. Even if the binding is not explicit, the server would need to do it eventually. For example, one can read JSON content in a stream without binding it to the java classes, but eventually in order to make sense of the data, the server would eventually have to organize it, which is analogous to data binding upfront. Popular choices for JSON parsing and data binding include jackson and gson. We use Jackson full data binding as our approach. Full data binding binds JSON content into plain old java objects (POJOs), i.e. this custom parser class can neither extend nor implement any other class. Jackson uses ObjectMapper with the custom parser class to parse JSON content into this class. The no inheritance assumption means that all custom parser classes are base classes. Nevertheless, this assumption is also not necessary--we can simply make any derived classes a new based class.

For now, we make a stronger assumption than Condition (2): we assume all arrays contain elements that are homogeneous non-array JSON values. We handle the case of heterogeneous objects and heterogeneous non-object non-array types later. The message encoding condition (3) is that all keys in each key-value pair in the JSON text must be pre-defined keywords. As the keys will become either class names and instance variable names, or be keys in the java maps, it is easy to see that this condition is equivalent to: there exists a full data binding in Jackson custom parser class without using any map structures (Map<String, ?>).”

We provide a recursive binding process from a JSON object to the Jackson custom parser class to be used by Jackson ObjectMapper.

where T=determine_type(value[0]); } // should not reach here. } Class build_parser_class(JSONObject obj) { create custom class C; for each key/value pair in the obj { add instance variable v in C; the name of variable v <- key; the type of variable v <- determine_type(value); } return C.class; } ]]> To ensure the naming in the YANG model is consistent with the JSON message, We follow the naming process: change everything into CamelCase (i.e. remove dashes, etc.); for instance variables, use “my” prefix, (e.g. myVariable, myNetworkMap, etc.); for the custom class name, if the object is an element of the array, use “Element” suffix; if a class already exists, we add a number after it which we then remove at the next stage.

Given a Jackson Parser Java Class, the following algorithm generates a YANG model:

; }” } if (Type is an ArrayList) { if (TypeElement is primitive type) { add the following to the YANG module: “leaf-list Name { type ; }” } else { // TypeElement is a custom parser class add the following to the YANG module: “list Name { build_yang_model(TypeElement.class) }” } } if (Type is a custom parser class) { add the following to the YANG module: “container Name { build_yang_model(Type.class) }” } } } ]]>

Due to Java's strong typing, handling heterogeneous arrays is difficult, but not impossible. For simplicity, we provide the YANG model directly for heterogeneous arrays and provide guidelines for Jackson parsing.

If the array is consisted of entirely non-object, non-value types type_1, type_2, ..., type_n, the corresponding YANG model is a leaf-list with a derived type.

In Jackson, one may parse such arrays as String[] and then convert to whichever types needed.

If the array is consisted of entirely JSON object, we create a YANG grouping for each object (by following the same process as build_parser_class and build_yang_object above). Then use list, choice, and case statements.

In Jackson, one may use annotations or custom deserializers to parse such structures.

Take the cost map response from the ALTO protocol as an example:

This JSON object does not have a syntactically equivalent model in YANG because some of its keys are variables, e.g., the "PID1", "PID2", etc. If we change this cost map object to the following, we will be able to model it.

Result of build_parser_class(obj):

myCostMap; } Class Meta { ArrayList myDependentVtags; CostType myCostType; } Class DependentVtagsElement { String myResourceId; String myTag; } Class CostType { String myCostMode; String myCostMetric; } Class CostMapElement { String mySrc; ArrayList myDstCosts; } Class DstCostsElement { String myDst; int myCost; } ]]> Generated YANG model:

This YANG model does validate the JSON cost map object with JSON encoding defined in .

The original JSON message is the following:

Result of build_parser_class(obj):

myMetaData; ArrayList myPaths; } Class MetaDataElement { String myGenericMetadataType; GenericMetadataValue myGenericMetadataValue; } Class GenericMetadataValue { ArrayList mySources; ArrayList myLocations; ArrayList myProtocols; } Class SourcesElement { _Links my_Links; String myEndpoint; String myProtocol; } Class _Links { Auth myAuth; } Class Auth { String myAuthType; String myHref; } Class LocationsElement { ArrayList myLocations; String myAction; } Class LocationsElement2 { String myFootprintType; String myFootprintValue; } Class ProtocolsElement { ArrayList myProtocols; String myAction; } Class PathsElement { PathPattern myPathPattern; _Links2 my_Links; } Class PathPattern { String myPattern; } Class _Links2 { PathMetadata myPathMetadata; } Class PathMetadata { String myType; String myHref; } ]]> Generated YANG model:

This YANG model does validate the JSON cost map object with JSON encoding defined in .

This process proves that the message encoding condition is a sufficient condition for the JSON object to have a YANG model. Note the model generated is very crude and lose almost all constraints and all inheritance features (if any), because it focuses on the syntax and is essentially converted from an JSON object compliant with a protocol instead of from the protocol itself. Hence this result is more useful in determining which JSON based protocols cannot have a syntactically equivalent YANG model, than in generating a usable YANG model.

For JSON based protocols that don’t satisfy the message encoding condition, it is still possible to have a semantically equivalent YANG model. All that is required for the protocol compliant clients and the YANG model compliant server to interoperate is an adapter which does the following: 1) translate FROM YANG server compliant response msg TO alto compliant response msg; 2) translate FROM alto compliant request msg TO YANG server compliant request msg;

This adapter needs to be protocol-aware. Ideally, given any YANG model, we would like to be able to automatically (or at least mechanically) generate this message adapter, which means not looking at the protocol or its compliant msgs. However, without knowing the specific protocol that we are working with (i.e. human intervention, i.e. looking at the protocol compliant msgs), such an adapter cannot be auto-generated.

Suppose both the YANG server compliant msg m_y and the actually protocol compliant msg m_p are in JSON (or have been encoded into JSON). Looking at the differences between the two messages, call these differences {d1, d2, ..., dn}. The goal for the auto-generated adapter would be to identify and eliminate these differences. Construct a new JSON msg m' where all but one difference di is the same as m_p and di is the same as the m_y. Without looking at the protocol (or m_p), the auto-generated adapter would not be able to distinguish between m' and m_p in its translation process, which means, it won't be able to tell whether it should change di or not. Hence, such an adapter must be protocol-aware. A good example is the dependent-vtag in the ALTO protocol:

It was specified this way in the alto protocol. However, it could conceivably be the case that it was originally the following map structure, and was converted into the above encoding because of the map->list+key issue. (This case is actually one of the few differences in the m_y and m_p where the adapter does not need to convert it back to a map structure.)

Without knowing the protocol or protocol messages, it is impossible to distinguish.

We now understand the basic condition for a JSON based protocol to have a YANG Model. For the protocols that don’t meet this condition, there can be a semantic equivalent YANG model, but there won’t be a generic process of generating the adapter for all protocols.

This document does not introduce security or privacy concerns.

This document does not have IANA considerations.