Restconf subscription and HTTP push for YANG datastores

Requirements for subscriptions to YANG datastores are defined in . Mechanisms to support YANG subscriptions and datastore object push over a NETCONF are defined in . Restconf support of subscriptions, with HTTP transport of pushed updates is also needed by the market. This document provides such a specification. Key benefits of pushing data via HTTP include: Ability to configure static subscriptions on a Publisher Ability for the Publisher to initiate communications with the Receiver Ability of a Subscriber to be different from the Receiver There are also additional benefits which can be realized when pushing updates via HTTP/2 : Subscription multiplexing over independent HTTP/2 streams Stream prioritization Stream dependencies Flow control on independent streams Header compression These additional benefits will address issues resulting from head-of-line blocking and relative subscription priority. To maximize transport independence of YANG subscription methods, this document reuses many capabilities of [] including: Operations for creating, modifying and deleting subscriptions Syntax and parameters for negotiating the subscription YANG data model to manage subscriptions Mechanisms to communicate subscription filters and data streams

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. Datastore: a conceptual store of instantiated management information, with individual data items represented by data nodes which are arranged in hierarchical manner. Dynamic YANG Subscription: Subscription negotiated with Publisher via create, modify, and delete control plane signaling messages. Publisher: an entity responsible for distributing subscribed YANG object data per the terms of a Subscription. In general, a Publisher is the owner of the YANG datastore that is subjected to the Subscription. Receiver: the target where a Publisher pushes updates. In many deployments, the Receiver and Subscriber will be the same entity. Static YANG Subscription: A Subscription installed via a configuration interface. Subscriber: An entity able to request and negotiate a contract for push updates from a Publisher. Subscription: A contract between a Subscriber and a Publisher, stipulating which information the Receiver wishes to have pushed from the Publisher without the need for further solicitation. Subscription Update: Set of data nodes and object values pushed together as a unit and intended to meet the obligations of a single subscription at a snapshot in time.

This document specifies mechanisms that allow subscribed information updates to be pushed from a YANG datastore. Subscriptions may either be initiated via requests by Subscribers, or statically configured on a Publisher. As in , Publisher must respond to a subscription request explicitly positively or negatively. Negative responses will include information about why the Subscription was not accepted, in order to facilitate converging on an acceptable set of Subscription parameters. Once a Subscription has been established, updates are pushed to the Receiver until the Subscription terminates. Based on parameters within the Subscription, these updates can be streamed immediately as any subscribed objects change, or sent periodically.

Subscriptions use the base data model from . This model is extended with several optional parameters for Subscription Priority and Subscription Dependency. These parameters allow a Subscriber or other configuration interface to assert how it prefers the Publisher allocate resources when handling multiple Subscriptions. These parameters are intended to be used in conjunction with the transport layer. Specifically, when a new Subscription is being established with an underlying transport is HTTP/2, these parameters may be directly mapped into HTTP/2 to prioritize transport and to assist with flow control of individual streams.

Below is the state machine for the Publisher. It is important to note that a Subscription doesn’t exist at the Publisher until it is accepted and made active. The assertion of a <create-subscription> by a Subscriber is insufficient for that asserted subscription to be externally visible via this state machine.

Subscription states at Publisher suspend------->| | modify '| active | | suspended | '--------->| |<----reactivate<------| | '-----------' '-----------' | | delete delete | | v | .-------. | | end |<-----------------------------' '-------']]> Of interest in this state machine are the following: Successful <create-subscription> or <modify-subscription> actions must put the subscription into an active state. Failed <modify-subscription> actions will leave the subscription in its previous state, with no visible change to any streaming updates. A <delete-subscription> action will delete the entire subscription.

On a Publisher, it must be possible to instantiate a Subscription via dynamic Subscriber signaling, as well as via Static configuration. Dynamic YANG Subscriptions are signaled Subscriptions aimed at the running datastore and are unable to impact the startup configuration. They should always terminate when there is loss of transport session connectivity between the Publisher and Receiver. Static Subscriptions are applied via an operations interface to the startup and running configurations. Loss or non-availability of transport session connectivity will place the Subscription into the suspended state. Logic beyond the scope of this specification will dictate when any particular Subscription should be reactivated. There are three models for Subscription establishment and maintenance: Dynamic YANG Subscription: Subscriber and Receiver are the same Static YANG Subscription Dynamic YANG Subscription: Subscriber and Receiver are different The first two are described in this section. The third is described in . This third option can be moved into the body of this specification should the IETF community desire. In theory, all three models may be intermixed in a single deployment. Figure 2 shows such a scenario.

With all Dynamic YANG Subscriptions, as with it must be possible to configure and manage Subscriptions via signaling. This signaling is transported over . Once established, streaming Subscription Updates are then delivered via Restconf SSE.

With a Static YANG Subscription, all information needed to establish a secure object push relationship with that Receiver must be configured via a configuration interface on the Publisher. This information includes all the <create-subscription> information identified in section 3.3.1. This information also includes the Receiver address, encoding selection, and any security credentials required to establish TLS between the Publisher and Receiver. Mechanisms for locally configuring these parameters are outside the scope of this document. With this information, the Publisher will establish a secure transport connection with the Receiver and then begin pushing the streaming updates to the Receiver. Since Restconf might not exist on the Receiver, it is not desirable to require that updates be pushed via Restconf. In place of Restconf, a TLS secured HTTP Client connection must be established with an HTTP Server located on the Receiver. Subscription Updates will then be sent via HTTP Post messages to the Receiver. Post messages will be addressed to HTTP augmentation code on the Receiver capable accepting and responding to Subscription Updates. At least the initial Post message must include the URI for the subscribed resource. This URI can be retained for future use by the Receiver. After successful receipt of an initial Subscription Update for a particular Subscription, this augmentation should reply back with an HTTP status code of 201 (Created). Further successful receipts should result in the return of code of 202 (Accepted). At any point, receipt of any status codes from 300-510 with the exception of 408 (Request Timeout) should result in the movement of the Subscription to the suspended state. A sequential series of multiple 408 exceptions should also drive the Subscription to a suspended state. Security on an HTTP client/Publisher can be strengthened by only accepting Response code feedback for recently initiated HTTP POSTs. Figure 3 depicts this message flow.

| | | |HTTP POST (Sub ID, URI, data1) | |------------------------------>| | HTTP 201 (Created)| |<------------------------------| |HTTP POST (Sub ID, data2) | |------------------------------>| | HTTP 200 or 202 (Accepted)| |<------------------------------| | data3 | |<----------------------------->| ]]> If HTTP/2 transport is available to a Receiver, the Publisher should also: point individual Subscription Updates to a unique HTTP/2 stream for that Subscription, take any subscription-priority and provision it into the HTTP/2 stream priority, and take any subscription-dependency and provision it into the HTTP/2 stream dependency.

When using signaling to create a Dynamic YANG Subscription, negotiable parameters will include the same negotiable parameters defined within . Additionally, negotiation may also include Subscription Priority. A Publisher may accept a Subscriber asserted Priority, as well as rejecting a subscription with a hint at what priority might be accepted.

Implementations must support periodic and/or on-change subscriptions as defined in .

Implementations must support filters and streams as defined in .

Same authorization model for data as [netconf-yang-push] will be used. This includes functions of the Netconf Access Control Model applied to objects to be pushed via Restconf. A Subscription (including a Static YANG Subscription) may only be established if the Subscriber or some entity statically configuring via the Publisher’s operational interface has read access to the target data node.

When pushed directly over HTTP/2, it is expected that each Subscription Update will be allocated a separate Stream. The will enable multiplexing, and address issues of Head-of-line blocking with different priority Subscriptions. When pushed via Restconf over HTTP/2, different Subscriptions will not be mapped to independent HTTP/2 streams. When Restconf specifies this mapping, it should be integrated into this specification. Even without HTTP/2 multiplexing, it is possible that updates might be delivered in a different sequence than generated. Reasons for this might include (but are not limited to): different durations needed to create various Subscription Updates, marshalling and bundling of multiple Subscription Updates for transport, and parallel HTTP1.1 sessions Therefore each Subscription Update will include a microsecond level timestamp to ensure that a receiver understands the time when a that update was generated. Use of this timestamp can give an indication of the state of objects at a Publisher when state-entangled information is received across different subscriptions. The use of the latest Subscription Update timestamp for a particular object update can introduce errors. So when state-entangled updates have inconsistent object values and temporally close timestamps, a Receiver might consider performing a ‘get’ to validate the current state of objects.

Transported updates will contain data for one or more Subscription Updates. Each transported Subscription Update notification contains several parameters: A global subscription ID correlator, referencing the name of the Subscription on whose behalf the notification is sent. Data nodes containing a representation of the datastore subtree containing the updates. The set of data nodes must be filtered per access control rules to contain only data that the subscriber is authorized to see. An event time which contains the time stamp at publisher when the event is generated.

Subscribers can dynamically learn whether a RESTCONF server supports yang-push. This is done by issuing an HTTP request OPTIONS, HEAD, or GET on the stream push-update. E.g.:

If the server supports it, it may respond

yang-push Yang push stream xml https://example.com/streams/yang-push-xml json https://example.com/streams/yang-push-json ]]> If the server does not support yang push, it may respond

Subscribers can determine the URL to receive updates by sending an HTTP GET request for the "location" leaf with the stream list entry. The stream to use for yang push is the push-update stream. The location returned by the publisher can be used for the actual notification subscription. Note that different encodings are supporting using different locations. For example, he subscriber might send the following request:

The publisher might send the following response:

https://example.com/streams/yang-push-xml ]]> To subscribe and start receiving updates, the subscriber can then send an HTTP GET request for the URL returned by the publisher in the request above. The accept header must be "text/event -stream". The publisher handles the connection as an event stream, using the Server Sent Events transport strategy. The publisher MUST support as query parameters for a GET method on this resource all the parameters of a subscription. The only exception is the encoding, which is embedded in the URI. An example of this is:

Should the publisher not support the requested subscription, it may reply:

application

operation-not-supported

error

Xpath filters not supported

with an equivalent JSON encoding representation of: HTTP/1.1 501 Not Implemented Date: Mon, 23 Apr 2012 17:11:00 GMT Server: example-server Content-Type: application/yang.errors+json { "ietf-restconf:errors": { "error": { "error-type": "protocol", "error-tag": "operation-not-supported", "error-message": "Xpath filters not supported." "error-info": { "datastore-push:supported-subscription": { "subtree-filter": [null] } } } } } ]]> The following is an example of a push Subscription Update data for the subscription above. It contains a subtree with root foo that contains a leaf called bar:

my-sub

2015-03-09T19:14:56Z

some_string

]]> Or with the equivalent YANG over JSON encoding representation as defined in :

To modify a subscription, the subscriber issues another GET request on the provided URI using the same subscription-id as in the original request. For example, to modify the update period to 10 seconds, the subscriber may send:

To delete a subscription, the subscriber issues a DELETE request on the provided URI using the same subscription-id as in the original request

For any version of HTTP, the basic encoding will look as below is the above JSON representation wrapped in an HTTP header. Mechanism will be

Below is the object tree for the model. All items are imported from [netconf-yang-push] except for the addition of “subscription-priority” and “subscription-dependency”.

WG List: WG Chair: Mahesh Jethanandani WG Chair: Mehmet Ersue Editor: Eric Voit Editor: Alexander Clemm Editor: Ambika Prasad Tripathy Editor: Einar Nilsen-Nygaard Editor: Alberto Gonzalez Prieto "; description "This module contains conceptual YANG specifications for Restconf datastore push."; revision 2015-10-01 { description "Initial revision."; reference "restconf YANG Datastore push"; } grouping subscription-qos { description "This grouping describes Quality of Service information concerning a subscription. This information is passed to lower layers for transport priortization and treatment"; leaf subscription-priority { type uint8; description "Relative priority for a subscription. Allows an underlying transport layer perform informed load balance allocations between various subscriptions"; } leaf subscription-dependency { type string; description "Provides the Subscription ID of a parent subscription without which this subscription should not exist. In other words, there is no reason to stream these objects if another subscription is missing."; } } augment "/ds-push:subscription-config/" + "ds-push:datastore-push-subscription" { description "Aguments configured subscriptions with QoS parameters."; uses subscription-qos; } augment "/ds-push:subscriptions/" + "ds-push:datastore-push-subscription" { description "Aguments the list of currently active subscriptions with QoS parameters."; uses subscription-qos; } augment "/ds-push:create-subscription/" + "ds-push:input" { description "Aguments the create subscription rpc with QoS parameters."; uses subscription-qos; } augment "/ds-push:modify-subscription/" + "ds-push:input" { description "Aguments the modify subscription rpc with QoS parameters."; uses subscription-qos; } } ]]>

Subscriptions could be used to intentionally or accidentally overload resources of a Publisher. For this reason, it is important that the Publisher has the ability to prioritize the establishment and push of updates where there might be resource exhaust potential. In addition, a server needs to be able to suspend existing subscriptions when needed. When this occurs, the subscription status must be updated accordingly and the clients are notified. A Subscription could be used to retrieve data in subtrees that a client has not authorized access to. Therefore it is important that data pushed via a Subscription is authorized equivalently with regular data retrieval operations. Data being pushed to a client needs therefore to be filtered accordingly, just like if the data were being retrieved on-demand. The Netconf Authorization Control Model applies. One or more Publishers could be used to overwhelm a Receiver which doesn’t even support subscriptions. Therefore Updates MUST only be transmittable over Encrypted transports. Clients which do not want pushed data need only terminate or refuse any transport sessions from the Publisher. One or more Publishers could overwhelm a Receiver which is unable to control or handle the volume of Updates received. In deployments where this might be a concern, transports supporting per-subscription Flow Control and Prioritization (such as HTTP/2) should be selected. Another benefit is that a well-behaved Publisher implementation is that it is difficult to a Publisher to perform a DoS attack on a Receiver. DoS attack protection comes from: the requirement for trust of a TLS session before publication, the need for an HTTP transport augmentation on the Receiver, and that the Publication process is suspended when the Receiver doesn’t respond.

Subscribing to YANG datastore push updates Cisco Cisco



          
            Cisco



      
        
          Subscribing to datastore push updates

          
            Cisco

            
              
                

                

                

                

                
              

              

              

              

              
            

          

          
            Cisco
          

          
            Cisco

            
              
                

                

                

                

                
              

              

              

              

              
            

          

          
        
      

      
        
          RESTCONF Protocol

          

          

          

          
        
      

      
        
          JSON Encoding of Data Modeled with YANG

          
            
          

          
        
      

      
        
          Server-Sent Events, World Wide Web Consortium CR
          CR-eventsource-20121211

          
            
          

          
        
      

      
        
          NETCONF Call Home and RESTCONF Call Home



    
      The methods of Sections 3.3.1 and 3.3.2 can be combined to enable
      deployment models where the Subscriber and Receiver are different. Such
      separation can be useful with some combination of:

      
          An operator wants any Subscriptions immediately deleted should
          TLS connectivity be lost. (I.e., Subscriptions don’t default
          into a ‘Suspended’ state on the Publisher.)

          An operator wants the Publisher to include highly restrictive
          capacity management and security mechanisms outside of domain of
          existing operational or programmatic interfaces.

          Restconf is not desired on the Receiver.

          The Publisher doesn’t want to maintain Restconf
          subscriptions with many Receivers.
        To do this, first the necessary information must be signaled as
      part of the <create-subscription>. This includes all the
      information described in section 3.3.2, with the exception of the
      security credentials. (It is assumed that any security credentials
      required for establishing any transport connections are pre-provisioned
      on all devices.)

      Using this set of Subscriber provided information, the same process
      described within section 3.3.2 will be followed. There is one exception.
      When an HTTP status code is 201 is received by the Publisher, it will
      inform the Subscriber of Subscription establishment success via its
      Restconf connection.

      After successful establishment, if the Subscriber wishes to maintain
      the state of Receiver subscriptions, it can simply place a separate
      on-change Subscription into the “Subscriptions” subtree of
      the YANG Datastore on the Publisher.

      Putting it all together, the message flow is:

      
        |                            |
        |--------------------->|                            | 
        |                      |                            |
        |                      |<-----------TLS------------>| 
        |                      |                            |
        |                      |HTTP POST (Sub ID, data1,   |
        |                      |(stream ID, URI?))          | 
        |                      |--------------------------->|
        |                      |          HTTP 201 (Created)|
        |                      |<---------------------------| 
        |     Success: HTTP 204|                            |
        |<---------------------|                            |
        |                      |HTTP POST (Sub ID, data2)   | 
        |                      |--------------------------->| 
        |                      |  HTTP 200 or 202 (Accepted)| 
        |                      |<---------------------------| 
        |                      |            data3           | 
        |                      |<-------------------------->| 
        |                      |                            |
]]>
      

      
    

    
      Several technologies are expected to be seen within a deployment to
      achieve security and ease-of-use requirements. These are not necessary
      for an implementation of this specification, but will be useful to
      consider when considering the operational context.

      

      
        Pub/Sub implementations should have the ability to transparently
        incorporate lower layer technologies such as Call Home so that secure
        TLS connections are always originated from the Publisher. There is a
        Restconf Call home function in . For
        security reasons, this should be implemented as desired.

        
      

      
        Unlike NETCONF, HTTP sessions might not quickly allow a Subscriber
        to recognize when the communication path has been lost from the
        Publisher. To recognize this, it is possible for a Receiver (usually
        the subscriber) to establish a TLS heartbeat .
        In the case where a TLS heartbeat is included, it should be sent just
        from Receiver to Publisher. Loss of the heartbeat should result in the
        Subscription being terminated with the Subscriber (even when the
        Subscriber and Receiver are different). The Subscriber can then
        attempt to re-establish the subscription if desired. If the
        Subscription remains active on the Publisher, future receipt of
        objects associated with that (or any other unknown) subscription ID
        should result in a <delete-subscription> being returned to the
        Publisher from the Receiver.

        
      

      
        If Subscriber and receiver are same entity then subscriber can
        direct send create_subscription message to publisher. Once the
        subscription moved to accepted state, the receiver can use Server Sent
        Events  transport strategy to subscriber
        event notifications for the data as defined in.