Smart Filters for Push UpdatesHuawei - Futurewei Technologies, Inc.2330 Central ExpresswaySanta Clara,USACA 95050ludwig@clemm.orgCisco Systemsevoit@cisco.comVolta Networksxufeng.liu.ietf@gmail.comHuaweiigor.bryskin@huawei.comHuaweizhoutianran@huawei.comHuaweizhengguangying@huawei.comFraunhofer SIThenk.birkholz@sit.fraunhofer.deThis document defines a YANG model for Smart Filters for push
updates. Smart Filters allow to filter push updates based on values of
pushed datastore nodes and/or state, such as previous updates. Smart
Filters provide an important building block for service assurance and
network automation.This revision of the document is intended as a placeholder,
containing the problem statement of
draft-clemm-netconf-push-smart-filters-ps-00 that has recently expired.
The YANG model itself still needs to be defined.YANG-Push allows client applications to
subscribe to continuous datastore updates without needing to poll.
YANG-Push subscriptions allow client applications to select which
datastore nodes are of interest. For this purpose, filters that act as
node selectors are offered. However, what is currently not supported are
filters that filter updates based on values, such as sending updates
only when the value falls within a certain range. Also not supported are
filters that would require additional state, such as sending updates
only when the value exceeds a certain threshold for the first time but
not again until the threshold is cleared. We refer to such filters as
"Smart Filters", with further subcategories of "smart stateless filters"
and "smart stateful filters", respectively.Smart Filters involve more complex subscription and implementation
semantics than the simple selection filters that are currently offered
as part of YANG-Push. They involve post processing of updates that goes
beyond basic update generation for polling avoidance and place
additional intelligence at the server. Because of this, Smart Filter
functionality was not included in the YANG-Push specification, although
it was recognized that YANG-Push could be extended to include such
functionality if needed. This is the purpose of this specification.Smart Filters facilitate service assurance, because they allow client
applications to focus on "outliers" and updates that signify exceptions
and conditions of interest have the biggest operational significance.
They save network resources by avoiding the need to stream updates that
would be discarded anyway, and allow applications to scale better since
larger networks imply a larger amount of Smart Filtering operations
delegated away from the application to the network. Smart Filters also
facilitate network automation as they constitute an important ingredient
to specify triggers for automated actions.The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP 14
when, and only when,
they appear in all capitals, as shown here.Datastore node: An instance of management information in a
datastore. Also known as "object".Smart Filter: A filter that involves some processing, such as
comparing values or differentiating behavior depending on state.TCA: Threshold Crossing Alert.YANG-Push: A server capability that allows client applications to
subscribe to network management datastore updates.YANG-Push provides client applications with the ability to subscribe
to continuous updates from network management datastores, obviating the
need to perform polling and resulting in more robust and efficient
applications. However, many applications do not require every update,
only updates that are of certain interest.For example, an update concerning interface utilization may be only
needed when a certain utilization level is breached. Sending continuous
updates when utilization is low might divert processing resources away
from updates regarding interfaces whose utilization level may reach a
critical point that requires attention. Doing so will require a filter
based on an object value. Even sending continuous updates when
utilization is high may be too much and counterproductive. It may be
sufficient to send an update when a threshold is breached to raise a
flag of attention, but then not to continue sending updates while the
condition still persists but simply let the client application know when
the threshold is cleared. This behavior cannot be accomplished simply by
a value-based filter, but requires additional state to be maintained (so
that the server has a memory whether or not the condition of a breached
threshold has already been reported in prior update cycles).What is needed are "Smart Filters" that provide the ability to apply
filters based on object values, possibly also state state. Smart Filters
are useful for Service Assurance applications that need to monitor
operational data for values that fall outside normal operational ranges.
They are also useful for network automation, in which automated actions
are automatically triggered based on when certain events in the network
occur while certain conditions hold. A YANG-Push subscription with a
Smart Filter can in effect act as a source for such events. Combined
with an optional check for a condition when an event is observed, this
can serve as the basis of action triggers.Smart Filters for Push Updates will provide support for the following
features: Support for Smart Filter extensions to YANG-Push subscriptions.
The targeted model takes a "base" YANG-Push subscription and
subjects updates to an additional filtering stage that is based on
value.Support for selected stateful filters: This includes specifically support for generalized "threshold
crossing alert" filters, or filters that provide an update only
when a datastore node's value passes a filter for the first
time, and not again until the datastore node's value passes a
counter filter. In effect, the support involves attaching filter
and counter filter to a datastore node, including a switch at
the datastore node indicating which filter is in effect, and
providing a distinction in the update which filter (e.g. onset
of clear) was applied.It may include additional filters, such a "recent high water
mark" filters that allow to specify a time horizon until the
current high water mark clears. A recent high water mark filter
sends an update to an object only if its new value is greater
than the last value that had been previously reported.In addition to new filters, support for features to make them
easier to use: Support for refined on-change update semantics that allow
client to distinguish whether datastore node values were omitted
or included because the datastore node was created or deleted,
or because the datastore node's value fell outside filter
range.Support for a heartbeat that indicates that a filter is still
in effect after a longer period of inactivity.It is easy to conceive of filters that are very smart and powerful
yet also very complex. While filters as defined in YANG-Push may be a
tad too simple for the applications envisioned here, it is important to
keep filters still simple enough to ensure broad implementation and
support by networking devices. The purpose of Smart Filters defined in
this effort is to address the 90% of cases that can be addressed using
10% of the complexity. Items like the following will therefore be
outside the scope: Filters that involve freely programmable logic.Filters that aggregate or otherwise process information over
time. An example would be filters that compute an aggregate over a
time series of data (e.g. a datastore node's average or top
percentile value)Filters that aggregate or compare values of several datastore
nodes (e.g. the maximum or average from datastore nodes in a
list). The following section contains an initial YANG data model for smart filters. The model is at this point still incomplete and included as a starting point only. At this point, the model defines a simple threshold filter. When used with a subscription, objects that meet the filter criterion (i.e. the threshold comparison) are included in the update whereas any other object is filtered.
The model will be extended to define a full "smart threshold" model in a later revision. This will add the feature of a hysteresis threshold, i.e. a counter threshold that allows to define when a crossed threshold should be cleared. The value of the hysteresis threshold can be set to a lower value than the threshold itself to avoid unnecessary updates in case of oscillations). It will also add a notion of state to remember whether a threshold crossing has already been reported, to avoid repeated inclusion of objects in updates that remain above their threshold. By including metadata, clients will be able to distinguish between the violation and the clearing of thresholds.
The model will furthermore be extended for smart filters that are not threshold-related, such as the previously mentioned recent high water marks.
RFC Ed.: In this section, replace all occurrences of 'XXXX' with the
actual RFC number (and remove this note).IANA is requested to assign a new URI from the IETF XML Registry. The following URI is
suggested:This document also requests a new YANG module name in the YANG Module Names registry with the following
suggestion:The application of Smart Filters requires a certain amount of
processing resources at the server. An attacker could attempt to attack
a server by creating YANG-push subscriptions with a large number of
complex Smart Filters in an attempt to diminish server resources. Server
implementations can guard against such scenarios in several ways. For
one, they can implement NACM in order to require proper authorization
for requests to be made. Second, server implementations can reject
requests made for a a larger number of Smart Filters than the
implementation can reasonably sustain.Subscribing to YANG datastore push updatesCustom Subscriptions to a Publisher's Event Streams