Hi Yansen et al.

Here is my take on the anomaly detection use case (2.6). I know that I was
supposed to write just 'rough thoughts and requirements', but I decided to
write in advance regarding the upcoming tasks (sep/oct).

======================================================================

Definitions:

First, it is important to understand that the scientific literature allows
different names for anomaly detection, such as outlier detection, novelty
detection, noise detection, deviation detection, or mining exception. This
not so strict naming also brings a number of definitions for what could be
considered as an outlier or anomaly. A more general definition for an
outlier is an observation that is somewhat inconsistent when compared to
the remainder of the set of observations. In the specific case of a network
anomaly, such observation may cause network failures and performance
problems such as congestions and denial of services (DOS).

Anomalies can be generally classified into three categories, namely point,
contextual, and collective. Point anomaly is related to a single
observation whereas collective anomaly is related to a series of
observations. For instance, a SYN Flood attack can be considered a
collective anomaly since a single TCP SYN segment is valid and is not
considered a point anomaly. Contextual anomaly is the interpretation
whether the point or collective anomaly is, in fact, an anomaly given a
proper context.

Applications:

Abnormal behavior from packets or streams of packets (flows) requires
precise and, in some cases, quick detection so that the network could react
and take appropriate measures to mitigate its short and long term effects.
Ideally, the network must be intelligent enough to automatically learn what
is normal traffic so it could adapt to any abnormal traffic patterns,
including zero-day attacks. Of course, not all anomalies come from
intentional attacks. Abnormal behavior could also come from
misconfiguration or malfunctions in the network.

Data / Features:

Most techniques for anomaly detection use features extracted from transport
or network layer data, mainly due to the widespread adoption of
IPFIX/NetFlow on routers/switches. It is possible to create new variables
(a.k.a. Feature Engineering in the data mining/machine learning lingo) from
the raw data, such as packet or flow inter-arrival times. Depending on the
measurement process one could also include data from other Internet layers
to make the detection more accurate. One must be aware that scalability is
always a concern when dealing with massive amount of data.

The types and characteristics of the input data often limit the choices of
techniques that could be used for anomaly detection. Some techniques
require labeled data (i.e., using prior knowledge to identify an
observation as normal or abnormal), which in most cases requires enormous
processing efforts.

Techniques:

Methods for anomaly detection come from several fields and their
subdisciplines, such as Statistics, Machine Learning, Data Mining,
Information Theory, Spectral Theory, and the like. Therefore, there are a
number of techniques to handle (i.e., detection and/or removal) abnormal
observations. As the main scope and interest of IDNET are on techniques
that can learn from the incoming network traffic and events, the
behavioral-based anomaly detection ones seem the best fit.

Behavioral-based anomaly detection methods are usually classified as
supervised, semi-supervised, or unsupervised, depending on the availability
of labeled data for the training phase. Supervised and semi-supervised
learning require labeled data whereas unsupervised learning is able to work
with unlabeled data.

Unsupervised anomaly detection techniques create initially a region (e.g.,
a cluster in an n-dimensional hyperspace) that represents the limits of a
normal behavior so that any observation beyond those bounds is considered
an anomaly. They can easily (automatically) adapt to changes in the
incoming network traffic/events.

As far as we are concerned to recent systems for anomaly detection, there
is a clear trending on hybrid techniques (i.e., the combination of two or
more techniques) to overcome well-known limitations of each individual
class of techniques, such as low precision/recall, high processing
overhead, and the like. This means in general building a system with two or
more phases that combines supervised, semi-supervised, and unsupervised
learning in sequence.

The outputs of anomaly detection techniques can be scores and/or labels. Of
course, the objectives of the classification problem (i.e., either single
class or multiclass) define their outputs. Therefore, given the type of
classification problem, a number of methods can be applied, such as the
ones based on unsupervised clustering.

Challenges:

Current challenges for network anomaly detection includes i) dealing with
high dimensional data, class imbalance, and noise, ii) performing fast and
accurate feature engineering, iii) ensuring cluster homogeneity, iv)
lowering false alarm rate, and v) handling sequential, spatial, and graph
data simultaneously.

======================================================================

Cheers,

Stenio

On Mon, Aug 14, 2017 at 10:37 PM, yanshen <yanshen@huawei.com> wrote:

> Hi Haoyu,
>
> Agree. These two crucial cases in Network Management are what we are
> focusing on now. Since we plan to organize a dedicated session in NMRG, all
> the discussion will converge to the area of Network Management before Nov.
>
> Just expect Stenio's output few days later : )
>
> Yansen
>
>
> > -----Original Message-----
> > From: steniofernandes@gmail.com [mailto:steniofernandes@gmail.com] On
> > Behalf Of Stenio Fernandes
> > Sent: Tuesday, August 15, 2017 2:09 AM
> > To: Haoyu song <haoyu.song@huawei.com>
> > Cc: yanshen <yanshen@huawei.com>; idnet@ietf.org
> > Subject: Re: [Idnet] Summary 20170814 & IDN dedicated session call for
> case
> >
> > Haoyu,
> >
> > I'm working on the anomaly detection use case and will send it to the
> list this
> > week.
> >
> > Stenio
> >
> > On Mon, Aug 14, 2017 at 12:47 PM, Haoyu song <haoyu.song@huawei.com>
> > wrote:
> > > Yansen,
> > >
> > >
> > >
> > > I see two key use cases are missing in the current list: root cause
> > > analysis and anomaly detection. Those two are likely to use ML-based
> > > solutions and the first one has already received a lot of research.
> > >
> > >
> > >
> > > Haoyu
> > >
> > >
> > >
> > > From: IDNET [mailto:idnet-bounces@ietf.org] On Behalf Of yanshen
> > > Sent: Sunday, August 13, 2017 8:36 PM
> > > To: idnet@ietf.org
> > > Subject: [Idnet] Summary 20170814 & IDN dedicated session call for
> > > case
> > >
> > >
> > >
> > > Dear all,
> > >
> > >
> > >
> > > Here is a summary and some index (2017.08.14). Till now, whatever the
> > > case is supported or not, I tried to organize all the content and keep
> > > the core part. It is still welcome to contribute and discuss.
> > >
> > >
> > >
> > > If I miss something important, please let me know. Apologized in
> advance.
> > >
> > >
> > >
> > > Yansen
> > >
> > >
> > >
> > >
> > >
> > > ---------  Roadmap  ---------
> > >
> > > ***Aug. : Collecting the use cases (related with NM). Rough thoughts
> > > and requirements
> > >
> > > Sep. : Refining the cases and abstract the common elements
> > >
> > > Oct. : Deeply analysis. Especially on Data Format, control flow, or
> > > other key points
> > >
> > > Nov.: F2F discussions on IETF100
> > >
> > > ---------  Roadmap End  ---------
> > >
> > >
> > >
> > >
> > >
> > > 1. Gap and Requirement Analysis
> > >
> > >     1.1 Network Management requirement
> > >
> > >     1.2 TBD
> > >
> > > 2. Use Cases
> > >
> > >     2.1 Traffic Prediction
> > >
> > >                    Proposed by: yanshen@huawei.com
> > >
> > >                    Track:
> > > https://www.ietf.org/mail-archive/web/idnet/current/msg00131.html
> > >
> > >                    Abstract: Collect the history traffic data and
> > > external data which may influence the traffic. Predict the traffic in
> > > short/long/specific term. Avoid the congestion or risk in previously.
> > >
> > >
> > >
> > >     2.2 QoS Management
> > >
> > >                    Proposed by: yanshen@huawei.com
> > >
> > >                    Track:
> > > https://www.ietf.org/mail-archive/web/idnet/current/msg00131.html
> > >
> > >                    Abstract: Use multiple paths to distribute the
> > > traffic flows. Adjust the percentages. Avoid congestion and ensure QoS.
> > >
> > >
> > >
> > >     2.3 Application (and/or DDoS) detection
> > >
> > >                    Proposed by: aydinulas@gmx.net
> > >
> > >                    Track:
> > > https://www.ietf.org/mail-archive/web/idnet/current/msg00133.html
> > >
> > >                    Abstract: Detect the application (or attack) from
> > > network packets (HTTPS or plain) Collect the history traffic data and
> > > identify a service or attack (ex: Skype, Viber, DDoS attack etc.)
> > >
> > >
> > >
> > >          2.4 QoE Management
> > >
> > >                    Proposed by: albert.cabellos@gmail.com
> > >
> > >                    Track:
> > > https://www.ietf.org/mail-archive/web/idnet/current/msg00137.html
> > >
> > >                    Abstract: Collect low-level metrics (SNR, latency,
> > > jitter, losses, etc) and measure QoE. Then use ML to understand what
> > > is the relation between satisfactory QoE and the low-level metrics. As
> > > an example learn that when delay>N then QoE is degraded, but when
> > > M<delay<N then QoE is satisfactory for the customers (please note that
> > > QoE cannot be measured directly over your network). This is useful to
> > > understand how the network must be operated to provide satisfactory
> QoE.
> > >
> > >
> > >
> > >          2.5 (Encrypted) Traffic Classification
> > >
> > >                    Proposed by: jerome.francois@inria.fr;
> > > mskim16@etri.re.kr
> > >
> > >                    Track: [Jerome]
> > > https://www.ietf.org/mail-archive/web/idnet/current/msg00141.html ;
> > > [Min-Suk Kim]
> > > https://www.ietf.org/mail-archive/web/idnet/current/msg00153.html
> > >
> > >                    Abstract:
> > >
> > >                             [Jerome] collect flow-level traffic
> > > metrics such as protocol information but also meta metrics such as
> > > distribution of packet sizes, inter-arrival times... Then use such
> > > information to label the traffic with the underlying application
> > > assuming that the granularity of classification may vary (type of
> > > application, exact application name,
> > > version...)
> > >
> > >                             [Min-Suk Kim] continuously collect packet
> > > data, then applying learning process for traffic classification with
> > > generating application using deep learning models such as CNN
> > > (convolutional neural
> > > network) and RNN (recurrent neural network). Data-set to apply into
> > > the models are generated by processing with features of information
> > > from flow in packet data.
> > >
> > >
> > >
> > >          2.6 TBD
> > >
> > >
> > >
> > > 3. Data Focus
> > >
> > >     3.1 Data attribute
> > >
> > >     3.2 Data format
> > >
> > >     3.3 TBD
> > >
> > >
> > >
> > > 4. Support Technologies
> > >
> > >     4.1 Benchmarking Framework
> > >
> > >                    Proposed by: pedro@nict.go.jp
> > >
> > >                    Track:
> > > https://www.ietf.org/mail-archive/web/idnet/current/msg00146.html
> > >
> > >                    Abstract: A proper benchmarking framework
> > comprises
> > > a set of reference procedures, methods, and models that can (or better
> > > *must*) be followed to assess the quality of an AI mechanism proposed
> > > to be applied to the network management/control area. Moreover, and
> > > much more specific to the IDNET topics, is the inclusion, dependency,
> > > or just the general relation of a standard format enforced to the data
> > > that is used (input) and produced
> > > (output) by the framework, so a kind of "data market" can arise
> > > without requiring to transform the data. The initial scope of
> > > input/output data would be the datasets, but also the new knowledge
> > > items that are stated as a result of applying the benchmarking
> > > procedures defined by the framework, which can be collected together
> > > to build a database of benchmark results, or just contrasted with
> > > other existing entries in the database to know the position of the
> > > solution just evaluated. This increases the usefulness of IDNET.
> > >
> > >
> > >
> > >     4.2 TBD
> > >
> > >
> > > _______________________________________________
> > > IDNET mailing list
> > > IDNET@ietf.org
> > > https://www.ietf.org/mailman/listinfo/idnet
> > >
> >
> >
> >
> > --
> > Prof. Stenio Fernandes
> > CIn/UFPE
> > http://www.steniofernandes.com
>



-- 
Prof. Stenio Fernandes
CIn/UFPE
http://www.steniofernandes.com