idnits 2.17.1 draft-song-opsa-dnp4iq-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == It seems as if not all pages are separated by form feeds - found 0 form feeds but 18 pages Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (June 15, 2017) is 2500 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-07) exists of draft-brockners-inband-oam-data-02 == Outdated reference: A later version (-03) exists of draft-brockners-inband-oam-requirements-02 Summary: 0 errors (**), 0 flaws (~~), 4 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 OPSAWG H. Song, Ed. 3 Internet-Draft J. Gong 4 Intended status: Informational H. Chen 5 Expires: December 17, 2017 Huawei Technologies Co., Ltd 6 June 15, 2017 8 Requirements for Interactive Query with Dynamic Network Probes 9 draft-song-opsa-dnp4iq-00 11 Abstract 13 This document discusses the motivation and requirements for 14 supporting interactive network queries and data collection through a 15 mechanism called Dynamic Network Probes (DNP). Network applications 16 and OAM have various data requirements from the data plane. The 17 unpredictable and interactive nature of the query for network data 18 analytics asks for dynamic and on-demand data collection 19 capabilities. As user programmable data plane is becoming a reality, 20 it can be enhanced to support interactive query through DNPs. DNP 21 supports node, path, and flow-based data preprocessing and 22 collection. For example, in-situ OAM (iOAM) with user-defined flow- 23 based data collection can be programmed and configured through DNP. 24 DNPs serve as a building block of an integrated network data 25 telemetry and analytics platform which involves the network data 26 plane as an active component for user-defined data collection and 27 preparation. 29 Status of This Memo 31 This Internet-Draft is submitted in full conformance with the 32 provisions of BCP 78 and BCP 79. 34 Internet-Drafts are working documents of the Internet Engineering 35 Task Force (IETF). Note that other groups may also distribute 36 working documents as Internet-Drafts. The list of current Internet- 37 Drafts is at http://datatracker.ietf.org/drafts/current/. 39 Internet-Drafts are draft documents valid for a maximum of six months 40 and may be updated, replaced, or obsoleted by other documents at any 41 time. It is inappropriate to use Internet-Drafts as reference 42 material or to cite them other than as "work in progress." 44 This Internet-Draft will expire on December 17, 2017. 46 Copyright Notice 48 Copyright (c) 2017 IETF Trust and the persons identified as the 49 document authors. All rights reserved. 51 This document is subject to BCP 78 and the IETF Trust's Legal 52 Provisions Relating to IETF Documents 53 (http://trustee.ietf.org/license-info) in effect on the date of 54 publication of this document. Please review these documents 55 carefully, as they describe your rights and restrictions with respect 56 to this document. Code Components extracted from this document must 57 include Simplified BSD License text as described in Section 4.e of 58 the Trust Legal Provisions and are provided without warranty as 59 described in the Simplified BSD License. 61 Table of Contents 63 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 64 2. Motivation for Interactive Query with DNP . . . . . . . . . . 3 65 3. Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . 5 66 3.1. In-Situ OAM with User Defined Data Collection . . . . . . 6 67 3.2. DDoS Detection . . . . . . . . . . . . . . . . . . . . . 6 68 3.3. Elephant Flow Identification . . . . . . . . . . . . . . 6 69 3.4. Network Congestion Monitoring . . . . . . . . . . . . . . 7 70 4. Enabling Technologies for DNP . . . . . . . . . . . . . . . . 7 71 5. Dynamic Network Probes . . . . . . . . . . . . . . . . . . . 9 72 5.1. DNP Types . . . . . . . . . . . . . . . . . . . . . . . . 11 73 5.1.1. Node Based . . . . . . . . . . . . . . . . . . . . . 11 74 5.1.2. Path Based . . . . . . . . . . . . . . . . . . . . . 12 75 5.1.3. Flow Based . . . . . . . . . . . . . . . . . . . . . 13 76 6. Interactive Query Architecture . . . . . . . . . . . . . . . 13 77 7. Requirements for IQ with DNP . . . . . . . . . . . . . . . . 14 78 8. Considerations for IQ with DNP . . . . . . . . . . . . . . . 15 79 8.1. Technical Challenges . . . . . . . . . . . . . . . . . . 15 80 8.2. Standard Consideration . . . . . . . . . . . . . . . . . 16 81 9. Security Considerations . . . . . . . . . . . . . . . . . . . 16 82 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 16 83 11. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 16 84 12. Informative References . . . . . . . . . . . . . . . . . . . 16 85 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 18 87 1. Introduction 89 Network service provider's pain points are often due to the lack of 90 network visibility. For example, network congestion collapse could 91 be avoided in many cases if it were known exactly when and where 92 congestion is happening or even better, if it could be precisely 93 predicted well before any impact is made; sophisticated network 94 attacks could be prevented through stateful and distributed network 95 behavior analysis. 97 In order to provide better application-centric services, user flows 98 and their interaction with networks need to be tracked and 99 understood. 101 The emerging trend of network automation aims to keep people out of 102 the OAM and control loop to the greatest extent for automated health 103 prediction, fault recovery, demand planning, network optimization, 104 and intrusion prevention, based on big data analytics and machine 105 learning technologies. 107 These applications need all kinds of network data, either passing 108 through networks or generated by network devices. For such 109 applications to be effective, the data of interest needs to be 110 retrieved in real time and on demand in an interactive and iterative 111 fashion. Continuous streaming data is often required. Therefore, it 112 is valuable to build a unified and general-purpose network telemetry 113 and analytics platform with integrated data plane support to provide 114 the complete network visibility at the minimum data bandwidth. This 115 is in contrast to the piecemeal solutions which only deal with one 116 single problem at a time. 118 We propose two ideas to enable such a vision. First, we devise the 119 Dynamic Network Probe (DNP) as a flexible and dynamic means for data 120 plane data collection and preprocessing, which can prepare data for 121 data analytics applications (Note that most of the DNPs are so common 122 that it makes perfect sense to predefine the standard data models for 123 them such that the conventional data plane devices can still be 124 designed and configured to support them). Second, we show the 125 possibility to build a universal network telemetry and analytics 126 platform with an Interactive Query (IQ) interface to the data plane 127 which can compile and deploy DNPs at runtime (or configure DNPs 128 dynamically based on standard data models). In such a system, 129 network devices play an integral and active role. We show a layered 130 architecture based on a programmable data plane which supports 131 interactive queries on network data. 133 In this document We discuss requirements, use cases, working items, 134 and challenges, with the hope to trigger community interests to 135 develop corresponding technologies and standards. 137 2. Motivation for Interactive Query with DNP 139 Network applications, such as traffic engineering, network security, 140 network health monitoring, trouble shooting, and fault diagnosis, 141 require different types of data collection. The data are either 142 normal traffic packets that are filtered, sampled, or digested, or 143 metadata generated by network devices to convey network states and 144 status. Broadly speaking, there are three types of data to be 145 collected from network data plane: path-based, flow-based, and node- 146 based. Path-based data is usually collected through dedicated 147 probing packets (e.g., ping and traceroute); Flow-based data 148 collection designates user flows to carry data of interest (e.g., in- 149 situ OAM [I-D.brockners-inband-oam-requirements]); Node-based data is 150 directly retrieved from selected network devices (e.g., ipfix 151 [RFC7011]). 153 Some data is considered atomic or primitive. For example, a packet's 154 arrival timestamp at a particular node cannot be further 155 disintegrated. The atomic data can be used to generate synthetic and 156 combinational data. For example, a packet's latency on a path can be 157 calculated through the packet timestamps at the end of the path. 158 Depending on the application, either data may be required. If the 159 application's real intent is the latter, it makes sense to directly 160 provide such data to reduce the data transfer bandwidth, at the cost 161 of a small processing overhead in the data plane and/or control 162 plane. Some synthetic and combinational data can be acquired through 163 multiple data types, but the most efficient way is preferred for a 164 specific network. For the similar purpose of data traffic reduction, 165 applications may not need the "raw" data all the time. Instead, they 166 may want data that is sampled and filtered, or only when some 167 predefined condition is met. Anyway, application's requirements on 168 data are diversified and unpredictable. Applications may need some 169 data which is not readily available at the time of request. 171 Some applications are interactive or iterative. After analyzing the 172 initial data, these applications may quickly shift interests to new 173 data or need to keep refining the data to be collected based on 174 previous observations (e.g., an elephant flow detector continues to 175 narrow down the flow granularity and gather statistics). The control 176 loop algorithms of these applications continuously interact with the 177 data plane and modify the data source and content in a highly dynamic 178 manner. 180 Ideally, to support all potential applications, we need full 181 visibility to know any states anytime anywhere in the entire network 182 data plane. In reality, this is extremely difficult if not 183 impossible. A strawman option is to mirror all the raw traffic to 184 servers where data analytics engine is running. This brute-force 185 method requires to double the device port count and the traffic 186 bandwidth, and poses enormous computing and storage cost. As a 187 tradeoff, Test Access Port (TAP) or Switch Port Analyzer (SPAN) is 188 used to selectively mirror only a portion of the overall traffic. 189 Network Packet Broker (NPB) is deployed along with TAP or SPAN to 190 process and distribute the raw data to various data analytics tools. 191 There are some other solutions (e.g., sflow [RFC3176] and ipfix 192 [RFC7011]) which can provide sampled and digested packet data and 193 some traffic statistics. Meanwhile, network devices also generate 194 various log files to record miscellaneous events in the system. 196 When aggregating all these solutions together, we can gain a 197 relatively comprehensive view of the network. However, the main 198 problem is the lack of a unified platform to deal with the general 199 network telemetry problem and the highly dynamic and unpredictable 200 data requirements. Moreover, each piecemeal solution inevitably 201 loses information due to data plane resource limitations which makes 202 the data analytical results suboptimal. 204 Trying to design an omnipotent system to support all possible runtime 205 data requests is also unviable because the resources required are 206 prohibitive (e.g., even a simple counter per flow is impossible in 207 practice). An alternative is to reprogram or reconfigure the data 208 plane device whenever an unsupported data request appears. This is 209 possible thanks to the recently available programmable chips and the 210 trend to open the programmability to service providers. 211 Unfortunately, the static programming approach cannot meet the real 212 time requirements due to the latency incurred by the programming and 213 compiling process. The reprogramming process also risks breaking the 214 normal operation of network devices. 216 Then a viable solution left to us is: whenever applications request 217 data which is yet unavailable in the data plane, the data plane can 218 be configured in real time to return the requested data. That is, we 219 do not attempt to make the network data plane provide all data all 220 the time. Instead, we only need to ensure that any application can 221 acquire necessary data instantly whenever it actually needs it. This 222 data-on-demand model can support effectively omni network visibility, 223 Note that data collection is meant to be passive and should not 224 change the network forwarding behavior. The active forwarding 225 behavior modification is out of the scope of this draft. 227 Data can be customized dynamically and polled or pushed based on 228 application's request. Moderate data preprocessing and preparation 229 by data plane devices may be needed. Such "in-network" processing 230 capability can be realized through DNP. 232 3. Use Cases 233 3.1. In-Situ OAM with User Defined Data Collection 235 In-situ OAM [I-D.brockners-inband-oam-requirements] collects data on 236 user traffic's forwarding path. From the control and management 237 plane point of view, each data collection task is a query from the 238 OAM application. In case the data collection function is not hard 239 coded in network devices, DNP can be dynamically deployed to support 240 the in-situ OAM. 242 While the current in-situ OAM drafts only concern the data plane 243 packet format and use cases, the applications still need a control 244 and management interface to dynamically enable and disable the in- 245 situ OAM functions, which involves the tasks such as choosing the 246 source and destination nodes on the path, the flow to carry the OAM 247 data, and the way to handle the data at the path end. These 248 configuration tasks can be done through DNP. 250 More importantly, in-situ OAM [I-D.brockners-inband-oam-data] may 251 collect user-defined data which are not available at device 252 configuration time. In this case, the data can be defined by DNP. 253 DNP can further help to preproess the data before sending the data to 254 the subscribing application. This can help to reduce the OAM header 255 size and the application's work load. 257 3.2. DDoS Detection 259 In a data center the security application wants to find the servers 260 under possible DDoS attack with a suspiciously large number of 261 connections. It can deploy DNPs on all the portal switches to 262 periodically report the number of unique flows targeting the set of 263 the protected servers. Once the queried data are collected, it is 264 easy to aggregate the data to find the potential DDoS attacks. 266 3.3. Elephant Flow Identification 268 An application wants to query the network-wide top-n flows. Various 269 algorithms have been developed at each network device to detect local 270 elephant flows. These algorithms can be defined as DNPs. A set of 271 network devices are chosen to deploy the DNPs so each will 272 periodically report the local elephant flows. The application will 273 aggregate the data to find the global elephant flows. The elephant 274 flow identification can be an interactive process. The application 275 may need to adjust the existing DNPs or deploy new DNPs to refine the 276 detection results. 278 In some cases, the local resource in a network device is not 279 sufficient to monitor the entire flow space. We can partition the 280 flow space and configure one network device in a group with a DNP to 281 track only a subset of flows, given the assumption that each device 282 can see all the flows. 284 3.4. Network Congestion Monitoring 286 Network congestion is reflected by packet drops at routers or 287 switches. While it is easy to get the packet drop count at each 288 network device, it is difficult to gain insights on the victims, hot 289 spots, and lossy paths. We can deploy DNPs to acquire such 290 information. DNPs are deployed on all network devices to collect the 291 detailed information about the dropped packet such as its signature 292 and the port it is dropped. Based on the collected data, the 293 application can generate the report on the top victims, hot spots, 294 and the most lossy paths. 296 4. Enabling Technologies for DNP 298 Network data plane is becoming user programmable. It means the 299 network operators are in control of customizing the network device's 300 function and forwarding behavior. Figure 1 shows the industry trend, 301 which shapes new forms of network devices and inspires innovative 302 ways to use them. 304 +-------+ +-------+ +-------+ 305 | | | | | | 306 | NOS | | APP | | APP | 307 | | | | | | 308 +-------+ +-------+ +-------+ 309 ^ ^ ^ 310 | | | runtime 311 decouple | ------> | config time ---> | interactive 312 | | programming | programming 313 V V V 314 +----------+ +-------------+ +-------------+ 315 | box with | | box with | | box with | 316 | fixed | | programmable| | interactive | 317 | function | | chip | | programmable| 318 | ASIC | | | | chip | 319 +----------+ +-------------+ +-------------+ 321 Figure 1: Towards User Programmable Data Plane 323 The first trend is led by the OCP networking project, which advocates 324 the decoupling of the network operating system and the network device 325 hardware. A common Switch Abstract Interface (SAI) allows 326 applications to run on heterogeneous substrate devices. However, 327 such devices are built with fixed function ASICs, which provide 328 limited flexibility for application customization. 330 The second trend is built upon the first one yet makes a big leap. 331 Chip and device vendors are working on opening the programmability of 332 the NPU, CPU, and FPGA-based network devices to network operators. 333 Most recently, programmable ASIC has been proven feasible. High 334 level languages such as P4 [DOI_10.1145_2656877.2656890] have been 335 developed to make the network device programming easy and fast. Now 336 a network device can be programmed into different functioning boxes 337 depending on the program installed. 339 However, such programming process is considered static. Even a minor 340 modification to the existing application requires to recompile the 341 updated source code and reinstall the application. This incurs long 342 deployment latency and may also temporarily break the normal data 343 plane operation. 345 User programmable data plane should be stretched further to support 346 runtime interactive programming in order to extend its scope of 347 usability, as proposed in POF [DOI_10.1145_2491185.2491190] Dynamic 348 application requirements cannot be foreseen at design time, and 349 runtime data plane modifications are required to be done in real time 350 (for agile control loop) and on demand (to meet data plane resource 351 constraints). Meanwhile, the data plane devices are capable of doing 352 more complex things such as stateful processing without always 353 resorting to a controller for state tracking. This allows network 354 devices to offload a significant portion of the data processing task 355 and only hand off the preprocessed data to the data-requesting 356 applications. 358 We can still use static programming with high level languages such as 359 P4 to define the main data plane processing and forwarding function. 360 But at runtime, whenever an application requires to make some 361 modification to the data plane, we deploy the incremental 362 modification directly through the runtime control channel. The key 363 to make this dynamic and interactive programming work is to maintain 364 a unified interface to devices for both configuration and runtime 365 control, because both programming paths share the same data plane 366 abstraction and use the same back-end adapting and mapping method. 368 NPU-based network devices and virtual network devices running on CPU/ 369 GPU can easily support the static and runtime in-service data plane 370 programmability. ASIC and FPGA-based network devices may be 371 difficult to support runtime programming and update natively. 372 However, for telemetry data collection tasks, the device local 373 controller (or even remote servers) can be used in conjunction with 374 the forwarding chip to complete the data preprocessing and 375 preparation. After all, applications do not care how the data probes 376 are implemented as long as the same API is maintained. 378 5. Dynamic Network Probes 380 Network probes are passive monitors which are installed at specific 381 forwarding data path locations to process and collect specific data. 382 DNPs are dynamically deployed and revoked probes by applications at 383 runtime. The customizable DNPs can collect simple statistics or 384 conduct more complex data preprocessing. Since DNPs may require 385 actively modifying the existing data path pipeline beyond simple flow 386 entry manipulation, these operations need to be done through 387 interactive programming process. When a DNP is revoked, the involved 388 shared resources are automatically recycled and returned back to the 389 global resource pool. 391 DNPs can be deployed at various data path locations including port, 392 queue, buffer, table, and table entry. When the data plane 393 programmability is extended to cover other components (e.g., CPU 394 load, fan speed, GPS coordination, etc.), DNPs can be deployed to 395 collect corresponding data as well. A few data plane objectives can 396 be composed to form probes. These objectives are counter, meter, 397 timer, timestamp, register, and table. Combining these with the 398 packet filter through flow table entry configuration, one can easily 399 monitor and catch arbitrary states on the data plane. 401 In practice, DNP can be considered a virtual concept. Its deployment 402 can be done through either configuration or programming. For less 403 flexible platforms, probes can be predefined but support on-demand 404 runtime activation. Complex DNP functions can also be achieved 405 through collaboration between data plane and control plane. Most 406 common DNPs can be modeled for easy implementation. The goal is to 407 make DNP implementation transparent to upper layer applications. 409 The simplest probe is just a counter. The counter can be configured 410 to count bytes or packets and the counting can be conditional. The 411 more complex probes can be considered as Finite State Machines (FSM) 412 which are configured to capture specific events. FSMs essentially 413 preprocess the raw stream data and only report the necessary data to 414 subscribing applications. 416 Applications can use poll mode or push mode to access probes and 417 collect data. The normal counter probes are often accessed via poll 418 mode. Applications decide what time and how often the counter value 419 is read. On the other hand, the complex FSM probes are usually 420 accessed in push mode. When the target event is triggered, a report 421 is generated and pushed to the application. 423 Timer is a special global resource. A timer can be configured to 424 link to some action. When the time is up, the corresponding action 425 is executed. For example, to get notification when a port load 426 exceeds some threshold, we can set a timer with a fixed time-out 427 interval, and link the timer to an action which reads the counter and 428 generates the report packet if the condition is triggered. This way, 429 the application avoids the need to keep polling statistics from the 430 data plane. 432 With the use of global registers and state tables, more complex FSM 433 probes can be implemented. For example, to monitor the half-open TCP 434 connections, for each SYN request, we store the flow signature to a 435 state table. Then for each ACK packet, the state table is checked 436 and the matched entry is removed. The state table can be 437 periodically polled to acquire the list of half-open connections. 438 The application can also choose to only retrieve the counter of half- 439 open connections. When the counter exceeds some threshold, further 440 measures can be taken to examine if a SYN flood attack is going on. 442 Registers can be considered mini state tables which are good to track 443 a single flow and a few state transitions. For example, to get the 444 duration of a particular flow, when the flow is established, the 445 state and the timestamp are recorded in a register; when the flow is 446 torn down, the flow duration can be calculated with the old timestamp 447 and the new timestamp. In another example, we want to monitor a 448 queue by setting a low water mark and a high water mark for the fill 449 level. Every time when an enqueue or a dequeue event happens, the 450 queue depth is compared with the marks and a report packet is 451 generated when a mark is crossed. 453 Some probes are essentially packet filters which are used to filter 454 out a portion of the traffic and mirrored the traffic to the 455 application or some other target port for further processing. There 456 are two ways to implement a packet filter: use a flow table that 457 matches on the filtering criteria and specify the associated action; 458 or directly make a decision in the action. An example of the former 459 case is to filter all packets with a particular source IP address. 460 An example of the latter case is to filter all TCP FIN packets at the 461 edge. Although we can always use a flow table to filter traffic, 462 sometimes it is more efficient and convenient to directly work on the 463 action. As being programmed by the application, the filtered traffic 464 can be further processed before being sent. Two most common 465 processes are digest and sample, both aiming to reduce the quantity 466 of raw data. The digest process prunes the unnecessary data from the 467 original packet and only packs the useful information in the digest 468 packet. The sample process picks a subset of filtered traffic to 469 send based on some predefined sampling criteria. The two processes 470 can be used jointly to maximize the data reduction effect. 472 An application may need to install multiple DNPs in one device or 473 across multiple devices to finish one data analytical task. For 474 example, to measure the latency of any link in a network. We install 475 a DNP on the source node to generate probe packets with timestamp. 476 We install another DNP at the sink node to capture the probe packets 477 and report both the source timestamp and the sink timestamp to the 478 application for link latency calculation. The probe packets are also 479 dropped by the sink DNP. The source DNP can be configured to 480 generate probe packets at any rate. It can also generate just one 481 probe packet per application request. 483 Using the similar idea, we can deploy DNPs to measure the end-to-end 484 flow latency or trace exact flow paths. In this case, the DNPs can 485 be deployed to enable the corresponding iOAM in-situ data collection 486 service. At the path end, the DNP calculates the desired output 487 based on the collected data. 489 Applications could have many such custom data requests. Each request 490 lasts various time and consumes various network resources. Dynamic 491 probe configuration or programming is not only efficient but also 492 necessary. In summary, DNP is a versatile tool to prepare and 493 generate just-in-time telemetry data for data analytical 494 applications. 496 5.1. DNP Types 498 DNP can be roughly grouped into three types: node-based, path-based, 499 and flow-based. Following is the list of DNPs. Some are atomic and 500 the others can be derived from the atomic ones. Note that the list 501 is by no means comprehensive. The list does not include the device 502 state and status data that is steadily available. Depending on the 503 device capability, more complex DNPs can be implemented. 504 Applications can subscribe data from multiple DNPs to meet their 505 needs. The flow-based data can be directly provided by iOAM data or 506 derived from iOAM data. 508 5.1.1. Node Based 510 o Streaming Packets 512 * Filter flow by user-defined flow definition. 514 * Sample with user-defined sample rate. The sample can be based 515 on interval or probability. 517 * Generate packet digest with user defined format. 519 o Flow Counter 520 * Associate poll-mode counter for user-defined flow. 522 * Associate push-mode counter for user-defined flow. The counter 523 value is pushed at user-defined threshold or interval. 525 o Flow Meter 527 * Associate poll-mode meter for user-defined flow. 529 * Associate push-mode meter for user-defined flow. The meter 530 value is pushed at user-defined threshold or interval. 532 o Queue 534 * Queue depth for designated queue is polled or pushed at user- 535 defined threshold or interval. 537 * Designated buffer depth is polled or pushed at user-defined 538 threshold or interval. 540 o Time 542 * Time gap between user-defined flow packets is polled or pushed 543 in streaming data or at user-defined threshold. 545 * Arrival/Departure/Sojourn time of user-defined flow packets is 546 polled or pushed streaming data or at user defined threshold. 548 o Statistics 550 * Number of active flows, elephant flows, and mice flows. 552 5.1.2. Path Based 554 o Number of active flows per node on the path. 556 o Path latency. 558 o Round trip time of the path. 560 o Node ID and ingress/egress port of the path. 562 o Hop count of the path. 564 o Buffer/queue depth of the nodes on the path. 566 o Workload of the nodes on the path. 568 5.1.3. Flow Based 570 o Flow Latency: Latency at each hop or cumulative E2E latency for 571 user-defined flow. 573 o Flow Jitter: Jitter at each hop or on the entire path for user- 574 defined flow. 576 o Flow Bandwidth: Bandwidth at each hop or the bottleneck bandwidth 577 on the entire path for user-defined flow. 579 o Flow Path Trace: Port and Node ID, and other data of the path for 580 user-defined flow. 582 o Proof of Transit (PoT) for particular set of nodes. 584 6. Interactive Query Architecture 586 In the past, network data analytics is considered a separate function 587 from networks. They consume raw data extracted from networks through 588 piecemeal protocols and interfaces. With the advent of user 589 programmable data plane, we expect a paradigm shift that makes the 590 data plane be an active component of the data analytics solution. 591 The programmable in-network data preprocessing is efficient and 592 flexible to offload some light-weight data processing through dynamic 593 data plane programming or configuration. A universal network data 594 analytics platform built on top of this enables a tight and agile 595 network control and OAM feedback loop. 597 While DNP is a passive data plane data collection mechanism, we need 598 to provide a query interface for applications to use the DNPs for 599 data analytics. A proposed dynamic networking data analytical system 600 architecture is illustrated in Figure 2. An application translates 601 its data requirements into some dynamic transactional queries. The 602 queries are then compiled into a set of DNPs targeting a subset of 603 data plane devices (Note that in a less flexible target with 604 predefined models, DNPs are configured). After the DNPs are 605 deployed, each DNP conducts in-network data preprocessing and feeds 606 the preprocessed data to the collector. The collector finishes the 607 data post-processing and presents the results to the data-requesting 608 application. 610 +------------------------------------+ 611 |network data analytics applications | 612 +----------- ------------------------+ 613 ^ 614 V 615 +------------------------------------+ 616 |dynamic and interactive query | 617 +------------------------------------+ 618 ^ | 619 | V 620 +---------------+ +------------------+ 621 |post process | |DNP compile/config| 622 +---------------+ +------------------+ 623 ^ | 624 | V 625 +---------------+ +---------------+ 626 |data collection| |DNP deployment | 627 +---------------+ +---------------+ 628 ^ ^ ^ | | | 629 | | | V V V 630 +------------------------------------+ 631 |network data plane | 632 |(in-network data preprocessing) | 633 +------------------------------------+ 635 Figure 2: Architecture of IQ with DNP 637 A query can be either continuous or one-shot. The continuous query 638 may require the application to refine the existing DNPs or deploy new 639 DNPs. When an application revokes its queries, the idle DNP resource 640 is released. Since one DNP may be subscribed by multiple 641 applications, the runtime system needs to keep track of the active 642 DNPs. 644 7. Requirements for IQ with DNP 646 This section lists the requirements for interactive query with DNP: 648 o Applications should conduct interactive query through a standard 649 interface (i.e., API). The system is responsible to compile the 650 IQ into DNPs and deploy the DNPs to the corresponding network 651 nodes. 653 o DNPs can be deployed through some standard south bound interface 654 and protocols such as gRPC, NETCONF, etc. 656 o The interactive query should not modify the forwarding behavior. 657 The API should provide the necessary isolation. 659 o The deployed DNP should not lower the forwarding performance of 660 the data plane devices. If the DNP would affect the forwarding 661 performance, the query should be denied. 663 o The system should support multiple parallel queries from multiple 664 applications. 666 o One application can deploy different DNPs to a set of network 667 nodes and these DNPs work jointly to finish a function. 669 o DNP may be revoked and preempted by the controller due to resource 670 conflict and application priority. 672 8. Considerations for IQ with DNP 674 8.1. Technical Challenges 676 Some technical issues need to be addressed to realize interactive 677 query with DNP on general network data plane: 679 o Allowing applications to modify the data plane has security and 680 safety risks (e.g., DoS attack). The counter measure is to supply 681 a standard and safe API to segregate applications from the runtime 682 system and provide applications limited accessibility to the data 683 plane. Each API can be easily compiled and mapped to standard 684 DNPs. An SQL-like query language which adapts to the stream 685 processing system might be feasible for the applications. 687 o When multiple correlated DNPs are deployed across multiple network 688 devices or function blocks, or when multiple applications request 689 the same DNPs, the deployment consistency needs to be guaranteed 690 for correctness. This requires a robust runtime compiling and 691 management system which keeps track of the subscription to DNPs 692 and controls the DNP execution time and order. 694 o The performance impact of DNPs must be evaluated before deployment 695 to avoid unintentionally reducing the forwarding throughput. 696 Fortunately, the resource consumption and performance impact of 697 standard DNPs can be accurately profiled in advance. A device is 698 usually over provisioned and is capable of absorbing extra 699 functions up to a limit. Moreover, programmable data plane allows 700 users to tailor their forwarding application to the bare bones so 701 more resources can be reserved for probes. The runtime system 702 needs to evaluate the resulting throughput performance before 703 committing a DNP. If it is unacceptable, either some old DNPs 704 need to be revoked or the new request must be denied. 706 o While DNP is relatively easy to be implemented in software-based 707 platform (e.g., NPU and CPU), it is harder in ASIC-based 708 programmable chips. Architectural and algorithmic innovations are 709 needed to support a more flexible pipeline which allows new 710 pipeline stage, new tables, and new custom actions to be inserted 711 at runtime through hitless in-service updates. An architecture 712 with shared memory and flexible processor cores might be viable to 713 meet these requirements. Alternatively, DNPs can be implemented 714 using an "out-of-band" fashion. That is, the slow path processor 715 is engaged in conjunction with the forwarding chip to complete the 716 DNP function. 718 8.2. Standard Consideration 720 The query API can be potentially standardized. The actually DNP 721 deployment interface may consider to reuse or extend the IETF 722 standards and drafts such as gRPC [I-D.talwar-rtgwg-grpc-use-cases] 723 and NETCONF [RFC6241]. We may also define standard telemetry YANG 724 [RFC6020] models for common DNPs so these DNPs can be used in a 725 configurable way. 727 9. Security Considerations 729 Allowing applications to modify the data plane has security and 730 safety risks (e.g., DoS attack). The counter measure is to supply 731 standard and safe API to segregate applications from the runtime 732 system and provide applications limited accessibility to the data 733 plane. Each API can be easily compiled and mapped to standard DNPs. 734 An SQL-like query language which adapts to the stream processing 735 system might be feasible and secure for the applications. 737 10. IANA Considerations 739 This memo includes no request to IANA. 741 11. Acknowledgments 743 The authors would like to thank Frank Brockners, Carlos Pignataro, 744 Tom Tofigh, Bert Wijnen, Stewart Bryant, James Guichard, and Tianran 745 Zhou for the valuable comments and advice. 747 12. Informative References 749 [DOI_10.1145_2491185.2491190] 750 Song, H., "Protocol-oblivious forwarding", Proceedings of 751 the second ACM SIGCOMM workshop on Hot topics in software 752 defined networking - HotSDN '13 , 753 DOI 10.1145/2491185.2491190, 2013. 755 [DOI_10.1145_2656877.2656890] 756 Bosshart, P., Varghese, G., Walker, D., Daly, D., Gibb, 757 G., Izzard, M., McKeown, N., Rexford, J., Schlesinger, C., 758 Talayco, D., and A. Vahdat, "P4", ACM SIGCOMM Computer 759 Communication Review Vol. 44, pp. 87-95, 760 DOI 10.1145/2656877.2656890, July 2014. 762 [I-D.brockners-inband-oam-data] 763 Brockners, F., Bhandari, S., Pignataro, C., Gredler, H., 764 Leddy, J., Youell, S., Mizrahi, T., Mozes, D., Lapukhov, 765 P., and R. <>, "Data Formats for In-situ OAM", draft- 766 brockners-inband-oam-data-02 (work in progress), October 767 2016. 769 [I-D.brockners-inband-oam-requirements] 770 Brockners, F., Bhandari, S., Dara, S., Pignataro, C., 771 Gredler, H., Leddy, J., Youell, S., Mozes, D., Mizrahi, 772 T., <>, P., and r. remy@barefootnetworks.com, 773 "Requirements for In-situ OAM", draft-brockners-inband- 774 oam-requirements-02 (work in progress), October 2016. 776 [I-D.talwar-rtgwg-grpc-use-cases] 777 Specification, g., Kolhe, J., Shaikh, A., and J. George, 778 "Use cases for gRPC in network management", draft-talwar- 779 rtgwg-grpc-use-cases-01 (work in progress), January 2017. 781 [RFC3176] Phaal, P., Panchen, S., and N. McKee, "InMon Corporation's 782 sFlow: A Method for Monitoring Traffic in Switched and 783 Routed Networks", RFC 3176, DOI 10.17487/RFC3176, 784 September 2001, . 786 [RFC6020] Bjorklund, M., Ed., "YANG - A Data Modeling Language for 787 the Network Configuration Protocol (NETCONF)", RFC 6020, 788 DOI 10.17487/RFC6020, October 2010, 789 . 791 [RFC6241] Enns, R., Ed., Bjorklund, M., Ed., Schoenwaelder, J., Ed., 792 and A. Bierman, Ed., "Network Configuration Protocol 793 (NETCONF)", RFC 6241, DOI 10.17487/RFC6241, June 2011, 794 . 796 [RFC7011] Claise, B., Ed., Trammell, B., Ed., and P. Aitken, 797 "Specification of the IP Flow Information Export (IPFIX) 798 Protocol for the Exchange of Flow Information", STD 77, 799 RFC 7011, DOI 10.17487/RFC7011, September 2013, 800 . 802 Authors' Addresses 804 Haoyu Song (editor) 805 Huawei Technologies Co., Ltd 806 2330 Central Expressway 807 Santa Clara, 95050 808 USA 810 Email: haoyu.song@huawei.com 812 Jun Gong 813 Huawei Technologies Co., Ltd 814 156 Beiqing Road 815 Beijing, 100095 816 P.R. China 818 Email: gongjun@huawei.com 820 Hongfei Chen 821 Huawei Technologies Co., Ltd 822 156 Beiqing Road 823 Beijing, 100095 824 P.R. China 826 Email: chenhongfei@huawei.com