idnits 2.17.1 draft-song-opsawg-dnp4iq-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (June 19, 2017) is 2500 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-07) exists of draft-brockners-inband-oam-data-02 == Outdated reference: A later version (-03) exists of draft-brockners-inband-oam-requirements-02 Summary: 0 errors (**), 0 flaws (~~), 3 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 OPSAWG H. Song, Ed. 3 Internet-Draft J. Gong 4 Intended status: Informational Huawei Technologies Co., Ltd 5 Expires: December 21, 2017 June 19, 2017 7 Requirements for Interactive Query with Dynamic Network Probes 8 draft-song-opsawg-dnp4iq-01 10 Abstract 12 This document discusses the motivation and requirements for 13 supporting interactive network queries and data collection through a 14 mechanism called Dynamic Network Probes (DNP). Network applications 15 and OAM have various data requirements from the data plane. The 16 unpredictable and interactive nature of the query for network data 17 analytics asks for dynamic and on-demand data collection 18 capabilities. As user programmable and configurable data plane is 19 becoming a reality, it can be extended to support interactive query 20 through DNPs. DNP supports node, path, and flow-based data 21 preprocessing and collection. For example, in-situ OAM (iOAM) with 22 user-defined flow-based data collection can be programmed and 23 configured through DNP. DNPs serve as a building block of an 24 integrated network data telemetry and analytics platform which 25 involves the network data plane as an active component for user- 26 defined data collection and preparation. 28 Status of This Memo 30 This Internet-Draft is submitted in full conformance with the 31 provisions of BCP 78 and BCP 79. 33 Internet-Drafts are working documents of the Internet Engineering 34 Task Force (IETF). Note that other groups may also distribute 35 working documents as Internet-Drafts. The list of current Internet- 36 Drafts is at http://datatracker.ietf.org/drafts/current/. 38 Internet-Drafts are draft documents valid for a maximum of six months 39 and may be updated, replaced, or obsoleted by other documents at any 40 time. It is inappropriate to use Internet-Drafts as reference 41 material or to cite them other than as "work in progress." 43 This Internet-Draft will expire on December 21, 2017. 45 Copyright Notice 47 Copyright (c) 2017 IETF Trust and the persons identified as the 48 document authors. All rights reserved. 50 This document is subject to BCP 78 and the IETF Trust's Legal 51 Provisions Relating to IETF Documents 52 (http://trustee.ietf.org/license-info) in effect on the date of 53 publication of this document. Please review these documents 54 carefully, as they describe your rights and restrictions with respect 55 to this document. Code Components extracted from this document must 56 include Simplified BSD License text as described in Section 4.e of 57 the Trust Legal Provisions and are provided without warranty as 58 described in the Simplified BSD License. 60 Table of Contents 62 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 63 2. Motivation for Interactive Query with DNP . . . . . . . . . . 3 64 3. Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . 5 65 3.1. In-Situ OAM with User Defined Data Collection . . . . . . 6 66 3.2. DDoS Detection . . . . . . . . . . . . . . . . . . . . . 6 67 3.3. Elephant Flow Identification . . . . . . . . . . . . . . 6 68 3.4. Network Congestion Monitoring . . . . . . . . . . . . . . 7 69 4. Enabling Technologies for DNP . . . . . . . . . . . . . . . . 7 70 5. Dynamic Network Probes . . . . . . . . . . . . . . . . . . . 9 71 5.1. DNP Types . . . . . . . . . . . . . . . . . . . . . . . . 11 72 5.1.1. Node Based . . . . . . . . . . . . . . . . . . . . . 11 73 5.1.2. Path Based . . . . . . . . . . . . . . . . . . . . . 12 74 5.1.3. Flow Based . . . . . . . . . . . . . . . . . . . . . 13 75 6. Interactive Query Architecture . . . . . . . . . . . . . . . 13 76 7. Requirements for IQ with DNP . . . . . . . . . . . . . . . . 14 77 8. Considerations for IQ with DNP . . . . . . . . . . . . . . . 15 78 8.1. Technical Challenges . . . . . . . . . . . . . . . . . . 15 79 8.2. Standard Consideration . . . . . . . . . . . . . . . . . 16 80 9. Security Considerations . . . . . . . . . . . . . . . . . . . 16 81 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 16 82 11. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 16 83 12. Informative References . . . . . . . . . . . . . . . . . . . 16 84 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 18 86 1. Introduction 88 Network service provider's pain points are often due to the lack of 89 network visibility. For example, network congestion collapse could 90 be avoided in many cases if it were known exactly when and where 91 congestion is happening or even better, if it could be precisely 92 predicted well before any impact is made; sophisticated network 93 attacks could be prevented through stateful and distributed network 94 behavior analysis. 96 In order to provide better application-centric services, user flows 97 and their interaction with networks need to be tracked and 98 understood. 100 The emerging trend of network automation aims to keep people out of 101 the OAM and control loop to the greatest extent for automated health 102 prediction, fault recovery, demand planning, network optimization, 103 and intrusion prevention, based on big data analytics and machine 104 learning technologies. 106 These applications need all kinds of network data, either passing 107 through networks or generated by network devices. For such 108 applications to be effective, the data of interest needs to be 109 retrieved in real time and on demand in an interactive and iterative 110 fashion. Continuous streaming data is often required. Therefore, it 111 is valuable to build a unified and general-purpose network telemetry 112 and analytics platform with integrated data plane support to provide 113 the complete network visibility at the minimum data bandwidth. This 114 is in contrast to the piecemeal solutions which only deal with one 115 single problem at a time. 117 We propose two ideas to enable such a vision. First, we devise the 118 Dynamic Network Probe (DNP) as a flexible and dynamic means for data 119 plane data collection and preprocessing, which can prepare data for 120 data analytics applications (Note that most of the DNPs are so common 121 that it makes perfect sense to predefine the standard data models for 122 them such that the conventional data plane devices can still be 123 designed and configured to support them). Second, we show the 124 possibility to build a universal network telemetry and analytics 125 platform with an Interactive Query (IQ) interface to the data plane 126 which can compile and deploy DNPs at runtime (or configure DNPs 127 dynamically based on standard data models). In such a system, 128 network devices play an integral and active role. We show a layered 129 architecture based on a programmable data plane which supports 130 interactive queries on network data. 132 In this document We discuss requirements, use cases, working items, 133 and challenges, with the hope to trigger community interests to 134 develop corresponding technologies and standards. 136 2. Motivation for Interactive Query with DNP 138 Network applications, such as traffic engineering, network security, 139 network health monitoring, trouble shooting, and fault diagnosis, 140 require different types of data collection. The data are either 141 normal traffic packets that are filtered, sampled, or digested, or 142 metadata generated by network devices to convey network states and 143 status. Broadly speaking, there are three types of data to be 144 collected from network data plane: path-based, flow-based, and node- 145 based. Path-based data is usually collected through dedicated 146 probing packets (e.g., ping and traceroute); Flow-based data 147 collection designates user flows to carry data of interest (e.g., in- 148 situ OAM [I-D.brockners-inband-oam-requirements]); Node-based data is 149 directly retrieved from selected network devices (e.g., ipfix 150 [RFC7011]). 152 Some data is considered atomic or primitive. For example, a packet's 153 arrival timestamp at a particular node cannot be further 154 disintegrated. The atomic data can be used to generate synthetic and 155 combinational data. For example, a packet's latency on a path can be 156 calculated through the packet timestamps at the end of the path. 157 Depending on the application, either data may be required. If the 158 application's real intent is the latter, it makes sense to directly 159 provide such data to reduce the data transfer bandwidth, at the cost 160 of a small processing overhead in the data plane and/or control 161 plane. Some synthetic and combinational data can be acquired through 162 multiple data types, but the most efficient way is preferred for a 163 specific network. For the similar purpose of data traffic reduction, 164 applications may not need the "raw" data all the time. Instead, they 165 may want data that is sampled and filtered, or only when some 166 predefined condition is met. Anyway, application's requirements on 167 data are diversified and unpredictable. Applications may need some 168 data which is not readily available at the time of request. 170 Some applications are interactive or iterative. After analyzing the 171 initial data, these applications may quickly shift interests to new 172 data or need to keep refining the data to be collected based on 173 previous observations (e.g., an elephant flow detector continues to 174 narrow down the flow granularity and gather statistics). The control 175 loop algorithms of these applications continuously interact with the 176 data plane and modify the data source and content in a highly dynamic 177 manner. 179 Ideally, to support all potential applications, we need full 180 visibility to know any states anytime anywhere in the entire network 181 data plane. In reality, this is extremely difficult if not 182 impossible. A strawman option is to mirror all the raw traffic to 183 servers where data analytics engine is running. This brute-force 184 method requires to double the device port count and the traffic 185 bandwidth, and poses enormous computing and storage cost. As a 186 tradeoff, Test Access Port (TAP) or Switch Port Analyzer (SPAN) is 187 used to selectively mirror only a portion of the overall traffic. 188 Network Packet Broker (NPB) is deployed along with TAP or SPAN to 189 process and distribute the raw data to various data analytics tools. 190 There are some other solutions (e.g., sflow [RFC3176] and ipfix 191 [RFC7011]) which can provide sampled and digested packet data and 192 some traffic statistics. Meanwhile, network devices also generate 193 various log files to record miscellaneous events in the system. 195 When aggregating all these solutions together, we can gain a 196 relatively comprehensive view of the network. However, the main 197 problem is the lack of a unified platform to deal with the general 198 network telemetry problem and the highly dynamic and unpredictable 199 data requirements. Moreover, each piecemeal solution inevitably 200 loses information due to data plane resource limitations which makes 201 the data analytical results suboptimal. 203 Trying to design an omnipotent system to support all possible runtime 204 data requests is also unviable because the resources required are 205 prohibitive (e.g., even a simple counter per flow is impossible in 206 practice). An alternative is to reprogram or reconfigure the data 207 plane device whenever an unsupported data request appears. This is 208 possible thanks to the recently available programmable chips and the 209 trend to open the programmability to service providers. 210 Unfortunately, the static programming approach cannot meet the real 211 time requirements due to the latency incurred by the programming and 212 compiling process. The reprogramming process also risks breaking the 213 normal operation of network devices. 215 Then a viable solution left to us is: whenever applications request 216 data which is yet unavailable in the data plane, the data plane can 217 be configured in real time to return the requested data. That is, we 218 do not attempt to make the network data plane provide all data all 219 the time. Instead, we only need to ensure that any application can 220 acquire necessary data instantly whenever it actually needs it. This 221 data-on-demand model can support effectively omni network visibility, 222 Note that data collection is meant to be passive and should not 223 change the network forwarding behavior. The active forwarding 224 behavior modification is out of the scope of this draft. 226 Data can be customized dynamically and polled or pushed based on 227 application's request. Moderate data preprocessing and preparation 228 by data plane devices may be needed. Such "in-network" processing 229 capability can be realized through DNP. 231 3. Use Cases 232 3.1. In-Situ OAM with User Defined Data Collection 234 In-situ OAM [I-D.brockners-inband-oam-requirements] collects data on 235 user traffic's forwarding path. From the control and management 236 plane point of view, each data collection task is a query from the 237 OAM application. In case the data collection function is not hard 238 coded in network devices, DNP can be dynamically deployed to support 239 the in-situ OAM. 241 While the current in-situ OAM drafts only concern the data plane 242 packet format and use cases, the applications still need a control 243 and management interface to dynamically enable and disable the in- 244 situ OAM functions, which involves the tasks such as choosing the 245 source and destination nodes on the path, the flow to carry the OAM 246 data, and the way to handle the data at the path end. These 247 configuration tasks can be done through DNP. 249 More importantly, in-situ OAM [I-D.brockners-inband-oam-data] may 250 collect user-defined data which are not available at device 251 configuration time. In this case, the data can be defined by DNP. 252 DNP can further help to preproess the data before sending the data to 253 the subscribing application. This can help to reduce the OAM header 254 size and the application's work load. 256 3.2. DDoS Detection 258 In a data center the security application wants to find the servers 259 under possible DDoS attack with a suspiciously large number of 260 connections. It can deploy DNPs on all the portal switches to 261 periodically report the number of unique flows targeting the set of 262 the protected servers. Once the queried data are collected, it is 263 easy to aggregate the data to find the potential DDoS attacks. 265 3.3. Elephant Flow Identification 267 An application wants to query the network-wide top-n flows. Various 268 algorithms have been developed at each network device to detect local 269 elephant flows. These algorithms can be defined as DNPs. A set of 270 network devices are chosen to deploy the DNPs so each will 271 periodically report the local elephant flows. The application will 272 aggregate the data to find the global elephant flows. The elephant 273 flow identification can be an interactive process. The application 274 may need to adjust the existing DNPs or deploy new DNPs to refine the 275 detection results. 277 In some cases, the local resource in a network device is not 278 sufficient to monitor the entire flow space. We can partition the 279 flow space and configure one network device in a group with a DNP to 280 track only a subset of flows, given the assumption that each device 281 can see all the flows. 283 3.4. Network Congestion Monitoring 285 Network congestion is reflected by packet drops at routers or 286 switches. While it is easy to get the packet drop count at each 287 network device, it is difficult to gain insights on the victims, hot 288 spots, and lossy paths. We can deploy DNPs to acquire such 289 information. DNPs are deployed on all network devices to collect the 290 detailed information about the dropped packet such as its signature 291 and the port it is dropped. Based on the collected data, the 292 application can generate the report on the top victims, hot spots, 293 and the most lossy paths. 295 4. Enabling Technologies for DNP 297 Network data plane is becoming user programmable. It means the 298 network operators are in control of customizing the network device's 299 function and forwarding behavior. Figure 1 shows the industry trend, 300 which shapes new forms of network devices and inspires innovative 301 ways to use them. 303 +-------+ +-------+ +-------+ 304 | | | | | | 305 | NOS | | APP | | APP | 306 | | | | | | 307 +-------+ +-------+ +-------+ 308 ^ ^ ^ 309 | | | runtime 310 decouple | ------> | config time ---> | interactive 311 | | programming | programming 312 V V V 313 +----------+ +-------------+ +-------------+ 314 | box with | | box with | | box with | 315 | fixed | | programmable| | interactive | 316 | function | | chip | | programmable| 317 | ASIC | | | | chip | 318 +----------+ +-------------+ +-------------+ 320 Figure 1: Towards User Programmable Data Plane 322 The first trend is led by the OCP networking project, which advocates 323 the decoupling of the network operating system and the network device 324 hardware. A common Switch Abstract Interface (SAI) allows 325 applications to run on heterogeneous substrate devices. However, 326 such devices are built with fixed function ASICs, which provide 327 limited flexibility for application customization. 329 The second trend is built upon the first one yet makes a big leap. 330 Chip and device vendors are working on opening the programmability of 331 the NPU, CPU, and FPGA-based network devices to network operators. 332 Most recently, programmable ASIC has been proven feasible. High 333 level languages such as P4 [DOI_10.1145_2656877.2656890] have been 334 developed to make the network device programming easy and fast. Now 335 a network device can be programmed into different functioning boxes 336 depending on the program installed. 338 However, such programming process is considered static. Even a minor 339 modification to the existing application requires to recompile the 340 updated source code and reinstall the application. This incurs long 341 deployment latency and may also temporarily break the normal data 342 plane operation. 344 User programmable data plane should be stretched further to support 345 runtime interactive programming in order to extend its scope of 346 usability, as proposed in POF [DOI_10.1145_2491185.2491190] Dynamic 347 application requirements cannot be foreseen at design time, and 348 runtime data plane modifications are required to be done in real time 349 (for agile control loop) and on demand (to meet data plane resource 350 constraints). Meanwhile, the data plane devices are capable of doing 351 more complex things such as stateful processing without always 352 resorting to a controller for state tracking. This allows network 353 devices to offload a significant portion of the data processing task 354 and only hand off the preprocessed data to the data-requesting 355 applications. 357 We can still use static programming with high level languages such as 358 P4 to define the main data plane processing and forwarding function. 359 But at runtime, whenever an application requires to make some 360 modification to the data plane, we deploy the incremental 361 modification directly through the runtime control channel. The key 362 to make this dynamic and interactive programming work is to maintain 363 a unified interface to devices for both configuration and runtime 364 control, because both programming paths share the same data plane 365 abstraction and use the same back-end adapting and mapping method. 367 NPU-based network devices and virtual network devices running on CPU/ 368 GPU can easily support the static and runtime in-service data plane 369 programmability. ASIC and FPGA-based network devices may be 370 difficult to support runtime programming and update natively. 371 However, for telemetry data collection tasks, the device local 372 controller (or even remote servers) can be used in conjunction with 373 the forwarding chip to complete the data preprocessing and 374 preparation. After all, applications do not care how the data probes 375 are implemented as long as the same API is maintained. 377 5. Dynamic Network Probes 379 Network probes are passive monitors which are installed at specific 380 forwarding data path locations to process and collect specific data. 381 DNPs are dynamically deployed and revoked probes by applications at 382 runtime. The customizable DNPs can collect simple statistics or 383 conduct more complex data preprocessing. Since DNPs may require 384 actively modifying the existing data path pipeline beyond simple flow 385 entry manipulation, these operations need to be done through 386 interactive programming process. When a DNP is revoked, the involved 387 shared resources are automatically recycled and returned back to the 388 global resource pool. 390 DNPs can be deployed at various data path locations including port, 391 queue, buffer, table, and table entry. When the data plane 392 programmability is extended to cover other components (e.g., CPU 393 load, fan speed, GPS coordination, etc.), DNPs can be deployed to 394 collect corresponding data as well. A few data plane objectives can 395 be composed to form probes. These objectives are counter, meter, 396 timer, timestamp, register, and table. Combining these with the 397 packet filter through flow table entry configuration, one can easily 398 monitor and catch arbitrary states on the data plane. 400 In practice, DNP can be considered a virtual concept. Its deployment 401 can be done through either configuration or programming. For less 402 flexible platforms, probes can be predefined but support on-demand 403 runtime activation. Complex DNP functions can also be achieved 404 through collaboration between data plane and control plane. Most 405 common DNPs can be modeled for easy implementation. The goal is to 406 make DNP implementation transparent to upper layer applications. 408 The simplest probe is just a counter. The counter can be configured 409 to count bytes or packets and the counting can be conditional. The 410 more complex probes can be considered as Finite State Machines (FSM) 411 which are configured to capture specific events. FSMs essentially 412 preprocess the raw stream data and only report the necessary data to 413 subscribing applications. 415 Applications can use poll mode or push mode to access probes and 416 collect data. The normal counter probes are often accessed via poll 417 mode. Applications decide what time and how often the counter value 418 is read. On the other hand, the complex FSM probes are usually 419 accessed in push mode. When the target event is triggered, a report 420 is generated and pushed to the application. 422 Timer is a special global resource. A timer can be configured to 423 link to some action. When the time is up, the corresponding action 424 is executed. For example, to get notification when a port load 425 exceeds some threshold, we can set a timer with a fixed time-out 426 interval, and link the timer to an action which reads the counter and 427 generates the report packet if the condition is triggered. This way, 428 the application avoids the need to keep polling statistics from the 429 data plane. 431 With the use of global registers and state tables, more complex FSM 432 probes can be implemented. For example, to monitor the half-open TCP 433 connections, for each SYN request, we store the flow signature to a 434 state table. Then for each ACK packet, the state table is checked 435 and the matched entry is removed. The state table can be 436 periodically polled to acquire the list of half-open connections. 437 The application can also choose to only retrieve the counter of half- 438 open connections. When the counter exceeds some threshold, further 439 measures can be taken to examine if a SYN flood attack is going on. 441 Registers can be considered mini state tables which are good to track 442 a single flow and a few state transitions. For example, to get the 443 duration of a particular flow, when the flow is established, the 444 state and the timestamp are recorded in a register; when the flow is 445 torn down, the flow duration can be calculated with the old timestamp 446 and the new timestamp. In another example, we want to monitor a 447 queue by setting a low water mark and a high water mark for the fill 448 level. Every time when an enqueue or a dequeue event happens, the 449 queue depth is compared with the marks and a report packet is 450 generated when a mark is crossed. 452 Some probes are essentially packet filters which are used to filter 453 out a portion of the traffic and mirrored the traffic to the 454 application or some other target port for further processing. There 455 are two ways to implement a packet filter: use a flow table that 456 matches on the filtering criteria and specify the associated action; 457 or directly make a decision in the action. An example of the former 458 case is to filter all packets with a particular source IP address. 459 An example of the latter case is to filter all TCP FIN packets at the 460 edge. Although we can always use a flow table to filter traffic, 461 sometimes it is more efficient and convenient to directly work on the 462 action. As being programmed by the application, the filtered traffic 463 can be further processed before being sent. Two most common 464 processes are digest and sample, both aiming to reduce the quantity 465 of raw data. The digest process prunes the unnecessary data from the 466 original packet and only packs the useful information in the digest 467 packet. The sample process picks a subset of filtered traffic to 468 send based on some predefined sampling criteria. The two processes 469 can be used jointly to maximize the data reduction effect. 471 An application may need to install multiple DNPs in one device or 472 across multiple devices to finish one data analytical task. For 473 example, to measure the latency of any link in a network. We install 474 a DNP on the source node to generate probe packets with timestamp. 475 We install another DNP at the sink node to capture the probe packets 476 and report both the source timestamp and the sink timestamp to the 477 application for link latency calculation. The probe packets are also 478 dropped by the sink DNP. The source DNP can be configured to 479 generate probe packets at any rate. It can also generate just one 480 probe packet per application request. 482 Using the similar idea, we can deploy DNPs to measure the end-to-end 483 flow latency or trace exact flow paths. In this case, the DNPs can 484 be deployed to enable the corresponding iOAM in-situ data collection 485 service. At the path end, the DNP calculates the desired output 486 based on the collected data. 488 Applications could have many such custom data requests. Each request 489 lasts various time and consumes various network resources. Dynamic 490 probe configuration or programming is not only efficient but also 491 necessary. In summary, DNP is a versatile tool to prepare and 492 generate just-in-time telemetry data for data analytical 493 applications. 495 5.1. DNP Types 497 DNP can be roughly grouped into three types: node-based, path-based, 498 and flow-based. Following is the list of DNPs. Some are atomic and 499 the others can be derived from the atomic ones. Note that the list 500 is by no means comprehensive. The list does not include the device 501 state and status data that is steadily available. Depending on the 502 device capability, more complex DNPs can be implemented. 503 Applications can subscribe data from multiple DNPs to meet their 504 needs. The flow-based data can be directly provided by iOAM data or 505 derived from iOAM data. 507 5.1.1. Node Based 509 o Streaming Packets 511 * Filter flow by user-defined flow definition. 513 * Sample with user-defined sample rate. The sample can be based 514 on interval or probability. 516 * Generate packet digest with user defined format. 518 o Flow Counter 519 * Associate poll-mode counter for user-defined flow. 521 * Associate push-mode counter for user-defined flow. The counter 522 value is pushed at user-defined threshold or interval. 524 o Flow Meter 526 * Associate poll-mode meter for user-defined flow. 528 * Associate push-mode meter for user-defined flow. The meter 529 value is pushed at user-defined threshold or interval. 531 o Queue 533 * Queue depth for designated queue is polled or pushed at user- 534 defined threshold or interval. 536 * Designated buffer depth is polled or pushed at user-defined 537 threshold or interval. 539 o Time 541 * Time gap between user-defined flow packets is polled or pushed 542 in streaming data or at user-defined threshold. 544 * Arrival/Departure/Sojourn time of user-defined flow packets is 545 polled or pushed streaming data or at user defined threshold. 547 o Statistics 549 * Number of active flows, elephant flows, and mice flows. 551 5.1.2. Path Based 553 o Number of active flows per node on the path. 555 o Path latency. 557 o Round trip time of the path. 559 o Node ID and ingress/egress port of the path. 561 o Hop count of the path. 563 o Buffer/queue depth of the nodes on the path. 565 o Workload of the nodes on the path. 567 5.1.3. Flow Based 569 o Flow Latency: Latency at each hop or cumulative E2E latency for 570 user-defined flow. 572 o Flow Jitter: Jitter at each hop or on the entire path for user- 573 defined flow. 575 o Flow Bandwidth: Bandwidth at each hop or the bottleneck bandwidth 576 on the entire path for user-defined flow. 578 o Flow Path Trace: Port and Node ID, and other data of the path for 579 user-defined flow. 581 o Proof of Transit (PoT) for particular set of nodes. 583 6. Interactive Query Architecture 585 In the past, network data analytics is considered a separate function 586 from networks. They consume raw data extracted from networks through 587 piecemeal protocols and interfaces. With the advent of user 588 programmable data plane, we expect a paradigm shift that makes the 589 data plane be an active component of the data analytics solution. 590 The programmable in-network data preprocessing is efficient and 591 flexible to offload some light-weight data processing through dynamic 592 data plane programming or configuration. A universal network data 593 analytics platform built on top of this enables a tight and agile 594 network control and OAM feedback loop. 596 While DNP is a passive data plane data collection mechanism, we need 597 to provide a query interface for applications to use the DNPs for 598 data analytics. A proposed dynamic networking data analytical system 599 architecture is illustrated in Figure 2. An application translates 600 its data requirements into some dynamic transactional queries. The 601 queries are then compiled into a set of DNPs targeting a subset of 602 data plane devices (Note that in a less flexible target with 603 predefined models, DNPs are configured). After the DNPs are 604 deployed, each DNP conducts in-network data preprocessing and feeds 605 the preprocessed data to the collector. The collector finishes the 606 data post-processing and presents the results to the data-requesting 607 application. 609 +------------------------------------+ 610 |network data analytics applications | 611 +----------- ------------------------+ 612 ^ 613 V 614 +------------------------------------+ 615 |dynamic and interactive query | 616 +------------------------------------+ 617 ^ | 618 | V 619 +---------------+ +------------------+ 620 |post process | |DNP compile/config| 621 +---------------+ +------------------+ 622 ^ | 623 | V 624 +---------------+ +---------------+ 625 |data collection| |DNP deployment | 626 +---------------+ +---------------+ 627 ^ ^ ^ | | | 628 | | | V V V 629 +------------------------------------+ 630 |network data plane | 631 |(in-network data preprocessing) | 632 +------------------------------------+ 634 Figure 2: Architecture of IQ with DNP 636 A query can be either continuous or one-shot. The continuous query 637 may require the application to refine the existing DNPs or deploy new 638 DNPs. When an application revokes its queries, the idle DNP resource 639 is released. Since one DNP may be subscribed by multiple 640 applications, the runtime system needs to keep track of the active 641 DNPs. 643 7. Requirements for IQ with DNP 645 This section lists the requirements for interactive query with DNP: 647 o Applications should conduct interactive query through a standard 648 interface (i.e., API). The system is responsible to compile the 649 IQ into DNPs and deploy the DNPs to the corresponding network 650 nodes. 652 o DNPs can be deployed through some standard south bound interface 653 and protocols such as gRPC, NETCONF, etc. 655 o The interactive query should not modify the forwarding behavior. 656 The API should provide the necessary isolation. 658 o The deployed DNP should not lower the forwarding performance of 659 the data plane devices. If the DNP would affect the forwarding 660 performance, the query should be denied. 662 o The system should support multiple parallel queries from multiple 663 applications. 665 o One application can deploy different DNPs to a set of network 666 nodes and these DNPs work jointly to finish a function. 668 o DNP may be revoked and preempted by the controller due to resource 669 conflict and application priority. 671 8. Considerations for IQ with DNP 673 8.1. Technical Challenges 675 Some technical issues need to be addressed to realize interactive 676 query with DNP on general network data plane: 678 o Allowing applications to modify the data plane has security and 679 safety risks (e.g., DoS attack). The counter measure is to supply 680 a standard and safe API to segregate applications from the runtime 681 system and provide applications limited accessibility to the data 682 plane. Each API can be easily compiled and mapped to standard 683 DNPs. An SQL-like query language which adapts to the stream 684 processing system might be feasible for the applications. 686 o When multiple correlated DNPs are deployed across multiple network 687 devices or function blocks, or when multiple applications request 688 the same DNPs, the deployment consistency needs to be guaranteed 689 for correctness. This requires a robust runtime compiling and 690 management system which keeps track of the subscription to DNPs 691 and controls the DNP execution time and order. 693 o The performance impact of DNPs must be evaluated before deployment 694 to avoid unintentionally reducing the forwarding throughput. 695 Fortunately, the resource consumption and performance impact of 696 standard DNPs can be accurately profiled in advance. A device is 697 usually over provisioned and is capable of absorbing extra 698 functions up to a limit. Moreover, programmable data plane allows 699 users to tailor their forwarding application to the bare bones so 700 more resources can be reserved for probes. The runtime system 701 needs to evaluate the resulting throughput performance before 702 committing a DNP. If it is unacceptable, either some old DNPs 703 need to be revoked or the new request must be denied. 705 o While DNP is relatively easy to be implemented in software-based 706 platform (e.g., NPU and CPU), it is harder in ASIC-based 707 programmable chips. Architectural and algorithmic innovations are 708 needed to support a more flexible pipeline which allows new 709 pipeline stage, new tables, and new custom actions to be inserted 710 at runtime through hitless in-service updates. An architecture 711 with shared memory and flexible processor cores might be viable to 712 meet these requirements. Alternatively, DNPs can be implemented 713 using an "out-of-band" fashion. That is, the slow path processor 714 is engaged in conjunction with the forwarding chip to complete the 715 DNP function. 717 8.2. Standard Consideration 719 The query API can be potentially standardized. The actually DNP 720 deployment interface may consider to reuse or extend the IETF 721 standards and drafts such as gRPC [I-D.talwar-rtgwg-grpc-use-cases] 722 and NETCONF [RFC6241]. We may also define standard telemetry YANG 723 [RFC6020] models for common DNPs so these DNPs can be used in a 724 configurable way. 726 9. Security Considerations 728 Allowing applications to modify the data plane has security and 729 safety risks (e.g., DoS attack). The counter measure is to supply 730 standard and safe API to segregate applications from the runtime 731 system and provide applications limited accessibility to the data 732 plane. Each API can be easily compiled and mapped to standard DNPs. 733 An SQL-like query language which adapts to the stream processing 734 system might be feasible and secure for the applications. 736 10. IANA Considerations 738 This memo includes no request to IANA. 740 11. Acknowledgments 742 The authors would like to thank Frank Brockners, Carlos Pignataro, 743 Tom Tofigh, Bert Wijnen, Stewart Bryant, James Guichard, and Tianran 744 Zhou for the valuable comments and advice. 746 12. Informative References 748 [DOI_10.1145_2491185.2491190] 749 Song, H., "Protocol-oblivious forwarding", Proceedings of 750 the second ACM SIGCOMM workshop on Hot topics in software 751 defined networking - HotSDN '13 , 752 DOI 10.1145/2491185.2491190, 2013. 754 [DOI_10.1145_2656877.2656890] 755 Bosshart, P., Varghese, G., Walker, D., Daly, D., Gibb, 756 G., Izzard, M., McKeown, N., Rexford, J., Schlesinger, C., 757 Talayco, D., and A. Vahdat, "P4", ACM SIGCOMM Computer 758 Communication Review Vol. 44, pp. 87-95, 759 DOI 10.1145/2656877.2656890, July 2014. 761 [I-D.brockners-inband-oam-data] 762 Brockners, F., Bhandari, S., Pignataro, C., Gredler, H., 763 Leddy, J., Youell, S., Mizrahi, T., Mozes, D., Lapukhov, 764 P., and R. <>, "Data Formats for In-situ OAM", draft- 765 brockners-inband-oam-data-02 (work in progress), October 766 2016. 768 [I-D.brockners-inband-oam-requirements] 769 Brockners, F., Bhandari, S., Dara, S., Pignataro, C., 770 Gredler, H., Leddy, J., Youell, S., Mozes, D., Mizrahi, 771 T., <>, P., and r. remy@barefootnetworks.com, 772 "Requirements for In-situ OAM", draft-brockners-inband- 773 oam-requirements-02 (work in progress), October 2016. 775 [I-D.talwar-rtgwg-grpc-use-cases] 776 Specification, g., Kolhe, J., Shaikh, A., and J. George, 777 "Use cases for gRPC in network management", draft-talwar- 778 rtgwg-grpc-use-cases-01 (work in progress), January 2017. 780 [RFC3176] Phaal, P., Panchen, S., and N. McKee, "InMon Corporation's 781 sFlow: A Method for Monitoring Traffic in Switched and 782 Routed Networks", RFC 3176, DOI 10.17487/RFC3176, 783 September 2001, . 785 [RFC6020] Bjorklund, M., Ed., "YANG - A Data Modeling Language for 786 the Network Configuration Protocol (NETCONF)", RFC 6020, 787 DOI 10.17487/RFC6020, October 2010, 788 . 790 [RFC6241] Enns, R., Ed., Bjorklund, M., Ed., Schoenwaelder, J., Ed., 791 and A. Bierman, Ed., "Network Configuration Protocol 792 (NETCONF)", RFC 6241, DOI 10.17487/RFC6241, June 2011, 793 . 795 [RFC7011] Claise, B., Ed., Trammell, B., Ed., and P. Aitken, 796 "Specification of the IP Flow Information Export (IPFIX) 797 Protocol for the Exchange of Flow Information", STD 77, 798 RFC 7011, DOI 10.17487/RFC7011, September 2013, 799 . 801 Authors' Addresses 803 Haoyu Song (editor) 804 Huawei Technologies Co., Ltd 805 2330 Central Expressway 806 Santa Clara, 95050 807 USA 809 Email: haoyu.song@huawei.com 811 Jun Gong 812 Huawei Technologies Co., Ltd 813 156 Beiqing Road 814 Beijing, 100095 815 P.R. China 817 Email: gongjun@huawei.com