idnits 2.17.1 draft-song-opsawg-ifit-framework-06.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an Introduction section. ** There is 1 instance of too long lines in the document, the longest one being 4 characters in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (October 21, 2019) is 1648 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- -- Looks like a reference, but probably isn't: '1' on line 698 == Unused Reference: 'RFC8321' is defined on line 690, but no explicit reference was found in the text == Outdated reference: A later version (-03) exists of draft-herbert-ipv4-eh-01 == Outdated reference: A later version (-09) exists of draft-ietf-ippm-multipoint-alt-mark-02 == Outdated reference: A later version (-07) exists of draft-kumar-ippm-ifa-01 == Outdated reference: A later version (-15) exists of draft-mirsky-ippm-hybrid-two-step-04 == Outdated reference: A later version (-16) exists of draft-song-ippm-postcard-based-telemetry-05 == Outdated reference: A later version (-13) exists of draft-song-mpls-extension-header-02 == Outdated reference: A later version (-14) exists of draft-zhou-ippm-enhanced-alternate-marking-03 -- Obsolete informational reference (is this intentional?): RFC 8321 (Obsoleted by RFC 9341) Summary: 2 errors (**), 0 flaws (~~), 10 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 OPSAWG H. Song, Ed. 3 Internet-Draft Futurewei 4 Intended status: Informational Z. Li 5 Expires: April 23, 2020 T. Zhou 6 Huawei 7 F. Qin 8 China Mobile 9 H. Chen 10 China Telecom 11 J. Jin 12 LG U+ 13 J. Shin 14 SK Telecom 15 October 21, 2019 17 In-situ Flow Information Telemetry 18 draft-song-opsawg-ifit-framework-06 20 Abstract 22 For efficient network operation, most operators rely on traditional 23 Operation, Administration and Maintenance (OAM) methods, which 24 include proactive and reactive techniques, running in active and 25 passive modes. As networks increase in scale they become more 26 susceptible to measurement accuracy and misconfiguration errors. 28 With the advent of programmable data-plane emerging on-path OAM 29 techniques, such as flow telemetry, provide unprecedented insight and 30 fast notification of network issues (e.g., jitter, increased latency, 31 packet loss, significant bit error variations, and unequal load- 32 balancing). 34 In-situ Flow Information Telemetry (iFIT) provides a method for 35 efficiently applying underlying on-path flow telemetry techniques, 36 applicable across various network environments. 38 This document outlines an iFIT framework, which enumerates several 39 critical functional components and describes how these components are 40 assembled together to achieve a complete and closed-loop working 41 solution for on-path flow telemetry. 43 Requirements Language 45 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 46 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 47 "OPTIONAL" in this document are to be interpreted as described in BCP 48 14 [RFC2119][RFC8174] when, and only when, they appear in all 49 capitals, as shown here. 51 Status of This Memo 53 This Internet-Draft is submitted in full conformance with the 54 provisions of BCP 78 and BCP 79. 56 Internet-Drafts are working documents of the Internet Engineering 57 Task Force (IETF). Note that other groups may also distribute 58 working documents as Internet-Drafts. The list of current Internet- 59 Drafts is at https://datatracker.ietf.org/drafts/current/. 61 Internet-Drafts are draft documents valid for a maximum of six months 62 and may be updated, replaced, or obsoleted by other documents at any 63 time. It is inappropriate to use Internet-Drafts as reference 64 material or to cite them other than as "work in progress." 66 This Internet-Draft will expire on April 23, 2020. 68 Copyright Notice 70 Copyright (c) 2019 IETF Trust and the persons identified as the 71 document authors. All rights reserved. 73 This document is subject to BCP 78 and the IETF Trust's Legal 74 Provisions Relating to IETF Documents 75 (https://trustee.ietf.org/license-info) in effect on the date of 76 publication of this document. Please review these documents 77 carefully, as they describe your rights and restrictions with respect 78 to this document. Code Components extracted from this document must 79 include Simplified BSD License text as described in Section 4.e of 80 the Trust Legal Provisions and are provided without warranty as 81 described in the Simplified BSD License. 83 Table of Contents 85 1. Requirements and Challenges . . . . . . . . . . . . . . . . . 3 86 2. iFIT Framework Overview . . . . . . . . . . . . . . . . . . . 5 87 2.1. Passport vs. Postcard . . . . . . . . . . . . . . . . . . 6 88 3. Architectural Components of iFIT . . . . . . . . . . . . . . 7 89 3.1. Smart Flow and Data Selection . . . . . . . . . . . . . . 7 90 3.1.1. Use Case: Sketch-guided Elephant Flow Selection . . . 8 91 3.1.2. Use Case: Adaptive Packet Sampling . . . . . . . . . 8 92 3.2. Smart Data Export . . . . . . . . . . . . . . . . . . . . 8 93 3.2.1. Use Case: On-demand Anomaly Monitor . . . . . . . . . 9 94 3.3. Dynamic Network Probe . . . . . . . . . . . . . . . . . . 9 95 3.4. Encapsulation and Tunneling . . . . . . . . . . . . . . . 10 96 3.5. On-demand Technique Selection and Integration . . . . . . 10 97 3.6. iFIT Closed-Loop Architecture . . . . . . . . . . . . . . 10 98 4. Intelligent Closed-Loop Network Telemetry Applications . . . 12 99 5. Summary and Future Work . . . . . . . . . . . . . . . . . . . 12 100 6. Security Considerations . . . . . . . . . . . . . . . . . . . 13 101 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 13 102 8. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 13 103 9. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 13 104 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 13 105 10.1. Normative References . . . . . . . . . . . . . . . . . . 13 106 10.2. Informative References . . . . . . . . . . . . . . . . . 13 107 10.3. URIs . . . . . . . . . . . . . . . . . . . . . . . . . . 15 108 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 15 110 1. Requirements and Challenges 112 The sheer complexity of today's networks requires radical rethinking 113 of existing methods network monitoring and troubleshooting. Current 114 dynamic networks require "on-path" fault monitoring and traffic 115 measurement solutions for a wide range of use cases which include 116 intelligent management of existing network traffic, and better 117 traffic visibility of emerging applications such as large scale 118 Virtual Server (VS) mobility, fluid content distribution, and elastic 119 bandwidth allocation. 121 Furthermore, the ability to expedite failure detection, fault 122 localization, and recovery mechanisms, particularly in the case of 123 soft failures or path degradation are experienced, without causing 124 extreme or obvious disruption. This is extremely important for since 125 these types of network issues are often difficult to localize with 126 existing Operation, Administration and Maintenance (OAM) methods and 127 reduce overall network efficiency. 129 Future networks must also support application-aware networking. 130 Application-aware networking is an emerging industry term and 131 typically used to describe the capacity of an intelligent network to 132 maintain current information about user and application connections 133 that use network resources and, as a result, the operator is able to 134 optimize the network resource usage and monitoring to ensure 135 application and traffic optimality. 137 Application-aware network operation is important for user SLA 138 compliance, service path enforcement, fault diagnosis, and network 139 resource optimization. A family of on-path flow telemetry 140 techniques, including In-situ OAM (IOAM) 141 [I-D.brockners-inband-oam-data], Postcard Based Telemetry (PBT) 142 [I-D.song-ippm-postcard-based-telemetry], In-band Flow Analyzer (IFA) 143 [I-D.kumar-ippm-ifa], Enhanced Alternate Marking (EAM) 145 [I-D.zhou-ippm-enhanced-alternate-marking], and Hybrid Two Steps 146 (HTS) [I-D.mirsky-ippm-hybrid-two-step], are emerging, which can 147 provide flow information on the entire forwarding path on a per- 148 packet basis in real time. These on-path flow telemetry techniques 149 are very different from the previous active and passive OAM schemes 150 in that they directly modify the user packets. Given the unique 151 characteristics of the aforementioned techniques, we may categorize 152 these on-path telemetry techniques as the hybrid OAM type III, 153 supplementing the classification defined in [RFC7799]. 155 These techniques are invaluable for application-aware network 156 operations not only in data center and enterprise networks but also 157 in carrier networks which may cross multiple domains. Carrier 158 network operators have shown strong interests in utilizing such 159 techniques for various purposes. For example, it is vital for the 160 operators who offer the bandwidth intensive, latency and loss 161 sensitive services such as video streaming and gaming to closely 162 monitor the relevant flows in real time as the indispensable first 163 step for any further measure. 165 However, successfully applying such techniques in carrier networks 166 poses several practical challenges: 168 o C1: On-path flow telemetry incurs extra packet processing which 169 may strain the network data plane. The potential impact on the 170 forwarding performance creates an unfavorable "observer effect" 171 which not only damages the fidelity of the measurement but also 172 defies the purpose of the measurement. 174 o C2: On-path flow telemetry can generate a huge amount of OAM data 175 which may claim too much transport bandwidth and inundate the 176 servers for data collection, storage, and analysis. Increasing 177 the data handling capacity is technically viable but expensive. 178 For example, assume IOAM is applied to all the traffic. One node 179 will collect a few tens of bytes as telemetry data for each 180 packets. The whole forwarding path might accumulate a data trace 181 with a size similar to the average size of the original packets. 182 Exporting the telemetry data will consume almost half of the 183 network bandwidth. 185 o C3: The collectible data defined currently are essential but 186 limited. As the network operation evolves to be declarative 187 (intent-based) and automated, and the trends of network 188 virtualization, network convergence, and packet-optical 189 integration continue, more data will be needed in an on-demand and 190 interactive fashion. Flexibility and extensibility on data 191 defining, acquisition, and filtering, must be considered. 193 o C4: If we were to apply some on-path telemetry technique in 194 today's carrier networks, we must provide solutions to tailor the 195 provider's network deployment base and support an incremental 196 deployment strategy. That is, we need to support established 197 encapsulation schemes for various predominant protocols such as 198 Ethernet, IPv4, and MPLS with backward compatibility and properly 199 handle various transport tunnels. 201 o C5: Applying only a single underlying telemetry technique may lead 202 to defective result. For example, packet drop can cause the lost 203 of the flow telemetry data and the packet drop location and reason 204 remains unknown if only In-situ OAM trace option is used. A 205 comprehensive solution needs the flexibility to switch between 206 different underlying techniques and adjust the configurations and 207 parameters at runtime. 209 o C6: Development of simplified on-path telemetry primitives and 210 models, including: telemetry data (e.g., nodes, links, ports, 211 paths, flows, timestamps) query primitives. These may be used by 212 an API-based telemetry service for external applications, for 213 monitoring end-to-end latency measurement of network paths and 214 application latency calculation. 216 2. iFIT Framework Overview 218 To address these challenges, we propose a framework based on multiple 219 network operators' requirements and the common industry practice, 220 which can help to build a workable on-path flow telemetry solution. 221 We name the framework "In-situ Flow Information Telemetry" (iFIT) to 222 reflect the fact that this framework is dedicated to the on-path 223 telemetry data about user/application flow experience. As a solution 224 framework, iFIT works a level higher than any specific OAM 225 techniques, be it active, passive, or hybrid. The framework is built 226 up on a few architectural components. By assembling these components 227 together, a closed-loop is formed to provide a complete solution for 228 a particular static, dynamic, and interactive telemetry applications. 230 iFIT is an open framework. It does not enforce any implementation 231 details for each component. Users are free to pick one or more 232 underlying techniques and design their own algorithms and 233 architectures to fit in each component and make all the components 234 work in concert. 236 The network architecture that applies iFIT is shown in Figure 1. The 237 iFIT domain is confined between the iFIT head nodes and the iFIT end 238 nodes. An iFIT domain may cross multiple network domains. iFIT 239 support two basic on-path telemetry data collection modes: passport 240 mode (e.g., IOAM trace option and IFA), in which telemetry data are 241 carried in user packets and exported at the iFIT end nodes, and 242 postcard mode (e.g., PBT), in which each node in the iFIT domain may 243 export telemetry data through independent OAM packets. Note that the 244 boundary between the two modes can be blurry. An application only 245 need to mix the two modes. 247 +---------------------------------+ 248 | | 249 | iFIT Applications | 250 | | 251 +---------------------------------+ 252 ^ ^ 253 | | 254 V | 255 +------------+ +-----+-----+ 256 | | | | 257 | Controller | | Collector | 258 | | | | 259 +-----:------+ +-----------+ 260 : ^ 261 :configuration |telemetry data 262 : | 263 ...............:.....................|.......... 264 : : : | : 265 : +---------:---+-------------:---++---------:---+ 266 : | : | : | : | 267 V | V | V | V | 268 +------+-+ +-----+--+ +------+-+ +------+-+ 269 usr pkts | iFIT | | Path | | Path | | iFIT | 270 ====>| Head |====>| Node |==//==>| Node |====>| End |====> 271 | Node | | A | | B | | Node | 272 +--------+ +--------+ +--------+ +--------+ 274 Figure 1: iFIT Network Architecture 276 2.1. Passport vs. Postcard 278 [passport-postcard] first uses the analogy of passport and postcard 279 to describe how the packet trace data can be collected and exported. 280 In the passport mode, each node on the path adds the telemetry data 281 to the user packets. The accumulated data trace is exported at a 282 configured end node. In the postcard mode, each node directly 283 exports the telemetry data using an independent packet while the user 284 packets are intact. 286 A prominent advantage of the passport mode is that it naturally 287 retains the telemetry data correlation along the entire path. The 288 passport mode also reduces the number of data export packets and the 289 bandwidth consumed by the data export packets. These can help to 290 make the data collector and analyzer's work easier. On the other 291 hand, the passport mode requires more processing on the user packets 292 and increases the size of user packets, which can cause various 293 problems. Some other issues are documented in 294 [I-D.song-ippm-postcard-based-telemetry]. 296 The postcard mode provides a perfect complement to the passport mode. 297 It addresses most of the issues faced by the passport mode, at a cost 298 of needing extra efforts to correlate the postcard packets. 300 3. Architectural Components of iFIT 302 The key components of iFIT are listed as follows: 304 o Smart flow and data selection policy to address C1. 306 o Smart data export to address C2. 308 o Dynamic network probe to address C3. 310 o Encapsulation and tunneling to address C4. 312 o On-demand technique selection and integration to address C5. 314 Next we provide the detailed description of each component. 316 3.1. Smart Flow and Data Selection 318 In most cases, it is impractical to enable the data collection for 319 all the flows and for all the packets in a flow due to the potential 320 performance and bandwidth impacts. Therefore, a workable solution 321 must select only a subset of flows and flow packets to enable the 322 data collection, even though this means the loss of some information. 324 In data plane, the Access Control List (ACL) provides an ideal means 325 to determine the subset of flow(s). 326 [I-D.song-ippm-ioam-data-validation-option] describes how one can set 327 a sample rate or probability to a flow to allow only a subset of flow 328 packets to be monitored, how one can collect different set of data 329 for different packets, and how one can disable or enable data 330 collection on any specific network node. The document further 331 introduces enhancement to IOAM to allow any node to accept or deny 332 the data collection in full or partially. 334 Based on these flexible mechanisms, iFIT allows applications to apply 335 smart flow and data selection policies to suit the requirements. The 336 applications can dynamically change the policies at any time based on 337 the network load, processing capability, focus of interest, and any 338 other criteria. We have developed some adaptive algorithm which can 339 limit the performance impact and yet achieve the satisfactory 340 telemetry data density. 342 3.1.1. Use Case: Sketch-guided Elephant Flow Selection 344 Network operators are usually more interested in elephant flows which 345 consume more resource and are sensitive to network condition changes. 346 We implement a CountMin Sketch [CMSketch] on the data path of the 347 head nodes, which identifies and reports the elephant flows 348 periodically. The controller maintains a current set of elephant 349 flows and dynamically enables the on-path telemetry for only these 350 flows. 352 3.1.2. Use Case: Adaptive Packet Sampling 354 Applying on-path telemetry on all packets of selected flows can still 355 be out of reach. We should set a sample rate for these flows and 356 only enable telemetry on the sampled packets. However, the head 357 nodes have no clue on the proper sampling rate. An overly high rate 358 would exhaust the network resource and even cause packet drops; An 359 overly low rate, on the contrary, would result in the loss of 360 information and inaccuracy of measurements. 362 We can use an adaptive approach based on the network conditions to 363 dynamically adjust the sampling rate. Every node gives user traffic 364 forwarding higher priority than telemetry data export. In case of 365 network congestion, the telemetry can sense some signals from the 366 data collected (e.g., deep buffer size, long delay, packet drop, and 367 data loss). The controller uses these signals to adjust the packet 368 sampling rate. In each adjustment period (i.e., RTT of the feedback 369 loop), the sampling rate is either decreased or increased in response 370 of the signals. We can use the AIMD policy similar to the TCP flow 371 control mechanism for the rate adjustment. 373 3.2. Smart Data Export 375 The flow telemetry data can catch the dynamics of the network and the 376 interactions between user traffic and network. Nevertheless, the 377 data inevitably contain redundancy. It is advisable to remove the 378 redundancy from the data in order to reduce the data transport 379 bandwidth and server processing load. 381 In addition to efficiently encode the export data (e.g., IPFIX 382 [RFC7011] or protobuf [1]), iFIT can also cache the data and send the 383 accumulated data in batch if the data is not time sensitive. Various 384 deduplication and compression techniques can be applied on the batch 385 data. 387 From the application perspective, an application may only be 388 interested in some special events which can be derived from the 389 telemetry data. For example, in case that the forwarding delay of a 390 packet exceeds a threshold or a flow changes its forwarding path is 391 of interest, it is unnecessary to send the original raw data to the 392 data collecting and processing servers. Rather, iFIT takes advantage 393 of the in-network computing capability of network devices to process 394 the raw data and only push the event notifications to the subscribing 395 applications. 397 3.2.1. Use Case: On-demand Anomaly Monitor 399 Network operators are interested in the anomalies such as path 400 change, network congestion, and packet drop. Such anomalies are 401 hidden in raw telemetry data (e.g., path trace, timestamp). We can 402 describe such anomalies as events and program them into the device 403 data plane. Only the triggered events are exported. For example, if 404 a new flow appears at any node, a path change event is triggered; if 405 the packet delay exceeds a predefined threshold in a node, the 406 congestion event is triggered; if a packet is dropped due to buffer 407 overflow, a packet drop event is triggered. 409 3.3. Dynamic Network Probe 411 Due to the limited data plane resource, it is unlikely one can 412 provide all the data all the time. On the other hand, the data 413 needed by applications may be arbitrary but ephemeral. It is 414 critical to meet the dynamic data requirements with limited resource. 416 Fortunately, data plane programmability allows iFIT to dynamically 417 load new data probes. These on-demand probes are called Dynamic 418 Network Probes (DNP) [I-D.song-opsawg-dnp4iq]. DNP is the technique 419 to enable probes for customized data collection in different network 420 planes. When working with IOAM or PBT, DNP is loaded to the data 421 plane through incremental programming or configuration. The DNP can 422 effectively conduct data generation, processing, and aggregation. 424 DNP introduces enough flexibility and extensibility to iFIT. It can 425 implement the optimizations for export data reduction motioned in the 426 previous section. It can also generate custom data as required by 427 today and tomorrow's applications. 429 The aforementioned sketch module and anomaly triggers can all be 430 implemented as DNPs so they can be loaded to or unloaded from the 431 data plane dynamically based on application requirments. 433 3.4. Encapsulation and Tunneling 435 Since MPLS and IPv4 network are still prevalent in carrier networks. 436 iFIT provides solutions to apply the on-path flow telemetry 437 techniques in such networks. PBT-M 438 [I-D.song-ippm-postcard-based-telemetry] does not introduce new 439 headers to the packets so the trouble of encapsulation for a new 440 header is avoided. In case a technique that requires a new header is 441 preferred, [I-D.song-mpls-extension-header] provides a means to 442 encapsulate the extra header using an MPLS extension header. As for 443 IPv4, it is possible to encapsulate the new header in an IP option. 444 For example, RAO [RFC2113] can be used to indicate the presence of 445 the new header. A recent proposal [I-D.herbert-ipv4-eh] that 446 introduces the IPv4 extension header may lead to a long term 447 solution. 449 In carrier networks, it is common for user traffic to traverse 450 various tunnels for QoS, traffic engineering, or security. iFIT 451 supports both the uniform mode and the pipe mode for tunnel support 452 as described in [I-D.song-ippm-ioam-tunnel-mode]. With such 453 flexibility, the operator can either gain a true end-to-end 454 visibility or apply a hierarchical approach which isolates the 455 monitoring domain between customer and provider. 457 3.5. On-demand Technique Selection and Integration 459 With multiple underlying data collection and export techniques at its 460 disposal, iFIT can flexibly adapt to different network conditions and 461 different application requirements. 463 For example, depending on the types of data that are of interest, 464 iFIT may choose either IOAM or PBT to collect the data; if an 465 application needs to track down where the packets are lost, it may 466 switch from IOAM to PBT. 468 iFIT can further integrate multiple data plane monitoring and 469 measurement techniques together and present a comprehensive data 470 plane telemetry solution to network operating applications. 472 3.6. iFIT Closed-Loop Architecture 474 Working together, the aforementioned components form a closed-loop 475 for complete iFIT applications, as shown in Figure 2. 477 +---------------------+ 478 | | 479 +------+ iFIT Applications |<------+ 480 | | | | 481 | +---------------------+ | 482 | Technique Selection | 483 | and Integration | 484 | | 485 |Smart Flow Smart | 486 |and Data closed-loop Data | 487 |Selection Export| 488 | | 489 | +----+----+ 490 V +---------+| 491 +----------+ Encapsulation +---------+|| 492 | iFIT | and Tunneling | iFIT ||| 493 | Head |----------------------->| ||+ 494 | Node | | Nodes |+ 495 +----------+ +---------+ 496 DNP DNP 498 Figure 2: iFIT Closed-Loop Architecture 500 An iFIT application would pick a suite of telemetry techniques based 501 on its requirements and apply an initial technique to the data plane. 502 It then configures the iFIT head nodes to decide the initial target 503 flows/packets and telemetry data set, the encapsulation and tunneling 504 scheme based on the underlying network architecture, and the iFIT- 505 capable nodes to decide the initial telemetry data export policy. 506 Based on the network condition and the analysis results of the 507 telemetry data, the iFIT application can change the telemetry 508 technique, the flow/data selection policy, and the data export 509 approach in realtime without breaking the normal network operation. 510 Many of such dynamic changes can be done through loading and 511 unloading DNPs. 513 We should avoid confusion between this closed telemetry loop and the 514 closed control loop. The latter term is often used in the context of 515 network automation. In such a closed control loop, telemetry also 516 plays an important role. Based on the telemetry results, 517 applications can automatically change the network policy or 518 configuration. In such a context, iFIT is just a part of the loop. 520 4. Intelligent Closed-Loop Network Telemetry Applications 522 The closed-loop nature of the iFIT framework allows numerous new 523 applications which enable future network operation architecture. 525 In general, it is resource consuming to monitor continuously all the 526 flows and all the paths of the network. So a flexible and dynamic 527 performance monitoring approach is desired. Some concepts like the 528 Interactive Query with Dynamic Network Probes, as previously 529 described, go in this direction. 531 Another example is shown in [I-D.ietf-ippm-multipoint-alt-mark], 532 which that describes an intelligent performance management based on 533 the network condition. The idea is to split the monitoring network 534 into Clusters. The Cluster partition that can be applied to every 535 type of network graph and the possibility to combine Clusters at 536 different levels enable the so called Network Zooming. It allows a 537 Controller to calibrate the network telemetry, so that it can start 538 without examining in depth and monitor the network as a whole. In 539 case of necessity (packet loss or too high delay), an immediate 540 detailed analysis can be reconfigured. In particular, the 541 Controller, that is aware of the network topology, can set up the 542 most suited Cluster partition by changing the traffic filter or 543 activate new measurement points and the problem can be localized with 544 a step-by-step process. 546 To apply this mechanism an iFIT application on top of the controllers 547 can manage and the iFIT closed loop allows its dynamic and flexible 548 operation. 550 5. Summary and Future Work 552 iFIT is a framework for applying on-path data plane telemetry 553 techniques. Combining with algorithmic and architectural schemes 554 that fit into the framework components, iFIT framework enables a 555 practical telemetry solution based on two basic on-path traffic data 556 collection modes: passport and postcard. 558 The operation of iFIT differs from both active OAM and passive OAM as 559 defined in [RFC7799]. It does not generate any active probe packets 560 or passively observe unmodified user packets. Instead, it modifies 561 selected user packets to collect useful information about them. 562 Therefore, the iFIT operation can be considered the hybrid type III 563 mode, which can provide more flexible and accurate network OAM. 565 More challenges and corresponding solutions for iFIT may need to be 566 covered. For example, how iFIT can fit in the big picture of 567 autonomous networking and support closed control loops. A complete 568 iFIT framework should also consider the cross-domain operations. We 569 leave these topics for future revisions. 571 6. Security Considerations 573 No specific security issues are identified other than those have been 574 discussed in the drafts on on-path flow information telemetry. 576 7. IANA Considerations 578 This document includes no request to IANA. 580 8. Contributors 582 Other major contributors of this document include Giuseppe Fioccola 583 and Daniel King. 585 9. Acknowledgments 587 We thank Shwetha Bhandari and Joe Clarke for their constructive 588 suggestions for improving this document. 590 10. References 592 10.1. Normative References 594 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 595 Requirement Levels", BCP 14, RFC 2119, 596 DOI 10.17487/RFC2119, March 1997, 597 . 599 [RFC7799] Morton, A., "Active and Passive Metrics and Methods (with 600 Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799, 601 May 2016, . 603 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 604 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 605 May 2017, . 607 10.2. Informative References 609 [CMSketch] 610 Cormode, G. and S. Muthukrishnan, "An improved data stream 611 summary: the count-min sketch and its applications", 2005, 612 . 614 [I-D.brockners-inband-oam-data] 615 Brockners, F., Bhandari, S., Pignataro, C., Gredler, H., 616 Leddy, J., Youell, S., Mizrahi, T., Mozes, D., Lapukhov, 617 P., Chang, R., and d. daniel.bernier@bell.ca, "Data Fields 618 for In-situ OAM", draft-brockners-inband-oam-data-07 (work 619 in progress), July 2017. 621 [I-D.herbert-ipv4-eh] 622 Herbert, T., "IPv4 Extension Headers and Flow Label", 623 draft-herbert-ipv4-eh-01 (work in progress), May 2019. 625 [I-D.ietf-ippm-multipoint-alt-mark] 626 Fioccola, G., Cociglio, M., Sapio, A., and R. Sisto, 627 "Multipoint Alternate Marking method for passive and 628 hybrid performance monitoring", draft-ietf-ippm- 629 multipoint-alt-mark-02 (work in progress), July 2019. 631 [I-D.kumar-ippm-ifa] 632 Kumar, J., Anubolu, S., Lemon, J., Manur, R., Holbrook, 633 H., Ghanwani, A., Cai, D., Ou, H., and L. Yizhou, "Inband 634 Flow Analyzer", draft-kumar-ippm-ifa-01 (work in 635 progress), February 2019. 637 [I-D.mirsky-ippm-hybrid-two-step] 638 Mirsky, G., Lingqiang, W., and G. Zhui, "Hybrid Two-Step 639 Performance Measurement Method", draft-mirsky-ippm-hybrid- 640 two-step-04 (work in progress), October 2019. 642 [I-D.song-ippm-ioam-data-validation-option] 643 Song, H. and T. Zhou, "In-situ OAM Data Validation 644 Option", draft-song-ippm-ioam-data-validation-option-02 645 (work in progress), April 2018. 647 [I-D.song-ippm-ioam-tunnel-mode] 648 Song, H., Li, Z., Zhou, T., and Z. Wang, "In-situ OAM 649 Processing in Tunnels", draft-song-ippm-ioam-tunnel- 650 mode-00 (work in progress), June 2018. 652 [I-D.song-ippm-postcard-based-telemetry] 653 Song, H., Zhou, T., Li, Z., Shin, J., and K. Lee, 654 "Postcard-based On-Path Flow Data Telemetry", draft-song- 655 ippm-postcard-based-telemetry-05 (work in progress), 656 September 2019. 658 [I-D.song-mpls-extension-header] 659 Song, H., Li, Z., Zhou, T., and L. Andersson, "MPLS 660 Extension Header", draft-song-mpls-extension-header-02 661 (work in progress), February 2019. 663 [I-D.song-opsawg-dnp4iq] 664 Song, H. and J. Gong, "Requirements for Interactive Query 665 with Dynamic Network Probes", draft-song-opsawg-dnp4iq-01 666 (work in progress), June 2017. 668 [I-D.zhou-ippm-enhanced-alternate-marking] 669 Zhou, T., Fioccola, G., Li, Z., Lee, S., Cociglio, M., and 670 Z. Li, "Enhanced Alternate Marking Method", draft-zhou- 671 ippm-enhanced-alternate-marking-03 (work in progress), 672 July 2019. 674 [passport-postcard] 675 Handigol, N., Heller, B., Jeyakumar, V., Mazieres, D., and 676 N. McKeown, "Where is the debugger for my software-defined 677 network?", 2012, 678 . 680 [RFC2113] Katz, D., "IP Router Alert Option", RFC 2113, 681 DOI 10.17487/RFC2113, February 1997, 682 . 684 [RFC7011] Claise, B., Ed., Trammell, B., Ed., and P. Aitken, 685 "Specification of the IP Flow Information Export (IPFIX) 686 Protocol for the Exchange of Flow Information", STD 77, 687 RFC 7011, DOI 10.17487/RFC7011, September 2013, 688 . 690 [RFC8321] Fioccola, G., Ed., Capello, A., Cociglio, M., Castaldelli, 691 L., Chen, M., Zheng, L., Mirsky, G., and T. Mizrahi, 692 "Alternate-Marking Method for Passive and Hybrid 693 Performance Monitoring", RFC 8321, DOI 10.17487/RFC8321, 694 January 2018, . 696 10.3. URIs 698 [1] https://developers.google.com/protocol-buffers/ 700 Authors' Addresses 702 Haoyu Song (editor) 703 Futurewei 704 2330 Central Expressway 705 Santa Clara 706 USA 708 Email: haoyu.song@futurewei.com 709 Zhenbin Li 710 Huawei 711 156 Beiqing Road 712 Beijing, 100095 713 P.R. China 715 Email: lizhenbin@huawei.com 717 Tianran Zhou 718 Huawei 719 156 Beiqing Road 720 Beijing, 100095 721 P.R. China 723 Email: zhoutianran@huawei.com 725 Fengwei Qin 726 China Mobile 727 No. 32 Xuanwumenxi Ave., Xicheng District 728 Beijing, 100032 729 P.R. China 731 Email: qinfengwei@chinamobile.com 733 Huanan Chen 734 China Telecom 735 P. R. China 737 Email: chenhuan6@chinatelecom.cn 739 Jaewhan Jin 740 LG U+ 741 South Korea 743 Email: daenamu1@lguplus.co.kr 745 Jongyoon Shin 746 SK Telecom 747 South Korea 749 Email: jongyoon.shin@sk.com