idnits 2.17.1 draft-song-ippm-postcard-based-telemetry-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an Introduction section. ** There are 4 instances of too long lines in the document, the longest one being 3 characters in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 270 has weird spacing: '...stcards gen p...' == Line 459 has weird spacing: '...stcards gen p...' -- The document date (October 20, 2018) is 2013 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: 'I-D.clemm-netconf-push-smart-filters-ps' is defined on line 598, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-netconf-udp-pub-channel' is defined on line 618, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-netconf-yang-push' is defined on line 623, but no explicit reference was found in the text == Unused Reference: 'I-D.sambo-netmod-yang-fsm' is defined on line 641, but no explicit reference was found in the text == Unused Reference: 'I-D.song-ippm-ioam-tunnel-mode' is defined on line 652, but no explicit reference was found in the text == Unused Reference: 'I-D.talwar-rtgwg-grpc-use-cases' is defined on line 667, but no explicit reference was found in the text == Unused Reference: 'RFC6241' is defined on line 685, but no explicit reference was found in the text == Outdated reference: A later version (-05) exists of draft-brockners-ippm-ioam-geneve-01 == Outdated reference: A later version (-17) exists of draft-ietf-ippm-ioam-data-00 == Outdated reference: A later version (-05) exists of draft-ietf-netconf-udp-pub-channel-01 == Outdated reference: A later version (-25) exists of draft-ietf-netconf-yang-push-12 == Outdated reference: A later version (-13) exists of draft-ietf-sfc-ioam-nsh-00 == Outdated reference: A later version (-05) exists of draft-sambo-netmod-yang-fsm-00 == Outdated reference: A later version (-01) exists of draft-song-ippm-ioam-data-extension-00 == Outdated reference: A later version (-13) exists of draft-song-mpls-extension-header-01 -- Obsolete informational reference (is this intentional?): RFC 2925 (Obsoleted by RFC 4560) Summary: 2 errors (**), 0 flaws (~~), 18 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 IPPM H. Song, Ed. 3 Internet-Draft T. Zhou 4 Intended status: Standards Track Z. Li 5 Expires: April 23, 2019 Huawei 6 October 20, 2018 8 Export User Flow Telemetry Data by Postcard Packets 9 draft-song-ippm-postcard-based-telemetry-00 11 Abstract 13 This document describes a proposal which allows network OAM 14 applications to collect telemetry data about any user packet. Unlike 15 similar techniques such as INT and in-situ OAM, the Postcard-Based 16 Telemetry (PBT) does not require inserting telemetry data into user 17 packets, but directly exports the telemetry data to a collector 18 through separated OAM packets called postcards. Two variations of 19 PBT are described in this document: one requires inserting an 20 instruction header to user packets to guide the data collection and 21 the other only marks the user packets to invoke the data collection. 22 Each has its own pros and cons. Either way, the postcards for a 23 single user packet are from multiple devices and need to be 24 correlated at the collector, which is a unique issue pertaining to 25 PBT. Whereas, PBT provides an alternative to INT and address several 26 implementation and deployment challenges of the INT-based solutions. 28 Status of This Memo 30 This Internet-Draft is submitted in full conformance with the 31 provisions of BCP 78 and BCP 79. 33 Internet-Drafts are working documents of the Internet Engineering 34 Task Force (IETF). Note that other groups may also distribute 35 working documents as Internet-Drafts. The list of current Internet- 36 Drafts is at https://datatracker.ietf.org/drafts/current/. 38 Internet-Drafts are draft documents valid for a maximum of six months 39 and may be updated, replaced, or obsoleted by other documents at any 40 time. It is inappropriate to use Internet-Drafts as reference 41 material or to cite them other than as "work in progress." 43 This Internet-Draft will expire on April 23, 2019. 45 Copyright Notice 47 Copyright (c) 2018 IETF Trust and the persons identified as the 48 document authors. All rights reserved. 50 This document is subject to BCP 78 and the IETF Trust's Legal 51 Provisions Relating to IETF Documents 52 (https://trustee.ietf.org/license-info) in effect on the date of 53 publication of this document. Please review these documents 54 carefully, as they describe your rights and restrictions with respect 55 to this document. Code Components extracted from this document must 56 include Simplified BSD License text as described in Section 4.e of 57 the Trust Legal Provisions and are provided without warranty as 58 described in the Simplified BSD License. 60 Table of Contents 62 1. Motivation . . . . . . . . . . . . . . . . . . . . . . . . . 2 63 2. PBT-M: Postcard-based Telemetry with Packet Marking . . . . . 4 64 2.1. New Requirements . . . . . . . . . . . . . . . . . . . . 5 65 2.2. Solution Description . . . . . . . . . . . . . . . . . . 6 66 2.3. New Challenges . . . . . . . . . . . . . . . . . . . . . 7 67 2.4. Considerations on PBT-M Design . . . . . . . . . . . . . 7 68 2.4.1. Packet Marking . . . . . . . . . . . . . . . . . . . 7 69 2.4.2. Flow Path Discovery . . . . . . . . . . . . . . . . . 8 70 2.4.3. Packet Identity for Export Data Correlation . . . . . 8 71 3. PBT-I: Postcard-based Telemetry with Instruction Header . . . 9 72 3.1. Solution Description . . . . . . . . . . . . . . . . . . 10 73 3.2. PBT-I Telemetry Instruction Header . . . . . . . . . . . 11 74 3.3. Considerations on PBT-I Design . . . . . . . . . . . . . 12 75 4. Security Considerations . . . . . . . . . . . . . . . . . . . 12 76 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 12 77 6. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 13 78 7. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 13 79 8. Informative References . . . . . . . . . . . . . . . . . . . 13 80 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 16 82 1. Motivation 84 In order to gain detailed data plane visibility to support effective 85 network OAM, it is important to be able to examine the trace of user 86 packets along their forwarding paths. Such path-associated data 87 reflect the state and status of each user packet's real-time 88 experience and provide valuable information for network monitoring, 89 measurement, and diagnosis. 91 The telemetry data include the detailed forwarding path, the 92 timestamp/latency at each network node, and other information. The 93 emerging programmable data plane devices allow more sophisticated 94 data to be retrieved [I-D.song-opsawg-dnp4iq]. Such path-associated 95 flow data can only be derived from the live user packets. The data 96 complement with the other data acquired through passive and active 97 OAM mechanisms such as IPFIX [RFC7011] and ICMP [RFC2925]. 99 In-band Network Telemetry (INT) was designed to cater exactly this 100 need. in-situ OAM (iOAM) [I-D.brockners-inband-oam-requirements] 101 represents the related standardization efforts. Essentially, INT 102 augments user packets with instructions to tell each network node on 103 the forwarding paths which data to collect. The requested data are 104 inserted into and travel along with the user packets. The path-end 105 node strips off the data trace and exports it to a data collector for 106 processing. 108 While the concept is simple and straightforward, INT faces several 109 technical challenges: 111 o Issue 1: INT header and data processing needs to be done in data 112 plane fast path. It may interfere with the normal traffic 113 forwarding (e.g., leading to forwarding performance degradation) 114 and lead to inaccurate measurements (e.g., resulting in longer 115 latency measurements than usual). This undesirable "observer 116 effect" is problematic to carrier networks where stringent SLA 117 must be observed. 119 o Issue 2: INT may significantly increase the user packet's original 120 size by adding the instruction header and data at each traversed 121 node. The longer the forwarding path and the more the data 122 collected, the larger the packet will become. The size may exceed 123 the path MTU so either INT cannot apply or the packet needs to be 124 fragmented. Limiting the data size or path length reduces the 125 effectiveness of INT. On the other hand, the INT header and data 126 can be deeply embedded in a packet due to various transport 127 protocol and tunnel configurations. The required deep packet 128 header inspection and processing may be infeasible to some data 129 plane fast path where only a limited number of header bytes are 130 accessible. 132 o Issue 3: INT requires attaching an instruction header to user 133 packets to inform network nodes what types of data to collect. 134 Due to the header overhead constraint and hardware-friendly 135 consideration, TLV is undesirable for data type encoding. 136 Instead, iOAM use a bitmap where each bit indicates one pre- 137 defined data type [I-D.ietf-ippm-ioam-data]. However, new use 138 cases may require new data types. The current allocated 16-bit 139 bitmap limits the data type scalability. The proposed bitmap 140 extension in [I-D.song-ippm-ioam-data-extension] provides a method 141 to support more data types but it also increases the iOAM header 142 size. 144 o Issue 4: INT header need to be encapsulated into user packets for 145 transport. [I-D.brockners-inband-oam-transport] has discussed 146 several encapsulation approaches for different transport 147 protocols. However, it is difficult to encapsulate extra header 148 in MPLS and IPv4 networks which happens to be the most widely 149 deployed and where the path-associated telemetry data is most 150 wanted by operators. The proposed NVGRE encapsulation for IPv4 in 151 [I-D.brockners-inband-oam-transport] requires a tunnel to be built 152 between each pair of nodes which may be unrealistic for plain IP 153 networks. 155 o Issue 5: The INT header and data are vulnerable to eavesdropping 156 and tampering as well as DoS attack. Extra protective measurement 157 is difficult on the fast data path. 159 o Issue 6: Since INT only exports the telemetry data at the 160 designated end node, if the packet is dropped in the network, the 161 data will be lost. It cannot pinpoint the packet drop location 162 which is required for fault diagnosis. 164 The above issues are inherent to the INT-based solutions. 165 Nevertheless, the path-associated data acquired by INT are valuable 166 for network operators. Therefore, alternative approaches which can 167 collect the same data but avoid or mitigate the above issues are 168 desired. This document provides a new proposal named Postcard-Based 169 Telemetry (PBT) with two different implementation variations, each 170 having its own trade-off and addressing some or all of the above 171 issues. The basic idea of PBT is simple: at each node, instead of 172 inserting the collected data into the user packets, the data are 173 directly exported through dedicated OAM packets. Such "postcard" 174 approach is in contrast to the "passport stamps" approach adopted by 175 INT [DOI_10.1145_2342441.2342453]. The OAM packets or postcards can 176 be transported in band or out of band, independent of the original 177 user packets. 179 2. PBT-M: Postcard-based Telemetry with Packet Marking 181 This section describes the first variation of PBT. PBT-M aims to 182 address all the challenges of INT listed above and introduce some new 183 benefits. We first list all the design requirements of PBT-M. 185 2.1. New Requirements 187 o Req. 1: We should avoid augmenting user packets with new headers 188 or introducing new data plane protocols. This helps to alleviate 189 or eliminate the issue 1, 2, 4, and 5. We expect the OAM data 190 collecting signaling remains in data plane. Simple packet marking 191 techniques suffice to serve this purpose. It is also possible to 192 configure the OAM data collecting from the control plane. 194 o Req. 2: We should make the scheme extensible for collecting 195 arbitrary new data to support possible future use cases. The data 196 set to be collected is preferred to be configured through 197 management plane or control plane. Since there is no limitation 198 on the types of data, any data including those generated by 199 customized DNPs [I-D.song-opsawg-dnp4iq] can be collected. Since 200 there is no size constraints any more, it is free to use the more 201 flexible TLV for data type definition. This addresses the issue 2 202 and 3. 204 o Req. 3: We should avoid interfering the normal forwarding and 205 affecting the forwarding performance when conducting data plane 206 OAM tasks. Hence, the collected data are better to be transported 207 independently by dedicated OAM packets through in-band or out-of- 208 band channels. The data collecting, processing, assembly, 209 encapsulation, and transport are therefore decoupled from the 210 forwarding of the corresponding user packets and can be performed 211 in data plane slow path if necessary. This addresses the issue 1, 212 4, and 5. 214 o Req. 4: The data collected from each node is not necessarily 215 identical, depending on application requirements and node 216 capability. Data for different operation modes can be collected 217 at the same time. These requirements are either impossible or 218 very difficult to be supported by INT in which data types 219 collected per node are supposed to be identical and for a single 220 mode. 222 o Req. 5: The flow's path-associated data can be sensitive and the 223 security concerns need to be carefully addressed. Sending OAM 224 data with independent packets also makes it easy to secure the 225 collected data without exposing it to unnecessary entities. For 226 example, the data can be encrypted before being sent to the 227 collector so passive eavesdropping and man-in-the-middle attack 228 can both be deterred. This addresses the issue 5. 230 o Req. 6: Even if a user packet under inspection is dropped in 231 network, the OAM data that have been collected should still be 232 exported and help to diagnose the packet drop location and reason. 233 This addresses the issue 6. 235 2.2. Solution Description 237 In light of the above discussion, the sketch of the proposed 238 solution, PBT-M, is as follows. The user packet, if its path- 239 associated data need to be collected, is marked at the path head 240 node. At each PBT-aware node, a postcard (i.e., the dedicated OAM 241 packet triggered by a marked user packet) is generated and sent to a 242 collector. The postcard contains the data requested by the 243 management plane. The requested data are configured by the 244 management plane through data templates (as in IPFIX [RFC7011]) or 245 other means. Once the collector receives all the postcards for a 246 single user packet, it can infer the packet's forwarding path and 247 analyze the data set. The path end node is configured to unmark the 248 packets to its original format if necessary. 250 The overall architecture of PBT-M is depict in Figure 1. 252 +------------+ +-----------+ 253 | Network | | Telemetry | 254 | Management |(-------| Data | 255 | | | Collector | 256 +-----:------+ +-----------+ 257 : ^ 258 :configurations |postcards (OAM pkts) 259 : | 260 ...............:.....................|........ 261 : : : | : 262 : +---------:---+-----------:---+--+-------:---+ 263 : | : | : | : | 264 V | V | V | V | 265 +------+-+ +-----+--+ +------+-+ +------+-+ 266 usr pkts | Head | | Path | | Path | | End | 267 ====>| Node |====>| Node |====>| Node |====>| Node |====> 268 | | | A | | B | | | 269 +--------+ +--------+ +--------+ +--------+ 270 gen postcards gen postcards gen postcards gen postcards 271 mark usr pkts unmark usr pkts 273 Figure 1: Architecture of PBT-M 275 Next we discuss the details of the PBT-M solution and the potential 276 standard gaps. 278 2.3. New Challenges 280 Although PBT-M solves the issues of INT, it does introduce a few new 281 challenges. 283 o Challenge 1: A user packet needs to be marked in order to trigger 284 the path-associated data collection. Since we do not want to 285 augment user packets with any new header fields (i.e., Req. 1), we 286 must take advantage of the existing header fields. 288 o Challenge 2: Since the packet header will not carry OAM 289 instructions any more, the data plane devices need to be 290 configured to know what data to collect. However, in general, the 291 forwarding path of a flow packet (due to ECMP or dynamic routing) 292 is unknown beforehand. Configuring the data set for each flow at 293 all data plane devices is expensive in terms of configuration load 294 and data plane resources. 296 o Challenge 3: Due to the variable transport latency, the dedicated 297 OAM packets for a single packet may arrive at the collector out of 298 order or be dropped in networks for some reason. In order to 299 infer the packet forwarding path, the collector needs some 300 information from the OAM packets to identify the user packet 301 affiliation and the order of path node traversal. 303 2.4. Considerations on PBT-M Design 305 To address the above challenges, we propose several design details of 306 PBT-M. 308 2.4.1. Packet Marking 310 Instead of stuffing new header fields into user packets, it is 311 preferred to reuse some existing header fields. To trigger the path- 312 associated data collection, usually a single bit is sufficient. 313 While no such bit is available, other packet marking techniques are 314 needed. we discuss three possible application scenarios. 316 o IPv4. IPFPM [I-D.ietf-ippm-alt-mark] is an IP flow performance 317 measurement framework which also requires a single bit for packet 318 coloring. The difference is that IPFPM does in-network 319 measurement while PBT only collects and exports data at network 320 nodes (i.e., the data analysis is done at the collector). IPFPM 321 suggests to use some reserved bit of the Flag field or some unused 322 bit of the TOS field. Actually, IPFPM can be considered a subcase 323 of PBT so the same bit can be used for PBT. The management plane 324 is responsible to configure the actual operation mode. 326 o SFC NSH. The OAM bit in NSH header can be used to trigger the 327 path-associated data collection [I-D.ietf-sfc-nsh]. PBT does not 328 add any other metadata to NSH. 330 o MPLS. Instead of choosing a header bit, we take advantage of the 331 synonymous flow label [I-D.bryant-mpls-synonymous-flow-labels] 332 approach to mark the packets. A synonymous flow label indicates 333 the path-associated data should be collected and forwarded through 334 a postcard. 336 2.4.2. Flow Path Discovery 338 By default, all PBT-aware nodes are configured to react to the marked 339 packets by exporting some basic data such as node ID and TTL before a 340 data set template for that flow is configured. This way, the 341 management plane can learn the flow path. 343 If the management plane wants to collect the path-associated data for 344 some flow, it configures the head node(s) with a probability or time 345 interval for the flow packet marking. When the first marked packet 346 is forwarded in the network, the PBT-aware nodes will export the 347 basic data to the collector. Hence, the flow path is identified. If 348 other types of data need to be collected, the management plane can 349 further configure the data set template to the target nodes. The 350 PBT-aware nodes would collect and export data accordingly if the 351 packet is marked and a data set template is present. 353 If for any reason, the flow path is changed. The new path nodes can 354 be learnt immediately by the collector, so the management plane 355 controller can be informed to configure the new path nodes. The 356 outdated configuration can be automatically timed out or explicitly 357 revoked by the management plane controller. 359 2.4.3. Packet Identity for Export Data Correlation 361 The collector needs to correlate all the OAM packets for a single 362 user packet. Once this is done, the TTL (or the timestamp, if the 363 network time is synchronized) can be used to infer the flow 364 forwarding path. The key issue here is to uniquely identify the user 365 packet affiliation of the OAM packet. 367 The first possible approach is to include the flow ID plus the user 368 packet ID in the OAM packets. The user packet ID can be some unique 369 information pertaining to a user packet (e.g., the sequence number of 370 a TCP packet). 372 If the packet marking interval is long enough, then the flow ID 373 itself is enough to identify the user packet. That is, we can assume 374 all the exported OAM packets for the same flow during a short period 375 of time belong to the same user packet. 377 If the network is synchronized, then the flow ID plus the timestamp 378 at each node can also infer the packet identity. However, some 379 errors may occur under some circumstances. For example, if two 380 consecutive user packets from the same flows are both marked and one 381 exported OAM packet from a node is lost, then it is difficult for the 382 collector to decide which user packet the remaining OAM packet 383 belongs to. In many cases, such rare errors may be tolerable. 385 3. PBT-I: Postcard-based Telemetry with Instruction Header 387 Since PBT-M has some challenges as listed in Section 2.3, this 388 section describes another variation of PBT, which essentially 389 compromises some of the design requirements listed in Section 2.1, 390 yet retains most of the benefits of PBT. 392 PBT-I can be seen as a trade-off between INT/iOAM and PBT-M. PBT-I 393 needs to add a fixed length instruction header to user packets for 394 OAM data collection. However, the collected data will be exported 395 through dedicated OAM packets. On the one hand, PBT-I violates the 396 Req. 1 in Section 2.1. It also makes it harder to meet the Req. 2. 397 On the other hand, the overhead of the instruction header is under 398 control and user packets will not inflate with path length or 399 telemetry data amount. We also introduce an optimization to mitigate 400 the impact on Req. 2. In return, PBT-I addresses all the challenges 401 of PBT-M: 403 o There is no need to find an existing header field to mark a user 404 packet. The encapsulation of the PBT-I instruction header can use 405 the same method for iOAM. So far, the iOAM header encapsulation 406 methods have been defined for several protocols, including IPv6, 407 VXLAN-GPE, NSH, SRv6 408 [I-D.brockners-inband-oam-transport],[I-D.ietf-sfc-ioam-nsh], 409 GENEVE [I-D.brockners-ippm-ioam-geneve], and GRE 410 [I-D.weis-ippm-ioam-gre]. [I-D.song-mpls-extension-header] 411 describes the approach to encapuslate the instruction header into 412 MPLS packets. 414 o There is no need to configure the node about the set of data to be 415 collected since all the information is carried in the instruction 416 header. We adopt the optimization in 417 [I-D.song-ippm-ioam-data-extension] to address the data 418 scalability issue incurred by the instruction header. 420 o The instruction header contains enough information to help 421 correlate the OAM packets belonging to a user packets. Even 422 better, new fields are added to track the flow and the packet, so 423 any packet under inspection can be easily identified even in 424 tunnels and the collector can easily check if any user packet 425 under inspection or its OAM data packet is missing. 427 3.1. Solution Description 429 The sketch of the proposed solution, PBT-I, is as follows. If the 430 path-associated data need to be collected for a user packet, an 431 instruction header named Telemetry Instruction Header (TIH) is 432 inserted into the packet at the path head node. At each PBT-aware 433 node, a postcard is generated and sent to a collector. Once the 434 collector receives all the postcards for a single user packet, it can 435 combine and analyze the data set. The path end node is configured to 436 remove the TIH. 438 The overall architecture of PBT-I is depict in Figure 2. 440 +-----------+ 441 | Telemetry | 442 | Data | 443 | Collector | 444 +-----------+ 445 ^ 446 |postcards (OAM pkts) 447 | 448 | 449 | 450 +--------------+------+-------+--------------+ 451 | | | | 452 | | | | 453 +---+----+ +---+----+ +---+----+ +---+----+ 454 usr pkts | Head | | Path | | Path | | End | 455 ====>| Node |====>| Node |====>| Node |====>| Node |====> 456 | | | A | | B | | | 457 +--------+ +--------+ +--------+ +--------+ 458 insert TIH remove TIH 459 gen postcards gen postcards gen postcards gen postcards 461 Figure 2: Architecture of PBT-I 463 3.2. PBT-I Telemetry Instruction Header 465 The proposed format of TIH is shown in Figure 3. 467 0 0 0 1 1 2 2 3 468 0 7 8 5 6 3 4 1 469 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 470 | Next Header | TIH Length | Reserved | Hop Count | 471 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 472 | Flow ID | 473 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 474 | Flow ID | 475 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 476 | Sequence Number | 477 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 478 | Data Element Bitmap |E| 479 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 480 | Data Element Bitmap Extension (optional) | 481 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 483 Figure 3: TIH Format 485 o Next Header: the 8-bit indicator indicating the next protocol 486 after TIH. 488 o TIH Length: the 8-bit Instruction Header Length field. The value 489 is in the unit of 4-octet words. 491 o Hop Count: the 8-bit Hop Count field. It is used to count the 492 hops of the TIH-aware nodes, starting for 0 and incremented by 1 493 at each PBT-aware node. 495 o Flow ID: the 64-bit flow ID field. If the actual flow ID is 496 shorter than 64 bits, it is right aligned with the leading bits 497 being filled with 0. The field is set at the head node. 499 o Sequence Number: the 32-bit sequence number starting from 0 and 500 increasing by 1 for each following monitored packet from the same 501 flow at the head node. 503 o Data Element Bitmap: the 31-bit bitmap indicating the list of 504 required data elements. 506 o E bit: Data Extension Indicator. This bit has a default value of 507 0. If set, it indicates a following data element bitmap extension 508 word, as described in [I-D.song-ippm-ioam-data-extension]. 510 The specification of the data element bitmap is as follows: 512 o Bit 0: Node ID (32b) 514 o Bit 1: TTL (8b), derived from the TTL field in the packet header 516 o Bit 2: Ingress Interface ID (32b) 518 o Bit 3: Egress Interface ID (32b) 520 o Bit 4: Ingress Timestamp (64b) 522 o Bit 5: Egress Timestamp (64b) 524 o Bit 6: Transit Delay (32b) 526 o Bit 7: Egress Queue Depth (32b) 528 o Bit 8: Ingress MTU (32b) 530 o Bit 9: Egress MTU (32b) 532 o Bit 10: Egress Packet Length (32b) 534 o Bit 11-30: Undefined 536 3.3. Considerations on PBT-I Design 538 4. Security Considerations 540 Several security issues need to be considered. 542 o Eavesdrop and tamper: the OAM packets can be encrypted and 543 authenticated. 545 o DoS attack: PBT can be limited to a single administration domain, 546 or the enforce mark or instruction header are checked at the 547 domain edge. The node can rate limit the extra traffic incurred 548 by the OAM data. 550 5. IANA Considerations 552 TBD. 554 6. Contributors 556 TBD. 558 7. Acknowledgments 560 TBD. 562 8. Informative References 564 [DOI_10.1145_2342441.2342453] 565 Handigol, N., Heller, B., Jeyakumar, V., MaziA(C)res, D., 566 and N. McKeown, "Where is the debugger for my software- 567 defined network?", Proceedings of the first workshop on 568 Hot topics in software defined networks - HotSDN '12, 569 DOI 10.1145/2342441.2342453, 2012. 571 [I-D.brockners-inband-oam-requirements] 572 Brockners, F., Bhandari, S., Dara, S., Pignataro, C., 573 Gredler, H., Leddy, J., Youell, S., Mozes, D., Mizrahi, 574 T., <>, P., and r. remy@barefootnetworks.com, 575 "Requirements for In-situ OAM", draft-brockners-inband- 576 oam-requirements-03 (work in progress), March 2017. 578 [I-D.brockners-inband-oam-transport] 579 Brockners, F., Bhandari, S., Govindan, V., Pignataro, C., 580 Gredler, H., Leddy, J., Youell, S., Mizrahi, T., Mozes, 581 D., Lapukhov, P., and R. Chang, "Encapsulations for In- 582 situ OAM Data", draft-brockners-inband-oam-transport-05 583 (work in progress), July 2017. 585 [I-D.brockners-ippm-ioam-geneve] 586 Brockners, F., Bhandari, S., Govindan, V., Pignataro, C., 587 Gredler, H., Leddy, J., Youell, S., Mizrahi, T., Mozes, 588 D., Lapukhov, P., and R. Chang, "Geneve encapsulation for 589 In-situ OAM Data", draft-brockners-ippm-ioam-geneve-01 590 (work in progress), June 2018. 592 [I-D.bryant-mpls-synonymous-flow-labels] 593 Bryant, S., Swallow, G., Sivabalan, S., Mirsky, G., Chen, 594 M., and Z. Li, "RFC6374 Synonymous Flow Labels", draft- 595 bryant-mpls-synonymous-flow-labels-01 (work in progress), 596 July 2015. 598 [I-D.clemm-netconf-push-smart-filters-ps] 599 Clemm, A., Voit, E., Liu, X., Bryskin, I., Zhou, T., 600 Zheng, G., and H. Birkholz, "Smart filters for Push 601 Updates - Problem Statement", draft-clemm-netconf-push- 602 smart-filters-ps-00 (work in progress), October 2017. 604 [I-D.ietf-ippm-alt-mark] 605 Fioccola, G., Capello, A., Cociglio, M., Castaldelli, L., 606 Chen, M., Zheng, L., Mirsky, G., and T. Mizrahi, 607 "Alternate Marking method for passive and hybrid 608 performance monitoring", draft-ietf-ippm-alt-mark-14 (work 609 in progress), December 2017. 611 [I-D.ietf-ippm-ioam-data] 612 Brockners, F., Bhandari, S., Pignataro, C., Gredler, H., 613 Leddy, J., Youell, S., Mizrahi, T., Mozes, D., Lapukhov, 614 P., Chang, R., and d. daniel.bernier@bell.ca, "Data Fields 615 for In-situ OAM", draft-ietf-ippm-ioam-data-00 (work in 616 progress), September 2017. 618 [I-D.ietf-netconf-udp-pub-channel] 619 Zheng, G., Zhou, T., and A. Clemm, "UDP based Publication 620 Channel for Streaming Telemetry", draft-ietf-netconf-udp- 621 pub-channel-01 (work in progress), November 2017. 623 [I-D.ietf-netconf-yang-push] 624 Clemm, A., Voit, E., Prieto, A., Tripathy, A., Nilsen- 625 Nygaard, E., Bierman, A., and B. Lengyel, "YANG Datastore 626 Subscription", draft-ietf-netconf-yang-push-12 (work in 627 progress), December 2017. 629 [I-D.ietf-sfc-ioam-nsh] 630 Brockners, F., Bhandari, S., Govindan, V., Pignataro, C., 631 Gredler, H., Leddy, J., Youell, S., Mizrahi, T., Mozes, 632 D., Lapukhov, P., and R. Chang, "NSH Encapsulation for In- 633 situ OAM Data", draft-ietf-sfc-ioam-nsh-00 (work in 634 progress), May 2018. 636 [I-D.ietf-sfc-nsh] 637 Quinn, P., Elzur, U., and C. Pignataro, "Network Service 638 Header (NSH)", draft-ietf-sfc-nsh-28 (work in progress), 639 November 2017. 641 [I-D.sambo-netmod-yang-fsm] 642 Sambo, N., Castoldi, P., Fioccola, G., Cugini, F., Song, 643 H., and T. Zhou, "YANG model for finite state machine", 644 draft-sambo-netmod-yang-fsm-00 (work in progress), October 645 2017. 647 [I-D.song-ippm-ioam-data-extension] 648 Song, H. and T. Zhou, "In-situ OAM Data Type Extension", 649 draft-song-ippm-ioam-data-extension-00 (work in progress), 650 October 2017. 652 [I-D.song-ippm-ioam-tunnel-mode] 653 Song, H., Li, Z., Zhou, T., and Z. Wang, "In-situ OAM 654 Processing in Tunnels", draft-song-ippm-ioam-tunnel- 655 mode-00 (work in progress), June 2018. 657 [I-D.song-mpls-extension-header] 658 Song, H., Li, Z., Zhou, T., and L. Andersson, "MPLS 659 Extension Header", draft-song-mpls-extension-header-01 660 (work in progress), August 2018. 662 [I-D.song-opsawg-dnp4iq] 663 Song, H. and J. Gong, "Requirements for Interactive Query 664 with Dynamic Network Probes", draft-song-opsawg-dnp4iq-01 665 (work in progress), June 2017. 667 [I-D.talwar-rtgwg-grpc-use-cases] 668 Specification, g., Kolhe, J., Shaikh, A., and J. George, 669 "Use cases for gRPC in network management", draft-talwar- 670 rtgwg-grpc-use-cases-01 (work in progress), January 2017. 672 [I-D.weis-ippm-ioam-gre] 673 Weis, B., Brockners, F., crhill@cisco.com, c., Bhandari, 674 S., Govindan, V., Pignataro, C., Gredler, H., Leddy, J., 675 Youell, S., Mizrahi, T., Kfir, A., Gafni, B., Lapukhov, 676 P., and M. Spiegel, "GRE Encapsulation for In-situ OAM 677 Data", draft-weis-ippm-ioam-gre-00 (work in progress), 678 March 2018. 680 [RFC2925] White, K., "Definitions of Managed Objects for Remote 681 Ping, Traceroute, and Lookup Operations", RFC 2925, 682 DOI 10.17487/RFC2925, September 2000, 683 . 685 [RFC6241] Enns, R., Ed., Bjorklund, M., Ed., Schoenwaelder, J., Ed., 686 and A. Bierman, Ed., "Network Configuration Protocol 687 (NETCONF)", RFC 6241, DOI 10.17487/RFC6241, June 2011, 688 . 690 [RFC7011] Claise, B., Ed., Trammell, B., Ed., and P. Aitken, 691 "Specification of the IP Flow Information Export (IPFIX) 692 Protocol for the Exchange of Flow Information", STD 77, 693 RFC 7011, DOI 10.17487/RFC7011, September 2013, 694 . 696 Authors' Addresses 698 Haoyu Song (editor) 699 Huawei 700 2330 Central Expressway 701 Santa Clara, 95050 702 USA 704 Email: haoyu.song@huawei.com 706 Tianran Zhou 707 Huawei 708 156 Beiqing Road 709 Beijing, 100095 710 P.R. China 712 Email: zhoutianran@huawei.com 714 Zhenbin Li 715 Huawei 716 156 Beiqing Road 717 Beijing, 100095 718 P.R. China 720 Email: lizhenbin@huawei.com