idnits 2.17.1 draft-claise-opsawg-collected-data-manifest-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 4 instances of too long lines in the document, the longest one being 64 characters in excess of 72. == There are 1 instance of lines with non-RFC2606-compliant FQDNs in the document. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 196 has weird spacing: '...mespace ine...' == Line 207 has weird spacing: '...mespace ine...' -- The document date (20 March 2022) is 762 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Outdated reference: A later version (-03) exists of draft-claise-netconf-metadata-for-collection-02 Summary: 1 error (**), 0 flaws (~~), 5 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 OPSAWG B. Claise 3 Internet-Draft J. Quilbeuf 4 Intended status: Standards Track Huawei 5 Expires: 21 September 2022 D. Lopez 6 Telefonica I+D 7 I. Dominguez 8 Universidad Politecnica de Madrid 9 T. Graf 10 Swisscom 11 20 March 2022 13 A Data Manifest for Contextualized Telemetry Data 14 draft-claise-opsawg-collected-data-manifest-02 16 Abstract 18 Most network equipment feature telemetry as a mean to monitoring 19 their status. Several protocols exist to this end, for example, the 20 model-driven telemetry governed by YANG models. Some of these 21 protocols provide the data itself, without any contextual information 22 about the collection method. This can render the data unusable if 23 that context is lost, for instance when the data is stored without 24 the relevant information. This document proposes a data manifest, 25 composed of two YANG data models, to store that contextual 26 information along with the collected data, in order to keep the 27 collected data exploitable in the future. 29 Status of This Memo 31 This Internet-Draft is submitted in full conformance with the 32 provisions of BCP 78 and BCP 79. 34 Internet-Drafts are working documents of the Internet Engineering 35 Task Force (IETF). Note that other groups may also distribute 36 working documents as Internet-Drafts. The list of current Internet- 37 Drafts is at https://datatracker.ietf.org/drafts/current/. 39 Internet-Drafts are draft documents valid for a maximum of six months 40 and may be updated, replaced, or obsoleted by other documents at any 41 time. It is inappropriate to use Internet-Drafts as reference 42 material or to cite them other than as "work in progress." 44 This Internet-Draft will expire on 21 September 2022. 46 Copyright Notice 48 Copyright (c) 2022 IETF Trust and the persons identified as the 49 document authors. All rights reserved. 51 This document is subject to BCP 78 and the IETF Trust's Legal 52 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 53 license-info) in effect on the date of publication of this document. 54 Please review these documents carefully, as they describe your rights 55 and restrictions with respect to this document. Code Components 56 extracted from this document must include Revised BSD License text as 57 described in Section 4.e of the Trust Legal Provisions and are 58 provided without warranty as described in the Revised BSD License. 60 Table of Contents 62 1. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 2 63 2. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 64 3. Platform Manifest . . . . . . . . . . . . . . . . . . . . . . 4 65 3.1. Overview of the model . . . . . . . . . . . . . . . . . . 4 66 3.2. YANG module ietf-collected-data-platform-manifest . . . . 6 67 4. Data Collection Manifest . . . . . . . . . . . . . . . . . . 9 68 4.1. Overview of the model . . . . . . . . . . . . . . . . . . 9 69 4.2. YANG module ietf-collected-data-manifest . . . . . . . . 10 70 5. Collecting Data Manifest and Mapping Data to Data Manifest . 13 71 5.1. Mapping Collected Data to the Data Manifest . . . . . . . 14 72 6. Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 73 7. Security Considerations . . . . . . . . . . . . . . . . . . . 16 74 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 16 75 9. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 16 76 10. Open Issues . . . . . . . . . . . . . . . . . . . . . . . . . 16 77 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 17 78 11.1. Normative References . . . . . . . . . . . . . . . . . . 17 79 11.2. Informative References . . . . . . . . . . . . . . . . . 17 80 Appendix A. Changes between revisions . . . . . . . . . . . . . 18 81 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . 19 82 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 19 84 1. Terminology 86 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 87 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 88 "OPTIONAL" in this document are to be interpreted as described in BCP 89 14 [RFC2119] [RFC8174] when, and only when, they appear in all 90 capitals, as shown here. 92 Data Manifest: all the necessary data required to interpret the 93 telemetry information. 95 Platform Manifest: part of the Data Manifest that completely 96 characterizes the platform producing the data. 98 Data Collection Manifest: part of the Data Manifest that completely 99 characterizes how and when the telemetry information was metered. 101 2. Introduction 103 Network elements use Model-driven Telemetry (MDT) to continuously 104 stream information, including both counters and state information. 105 This streamed information is used for network monitoring or closed- 106 loop automation. This streamed data can also be stored in a database 107 (sometimes called a big data lake) for further analysis. 109 When streaming YANG-structured data with YANG-Push [RFC8641], there 110 is a semantic definition in the corresponding YANG module definition. 111 On top of that definition, it is also important to maintain 112 contextual information about the collection environment. 114 As an example, a database could store a time series representing the 115 evolution of a specific counter. When analyzing the data, it is 116 important to understand that this counter was requested from the 117 network element at specific cadence, as this exact cadence might not 118 be observed in the time series, potentially implying that the network 119 element was under stress. The same time series might report some 120 values as 0 or might even omit some values. This might be explained 121 by a too small observation period, compared to the minimum-observed- 122 period [I-D.claise-netconf-metadata-for-collection]. Again, knowing 123 the conditions under which the counter was collected and streamed is 124 crucial. Indeed, taking into account the value of 0 might lead to a 125 wrong conclusion that the counter dropped to zero. This document 126 specifies the data collection manifest, which contains the required 127 information to characterize how and when the telemetry information 128 was metered. 130 Precisely characterizing the source used for producing the data (that 131 is the platform manifest) may also be useful to complete the data 132 collection context. As an example, knowing the exact data source 133 software specification might reveal a particularity in the observed 134 data, explained by a specific bug, or a specific bug fix. This is 135 also necessary to ensure the reliability of the collected data. On 136 top of that, in particular for MDT, it is crucial to know the set of 137 YANG modules supported by the device, along with their deviations. 138 In some cases, there might even be some backwards incompatible 139 changes in native modules between one OS version to the next one. 140 This information must be compiled in a platform manifest. 142 Some related YANG modules have been specified to retrieve the device 143 capabilities: 145 * [RFC9196] which models the device capabilities regarding the 146 production and export of telemetry data. 148 * [I-D.claise-netconf-metadata-for-collection], which is based on 149 the previous draft to define the optimal settings to stream 150 specific items (i.e., per path). 152 While these related YANG modules are important to discover the 153 capabilities before applying the telemetry configuration (such as on- 154 change), some of their content is part of the context for the 155 streamed data. The goal behind this specification is not to expose 156 new information via YANG objects but rather to define what needs to 157 be kept as metadata (the data manifest) to ensure that the collected 158 data can still be interpreted correctly, even if the source device is 159 not accesible (from the collection system), or if the device has been 160 updated (new operating system or new configuration). This manifest 161 contains two parts, the platform manifest and the data collection 162 manifest. The platform manifest is "pretty" stable and should change 163 only when the device is updated or patched. On the other hand, the 164 data collection manifest is likely to change each time a new MDT 165 subscription is requested and might even change if the device load 166 increases and collection periods are updated. To separate these two 167 parts, we enclose each of them in its own module. 169 We first present the module for the platform manifest in Section 3 170 and then the module for the data collection manifest in Section 4. 171 The full data manifest is obtained by combining these two modules. 172 We explain in Section 5 how the data-manifest can be collected and 173 how collected data is mapped to the data manifest. 175 3. Platform Manifest 177 3.1. Overview of the model 179 Figure 1 contains the YANG tree diagram [RFC8340] of the ietf- 180 collected-data-platform-manifest module. 182 module: ietf-collected-data-platform-manifest 183 +--ro platform 184 +--ro name? string 185 +--ro vendor? string 186 +--ro software-version? string 187 +--ro software-flavor? string 188 +--ro os-version? string 189 +--ro os-type? string 190 +--ro yang-library 191 | +--ro module-set* [name] 192 | | +--ro name string 193 | | +--ro module* [name] 194 | | | +--ro name yang:yang-identifier 195 | | | +--ro revision? revision-identifier 196 | | | +--ro namespace inet:uri 197 | | | +--ro location* inet:uri 198 | | | +--ro submodule* [name] 199 | | | | +--ro name yang:yang-identifier 200 | | | | +--ro revision? revision-identifier 201 | | | | +--ro location* inet:uri 202 | | | +--ro feature* yang:yang-identifier 203 | | | +--ro deviation* -> ../../module/name 204 | | +--ro import-only-module* [name revision] 205 | | +--ro name yang:yang-identifier 206 | | +--ro revision union 207 | | +--ro namespace inet:uri 208 | | +--ro location* inet:uri 209 | | +--ro submodule* [name] 210 | | +--ro name yang:yang-identifier 211 | | +--ro revision? revision-identifier 212 | | +--ro location* inet:uri 213 | +--ro schema* [name] 214 | | +--ro name string 215 | | +--ro module-set* -> ../../module-set/name 216 | +--ro datastore* [name] 217 | +--ro name ds:datastore-ref 218 | +--ro schema -> ../../schema/name 219 +--ro packages-set 220 +--ro package* [name version] 221 +--ro name -> /pkgs:packages/package/name 222 +--ro version -> /pkgs:packages/package[pkgs:name = current()/../name]/version 223 +--ro checksum? -> /pkgs:packages/package[pkgs:name = current()/../name][pkgs:version = current()/../version]/pkgs:checksum 225 Figure 1: YANG tree diagram for ietf-collected-data-platform- 226 manifest module 228 The platform manifest contains a comprehensive set of information 229 characterize a data source. The platform is identified by a set of 230 parameters ('name', 'software-version', 'software-flavor', 'os- 231 version', 'os-type') that are aligned with the YANG Catalog 232 www.yangcatalog.org [I-D.clacla-netmod-model-catalog] so that the 233 YANG catalog could be used to retrieve the YANG modules a posteriori. 235 The platform manifest also includes the contents of the YANG Library 236 [RFC8525]. That module set is particularly useful to define the 237 paths, as they are based on module names. Similarly, this module 238 defines the available datastores, which can be referred to from the 239 data-manifest, if necessary. If supported by the device, fetching 240 metrics from a specific datastore could enable some specific use 241 cases: monitoring configuration before it is committed, comparing 242 between the configuration and operational datastore. 244 Alternatively, the set of available YANG modules on the device can be 245 described via packages-set which contains a list of references to 246 YANG Packages [I-D.ietf-netmod-yang-packages]. 248 3.2. YANG module ietf-collected-data-platform-manifest 250 file "ietf-collected-data-platform- 251 manifest@2021-10-15.yang" 253 module ietf-collected-data-platform-manifest { 254 yang-version 1.1; 255 namespace "urn:ietf:params:xml:ns:yang:ietf-collected-data-platform-manifest"; 256 prefix platform-manifest; 258 import ietf-yang-library { 259 prefix yanglib; 260 reference 261 "RFC8525: YANG Library"; 262 } 263 import ietf-yang-packages { 264 prefix pkgs; 265 reference 266 "RFC XXXX: YANG Packages."; 267 } 269 organization 270 "IETF NETCONF (Network Configuration) Working Group"; 271 contact 272 "WG Web: 273 WG List: 274 Author: Benoit Claise 275 Author: Jean Quilbeuf "; 277 description 278 "This module describes the platform information to be used as 279 context of data collection from a given network element. The 280 contents of this model must be streamed along with the data 281 streamed from the network element so that the platform context 282 of the data collection can be retrieved later. 284 The data content of this model should not change except on 285 upgrade or patching of the device. 287 The key words 'MUST', 'MUST NOT', 'REQUIRED', 'SHALL', 288 'SHALL NOT', 'SHOULD', 'SHOULD NOT', 'RECOMMENDED', 289 'NOT RECOMMENDED', 'MAY', and 'OPTIONAL' in this document 290 are to be interpreted as described in BCP 14 (RFC 2119) 291 (RFC 8174) when, and only when, they appear in all 292 capitals, as shown here. 294 Copyright (c) 2022 IETF Trust and the persons identified as 295 authors of the code. All rights reserved. 297 Redistribution and use in source and binary forms, with or 298 without modification, is permitted pursuant to, and subject 299 to the license terms contained in, the Simplified BSD License 300 set forth in Section 4.c of the IETF Trust's Legal Provisions 301 Relating to IETF Documents 302 (https://trustee.ietf.org/license-info). 303 This version of this YANG module is part of RFC XXXX; see the 304 RFC itself for full legal notices. "; 306 revision 2021-10-15 { 307 description 308 "Initial revision"; 309 reference 310 "RFC xxxx: Title to be completed"; 311 } 313 container platform { 314 config false; 315 description 316 "Contains information about the platform that allows to identify 317 and understand the individual data collection information. "; 318 leaf name { 319 type string; 320 description 321 "Platform on which this module is implemented."; 322 } 323 leaf vendor { 324 type string; 325 description 326 "Organization that implements that platform."; 327 } 328 leaf software-version { 329 type string; 330 description 331 "Name of the version of software. With respect to most network 332 device appliances, this will be the operating system version. 333 But for other YANG module implementation, this would be a 334 version of appliance software. Ultimately, this should 335 correspond to a version string that will be recognizable by the 336 consumers of the platform."; 337 } 338 leaf software-flavor { 339 type string; 340 description 341 "A variation of a specific version where YANG model support 342 may be different. Depending on the vendor, this could be a 343 license, additional software component, or a feature set."; 344 } 345 leaf os-version { 346 type string; 347 description 348 "Version of the operating system using this module. This is 349 primarily useful if the software implementing the module is an 350 application that requires a specific operating system 351 version."; 352 } 353 leaf os-type { 354 type string; 355 description 356 "Type of the operating system using this module. This is 357 primarily useful if the software implementing the module is an 358 application that requires a specific operating system type."; 359 } 360 container yang-library { 361 description 362 "The YANG library of the device specifying the modules available 363 in each of the datastores."; 364 uses yanglib:yang-library-parameters; 365 } 366 container packages-set { 367 description 368 "Alternatively to module-set, use a list of yang packages to 369 describe the list of available schema on the platform"; 370 uses pkgs:yang-ds-pkg-ref; 371 } 372 } 374 } 376 378 4. Data Collection Manifest 380 4.1. Overview of the model 382 Figure 2 contains the YANG tree diagram [RFC8340] of the ietf- 383 collected-data-manifest module. 385 module: ietf-collected-data-manifest 386 +--ro data-collection 387 +--ro mdt-subscriptions* [subscription-id] 388 +--ro subscription-id uint64 389 +--ro datastore? ds:datastore-ref 390 +--ro mdt-path-data-manifest* [path] 391 +--ro path yang:xpath1.0 392 +--ro requested-period? uint64 393 +--ro actual-period? uint64 394 +--ro on-change? boolean 395 +--ro suppress-redundancy? boolean 397 Figure 2: YANG tree diagram for ietf-collected-data-manifest module 399 The data-collection container contains the information related to 400 individual items collection. This subtree currently contains only 401 information about MDT collection. It could be extended and 402 extendable to represent other kinds of data collection. 404 MDT collection is organized in subscriptions. A given collector can 405 subscribe to one ore more subscriptions that usually contain a list 406 of paths. Such a collector only needs the data manifest for 407 subscriptions it subscribed to. The data manifest for MDT is 408 organized by subscriptions as well so that a collector can select 409 only its subscriptions. 411 We now have a chicken-and-egg issue if the collector collects the 412 data-manifest via MDT and wants the data-manifest for the data- 413 manifest subscription. First the collector will collect the actual 414 paths that it needs in subscription A. Once it has the subscription 415 id for A, it will need an additional subscription B for the data 416 manifest of paths in A. Then, it would need another subscription C 417 to fetch the data manifest for the subscription B and so on... A 418 possible solution would be adding in the "mdt" container an 419 additional list in that contains the data manifest for every path 420 that is a data manifest. By including that list in subscription B, 421 the collector would have the information about subscription B here. 423 The "datastore" leaf of the subscription container specifies from 424 which datastore the YANG paths are streamed. 426 Within a given collection subscription, the granularity of the 427 collection is defined by the path. Note that all devices do not 428 support an arbitrary granularity up to the leaf, usually for 429 performance reasons. Each path currently collected by the device 430 should show up in the mdt-path-data-manifest list. 432 For each path, the collection context must be specified including: 434 * 'on-change': when set to true, an update is sent as soon as and 435 only when a value changes. This is also known as Event-Driven 436 Telemetry (EDT). When set to false, the values are sent 437 regularly. 439 * 'suppress-redundancy' (only when 'on-change' is false): reduce 440 bandwidth usage by sending a regular update only if the value is 441 different from the previous update. 443 * 'requested-period' (only when 'on-change' is false): period 444 between two updates requested by the client for this path 446 * 'actual-period' (only when 'on-change 'is false): actual period 447 retained by the platform between two updates. That period could 448 be larger than the requested one as the router can adjust it for 449 performance reasons. 451 This information is crucial to understand the collected values. For 452 instance, the 'on-change' and 'suppress-redundancy' options, if set, 453 might remove a lot of messages from the database because values are 454 sent only when there is a change. 456 4.2. YANG module ietf-collected-data-manifest 458 file "ietf-collected-data-manifest@2021-10-15.yang" 460 module ietf-collected-data-manifest { 461 yang-version 1.1; 462 namespace "urn:ietf:params:xml:ns:yang:ietf-collected-data-manifest"; 463 prefix data-manifest; 465 import ietf-datastores { 466 prefix ds; 467 reference 468 "RFC 8342: Network Management Datastore Architecture."; 469 } 470 import ietf-yang-types { 471 prefix yang; 472 } 474 organization 475 "IETF NETCONF (Network Configuration) Working Group"; 476 contact 477 "WG Web: 478 WG List: 479 Author: Benoit Claise 480 Author: Jean Quilbeuf "; 481 description 482 "This module describes the context of data collection from a 483 given network element. The contents of this model must be 484 streamed along with the data streamed from the network 485 element so that the context of the data collection can 486 be retrieved later. 488 This module must be completed with 489 ietf-collected-data-platform-manifest 490 to capture the whole context of a data collection session. 492 The key words 'MUST', 'MUST NOT', 'REQUIRED', 'SHALL', 493 'SHALL NOT', 'SHOULD', 'SHOULD NOT', 'RECOMMENDED', 494 'NOT RECOMMENDED', 'MAY', and 'OPTIONAL' in this document 495 are to be interpreted as described in BCP 14 (RFC 2119) 496 (RFC 8174) when, and only when, they appear in all 497 capitals, as shown here. 499 Copyright (c) 2022 IETF Trust and the persons identified as 500 authors of the code. All rights reserved. 502 Redistribution and use in source and binary forms, with or 503 without modification, is permitted pursuant to, and subject 504 to the license terms contained in, the Simplified BSD License 505 set forth in Section 4.c of the IETF Trust's Legal Provisions 506 Relating to IETF Documents 507 (https://trustee.ietf.org/license-info). 508 This version of this YANG module is part of RFC XXXX; see the 509 RFC itself for full legal notices. "; 511 revision 2021-10-15 { 512 description 513 "Initial revision"; 514 reference 515 "RFC xxxx: Title to be completed"; 516 } 518 container data-collection { 519 config false; 520 description 521 "Defines the information for each collected object"; 522 list mdt-subscriptions { 523 key "subscription-id"; 524 description 525 "Contains the list of current subscriptions on the 526 local device. Enables the collector to select its own 527 subscriptions in the list."; 528 leaf subscription-id { 529 type uint64; 530 description 531 "Id of the subscription generated by the telemetry emitter. 532 The collector can use this id to retrieve information 533 about the collection status for the corresponding paths. 535 The type is inspired by openconfig-telemetry. TODO check if 536 ietf has telemetry modules that we could leafref to. 538 path to subscription id in openconfig: 539 openconfig-telemetry:telemetry-system/subscriptions/ 540 persistent-subscriptions/persistent-subscription/state/oid"; 541 } 542 leaf datastore { 543 type ds:datastore-ref; 544 description 545 "The datastore from which the data for this subscription has 546 been collected."; 547 } 548 list mdt-path-data-manifest { 549 key "path"; 550 description 551 "Status of the collection for the given path"; 552 leaf path { 553 type yang:xpath1.0; 554 description 555 "The XPath context in which this XPath expression is 556 evaluated is the datastore selected for the containing 557 subscription object. If the datastore is not specified 558 then the context is the operational datastore context."; 559 } 560 leaf requested-period { 561 type uint64; 562 description 563 "Requested period, in millisecond, between two successive 564 updates."; 565 // when on-change is false; 566 } 567 leaf actual-period { 568 type uint64; 569 description 570 "Period in milisecond between two successive updates 571 actually applied by the plaftorm at configuration time. 572 This period can be larger than the requested period as the 573 platform might adjust it for performance reasons."; 574 // when on-change is false; 575 } 576 leaf on-change { 577 type boolean; 578 description 579 "Whether the path is collected only when there is a 580 change, i.e. Event-Driven Telemetry is enabled."; 581 } 582 leaf suppress-redundancy { 583 type boolean; 584 description 585 "Whether the information is sent at every period or only 586 when there is a change between two successive pollings."; 587 } 588 } 589 // we could augment here with other kind of collection items 590 } 591 } 592 } 594 596 5. Collecting Data Manifest and Mapping Data to Data Manifest 598 The data manifest MUST be streamed all with the data and stored along 599 with the collected data. In case the collected data are moved to a 600 different place (typically a database), the data manifest MUST follow 601 the collected data. This can render the data unusable if that 602 context is lost, for instance when the data is stored without the 603 relevant information. The data manifest MUST be updated when the 604 data manifest information changes (for example, when a router is 605 upgraded), when a new telemetry subscription is configured, or when 606 the telemetry subscription paremeters change. 608 The data should be mapped to the data manifest. Since the data 609 manifest will not change as frequently as the data itself, it makes 610 sense to map several data to the same data manifest. Somehow, the 611 collected data must include a metadata pointing to the corresponding 612 data manifest. 614 The platform manifest is likely to remain the same until the device 615 is updated. So, the platform manifest only needs to be collected 616 once per streaming session and updated after a device reboot. 618 As this draft specifically focuses on giving context on data 619 collected via streamed telemetry, we can assume that a streaming 620 telemetry system is available. Collecting the data and platform 621 manifests can be done either by reusing that streaming telemetry 622 system (in-band) or using another system (out-of-band), for instance 623 by adding headers or saving manifests into a YANG instace file 624 [RFC9195]. 626 We propose to reuse the existing telemetry system (in-band approach) 627 in order to lower the efforts for implementing this draft. To enable 628 a platform supporting streaming telemetry to also support data 629 collection manifests, it is sufficient that this device supports the 630 models from Section 3 and Section 4. Recall that each type of 631 manifest has its own rough frequency update, i.e. at reboot for the 632 platform manifest and at new subscription or CPU load variation for 633 the data collection manifest. The data manifest MUST be streamed 634 with the YANG-Push on-change feature [RFC8641] (also called event- 635 driven telemetry). 637 5.1. Mapping Collected Data to the Data Manifest 639 With MDT, a particular datapoint is always associated to a path that 640 is itself part of a subscription. In order to enable a posteriori 641 retrieval of the data manifest associated to a datapoint, the 642 collector must: 644 * keep the path in the metadata of the collected values 646 * collect as well the data-manifest for the subscription and path 647 associated to the datapoint. 649 With this information, to retrieve the data manifest from the 650 datapoint, the following happens: 652 * the path is retrieved from the datapoint metadata 654 * the data-manifest for that path is retrieved by looking up on the 655 collected data-manifest. 657 In that scenario, the reliability of the collection of the data 658 manifest is the same as the reliability of the data collection 659 itself, since the data manifest is like any other data. For 660 telemetry based on gRPC for instance, a disconnection to the server 661 would be detected as the HTTP connection would fail. 663 6. Example 665 Below is an example of a data-manifest file: 667 file "ietf-collected-data-manifest@2021-10-15.yang" 669 { 670 "ietf-yang-instance-data:instance-data-set": { 671 "name": "data-manifest-example", 672 "content-schema": { 673 "module": "ietf-collected-data-manifest@2021-10-15" 674 }, 675 "timestamp": "2022-02-24T09:45:03Z", 676 "description": [ 677 "Data manifest for the subscription 4242" 678 ], 679 "content-data": { 680 "ietf-collected-data-manifest:data-collection": { 681 "mdt-subscriptions": [ 682 { 683 "subscription-id": 4242, 684 "mdt-path-data-manifest": [ 685 { 686 "path": "/ietf-interfaces:interfaces/interface/enabled", 687 "requested-period": 100, 688 "current-period": 10000, 689 "on-change": false, 690 "suppress-redundancy": false 691 }, 692 { 693 "path": "/ietf-interfaces:interfaces/interface/statistics/in-octets", 694 "requested-period": 100, 695 "current-period": 100, 696 "on-change": false, 697 "suppress-redundancy": false 698 } 699 ] 700 } 701 ] 702 } 703 } 704 } 705 } 707 708 The file above contains the data manifest for paths collected in the 709 subscription with id 4242. The requested period for both path is 710 this subscription was 100ms, however the status of the interface 711 could only be collected every 10s. 713 7. Security Considerations 715 As we are reusing an existing telemetry system, the security 716 considerations lies with the new content divulged in the new 717 manifests. Appropriate access control must be associated to the 718 corresponding leafs and containers. 720 8. IANA Considerations 722 This document includes no request to IANA. 724 9. Contributors 726 10. Open Issues 728 * Do we want to the hardware specifications, next to the OS 729 information? How to fully characterize a virtual device? Do we 730 need to include the vendor (as PEN for instance 731 https://www.iana.org/assignments/enterprise-numbers/enterprise- 732 numbers) ? 734 * Do we want to handle the absence of values, i.e. add information 735 about missed collection or errors in the collection context ? It 736 could also explain why some values are missing. On the other 737 hand, this might also be out scope. 739 * How do we handle other kinds of collection than MDT like netflow, 740 SNMP, CLI ? How do we map the collected data to the data-manifest 741 ? 743 * Align the terms with the YANG Push specifications. Ex: path to 744 subscription (TBC) 746 * Better explain the on-change example. 748 * Regarding the inclusion of ietf-yang-library in our module, do we 749 want to include as well the changes from ietf-yang-library- 750 revisions? What if other information are present in the yang- 751 libary from the platform? Should we use a YANG mount to capture 752 them as well (they would not be captured with our use of the main 753 yang-library grouping). 755 * Henk: how does this interact with SBOM effort? 756 * Eliot: important to give integrity of the information a lot of 757 thought. Threat model to be considered. 759 11. References 761 11.1. Normative References 763 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 764 Requirement Levels", BCP 14, RFC 2119, 765 DOI 10.17487/RFC2119, March 1997, 766 . 768 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 769 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 770 May 2017, . 772 [RFC8340] Bjorklund, M. and L. Berger, Ed., "YANG Tree Diagrams", 773 BCP 215, RFC 8340, DOI 10.17487/RFC8340, March 2018, 774 . 776 [RFC8525] Bierman, A., Bjorklund, M., Schoenwaelder, J., Watsen, K., 777 and R. Wilton, "YANG Library", RFC 8525, 778 DOI 10.17487/RFC8525, March 2019, 779 . 781 [RFC8641] Clemm, A. and E. Voit, "Subscription to YANG Notifications 782 for Datastore Updates", RFC 8641, DOI 10.17487/RFC8641, 783 September 2019, . 785 [RFC9195] Lengyel, B. and B. Claise, "A File Format for YANG 786 Instance Data", RFC 9195, DOI 10.17487/RFC9195, February 787 2022, . 789 11.2. Informative References 791 [I-D.clacla-netmod-model-catalog] 792 Clarke, J. and B. Claise, "YANG module for 793 yangcatalog.org", Work in Progress, Internet-Draft, draft- 794 clacla-netmod-model-catalog-03, 3 April 2018, 795 . 798 [I-D.claise-netconf-metadata-for-collection] 799 Claise, B., Nayyar, M., and A. R. Sesani, "Per-Node 800 Capabilities for Optimum Operational Data Collection", 801 Work in Progress, Internet-Draft, draft-claise-netconf- 802 metadata-for-collection-02, 12 July 2021, 803 . 806 [I-D.ietf-netmod-yang-packages] 807 Wilton, R., Rahman, R., Clarke, J., Sterne, J., and B. Wu, 808 "YANG Packages", Work in Progress, Internet-Draft, draft- 809 ietf-netmod-yang-packages-03, 4 March 2022, 810 . 813 [RFC9196] Lengyel, B., Clemm, A., and B. Claise, "YANG Modules 814 Describing Capabilities for Systems and Datastore Update 815 Notifications", RFC 9196, DOI 10.17487/RFC9196, February 816 2022, . 818 Appendix A. Changes between revisions 820 Version 2 822 Alignment with YANGCatalog YANG module: name, vendor 824 Clarify the use of YANG instance file 826 Editorial improvements 828 Version 1 830 Adding more into data platform: yang packages, whole yanglib 831 module to specify datastores 833 Setting the right type for periods: int64 -> uint64 835 Specify the origin datastore for mdt subscription 837 Set both models to config false 839 Applying text comments from Mohamed Boucadair 841 Adding an example of data-manifest file 843 Adding rationale for reusing telemetry system for collection of 844 the manifests 845 Export manifest with on change telemetry as opposed to YANG 846 instance file 848 Version 0 850 Initial version 852 Acknowledgements 854 Thanks to Mohamed Boucadair and Tianran Zhou for their reviews and 855 comments. 857 Authors' Addresses 859 Benoit Claise 860 Huawei 861 Email: benoit.claise@huawei.com 863 Jean Quilbeuf 864 Huawei 865 Email: jean.quilbeuf@huawei.com 867 Diego R. Lopez 868 Telefonica I+D 869 Don Ramon de la Cruz, 82 870 Madrid 28006 871 Spain 872 Email: diego.r.lopez@telefonica.com 874 Ignacio Dominguez 875 Universidad Politecnica de Madrid 876 Avenida Complutense 30 877 Madrid 28040 878 Spain 879 Email: i.dominguezm@upm.es 881 Thomas Graf 882 Swisscom 883 Binzring 17 884 CH-8045 Zurich 885 Switzerland 886 Email: thomas.graf@swisscom.com