idnits 2.17.1 draft-ietf-ipfix-structured-data-06.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The abstract seems to contain references ([RFC5102], [RFC5101]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (May 3, 2011) is 4732 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Obsolete normative reference: RFC 5101 (Obsoleted by RFC 7011) ** Obsolete normative reference: RFC 5102 (Obsoleted by RFC 7012) ** Obsolete normative reference: RFC 5226 (Obsoleted by RFC 8126) Summary: 4 errors (**), 0 flaws (~~), 1 warning (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 IPFIX Working Group B. Claise 3 Internet-Draft G. Dhandapani 4 Update: RFC5102 P. Aitken 5 Intended Status: Standards Track S. Yates 6 Expires: November 3, 2011 Cisco Systems, Inc. 7 May 3, 2011 9 Export of Structured Data in IPFIX 10 draft-ietf-ipfix-structured-data-06.txt 12 Status of this Memo 14 This Internet-Draft is submitted to IETF in full conformance 15 with the provisions of BCP 78 and BCP 79. 17 Internet-Drafts are working documents of the Internet 18 Engineering Task Force (IETF), its areas, and its working 19 groups. Note that other groups may also distribute working 20 documents as Internet-Drafts. 22 Internet-Drafts are draft documents valid for a maximum of six 23 months and may be updated, replaced, or obsoleted by other 24 documents at any time. It is inappropriate to use Internet- 25 Drafts as reference material or to cite them other than as "work 26 in progress." 28 The list of current Internet-Drafts can be accessed at 29 http://www.ietf.org/ietf/1id-abstracts.txt 31 The list of Internet-Draft Shadow Directories can be accessed at 32 http://www.ietf.org/shadow.html 34 This Internet-Draft will expire on September, 2011. 36 Copyright Notice 38 Copyright (c) 2011 IETF Trust and the persons identified as the 39 document authors. All rights reserved. 41 This document is subject to BCP 78 and the IETF Trust's Legal 42 Provisions Relating to IETF Documents 43 (http://trustee.ietf.org/license-info) in effect on the date of 44 publication of this document. Please review these documents 45 carefully, as they describe your rights and restrictions with 46 respect to this document. Code Components extracted from this 47 document must include Simplified BSD License text as described 48 in Section 4.e of the Trust Legal Provisions and are provided 49 without warranty as described in the Simplified BSD License. 51 Abstract 53 This document specifies an extension to the IP Flow Information 54 eXport (IPFIX) protocol specification in [RFC5101] and the IPFIX 55 information model specified in [RFC5102] to support hierarchical 56 structured data and lists (sequences) of Information Elements in 57 data records. This extension allows definition of complex data 58 structures such as variable-length lists and specification of 59 hierarchical containment relationships between Templates. 60 Finally, the semantics are provided in order to express the 61 relationship among multiple list elements in a structured data 62 record. 64 Conventions used in this document 66 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL 67 NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and 68 "OPTIONAL" in this document are to be interpreted as described 69 in RFC 2119 [RFC2119]. 71 Table of Contents 73 1. Overview...................................................7 74 1.1. IPFIX Documents Overview..............................7 75 1.2. Relationship between IPFIX and PSAMP..................8 76 2. Introduction...............................................8 77 2.1. The IPFIX Track.......................................9 78 2.2. The IPFIX Limitations................................10 79 2.3. Structured Data Use Cases............................10 80 2.4. Specifications Summary...............................12 81 3. Terminology...............................................13 82 3.1. New Terminology......................................13 83 4. Linkage with the IPFIX Information Model..................13 84 4.1. New Abstract Data Types..............................14 85 4.1.1. basicList.......................................14 86 4.1.2. subTemplateList.................................14 87 4.1.3. subTemplateMultiList............................14 88 4.2. New Data Type Semantic...............................14 89 4.2.1. List............................................15 90 4.3. New Information Elements.............................15 91 4.3.1. basicList.......................................15 92 4.3.2. subTemplateList.................................15 93 4.3.3. subTemplateMultiList............................15 94 4.4. New Structured Data Type Semantics...................16 95 4.4.1. undefined.......................................16 96 4.4.2. noneOf..........................................16 97 4.4.3. exactlyOneOf....................................17 98 4.4.4. oneOrMoreOf.....................................18 99 4.4.5. allOf...........................................18 100 4.4.6. ordered.........................................19 101 4.5. Encoding of IPFIX Data Types.........................19 102 4.5.1. basicList.......................................19 103 4.5.2. subTemplateList.................................22 104 4.5.3. subTemplateMultiList............................24 105 5. Structured Data Format....................................28 106 5.1. Length Encoding Considerations.......................29 107 5.2. Recursive Structured Data............................29 108 5.3. Structured Data Information Elements Applicability in 109 Options Template Sets.....................................30 110 5.4. Usage Guidelines for Equivalent Data Representations.31 111 5.5. Padding..............................................32 112 5.6. Semantic.............................................32 113 6. Template Management.......................................36 114 7. The Collecting Process's Side.............................37 115 8. Defining New Information Elements Based on the New 116 Abstract Data Types..........................................38 117 9. Structured Data Encoding Examples.........................38 118 9.1. Encoding a Multicast Data Record with basicList......38 119 9.2. Encoding a Load-balanced Data Record with a basicList40 120 9.3. Encoding subTemplateList.............................41 121 9.4. Encoding subTemplateMultiList........................44 122 9.5. Encoding an Options Template Set using Structured 123 Data......................................................49 124 10. Relationship with the Other IFPIX Documents..............54 125 10.1. Relationship with Reducing Redundancy...............54 126 10.1.1. Encoding Structured Data Element using Common 127 Properties.............................................54 128 10.1.2. Encoding Common Properties elements With 129 Structured Data Information Element....................54 130 10.2. Relationship with Guidelines for IPFIX Testing......56 131 10.3. Relationship with IPFIX Mediation Function..........57 132 11. IANA Considerations......................................57 133 11.1. New Abstract Data Types.............................58 134 11.1.1. basicList......................................58 135 11.1.2. subTemplateList................................58 136 11.1.3. subTemplateMultiList...........................58 137 11.2. New Data Type Semantics.............................58 138 11.2.1. list...........................................59 139 11.3. New Information Elements............................59 140 11.3.1. basicList......................................59 141 11.3.2. subTemplateList................................59 142 11.3.3. subTemplateMultiList...........................60 143 11.4. New Structured Data Semantics.......................60 144 11.4.1. undefined......................................60 145 11.4.2. noneOf.........................................60 146 11.4.3. exactlyOneOf...................................61 147 11.4.4. oneOrMoreOf....................................61 148 11.4.5. allOf..........................................61 149 11.4.6. ordered........................................61 150 12. Security Considerations..................................62 151 13. References...............................................62 152 13.1. Normative References................................62 153 13.2. Informative References..............................62 154 14. Acknowledgement..........................................63 155 15. Authors' Addresses.......................................64 156 Appendix A. Additions to XML Specification of IPFIX 157 Information Elements and Abstract Data Types.................65 158 Appendix B. Encoding IPS Alert using Structured Data 159 Information Elements.........................................70 161 Table of Figures 163 Figure A: basicList Encoding...................................19 164 Figure B: basicList Encoding with Enterprise Number............21 165 Figure C: Variable-Length basicList Encoding (Length < 255 octets) 166 ...........................................................21 167 Figure D: Variable-Length basicList Encoding (Length 0 to 65535 168 octets) ....................................................22 169 Figure E: subTemplateList Encoding.............................22 170 Figure F: Variable-Length subTemplateList Encoding (Length < 255 171 octets) ....................................................23 172 Figure G: Variable-Length subTemplateList Encoding (Length 0 to 173 65535 octets) ..............................................24 174 Figure H: subTemplateMultiList Encoding........................25 175 Figure I: Variable-Length subTemplateMultiList Encoding (Length < 176 255 octets) ................................................27 177 Figure J: Variable-Length subTemplateMultiList Encoding (Length 0 178 to 65535 octets) ...........................................28 179 Figure K: Encoding basicList, Template Record..................39 180 Figure L: Encoding basicList, Data Record, Semantic allOf......40 181 Figure M: Encoding basicList, Data Record with Variable-Length 182 Elements, Semantic allOf ...................................40 183 Figure N: Encoding basicList, Data Record, Semantic ExactlyOneOf 184 ...........................................................41 185 Figure O: Encoding subTemplateList, Template for One-Way Delay 186 Metrics ....................................................42 187 Figure P: Encoding subTemplateList, Template Record............43 188 Figure Q: Encoding subTemplateList, Data Set...................44 189 Figure R: Encoding subTemplateMultiList, Template for Filtering 190 Attributes .................................................47 191 Figure S: Encoding subTemplateMultiList, Template for Sampling 192 Attributes .................................................47 193 Figure T: Encoding subTemplateMultiList, Template for Flow Record 194 ...........................................................48 195 Figure U: Encoding subTemplateMultiList, Data Set..............49 196 Note that the example could further be improved with a basicList 197 of selectorId if many Selector IDs have to be reported. ....51 198 Figure V: PSAMP SSRI to be encoded.............................51 199 Figure W: Options Template Record for PSAMP SSRI using 200 subTemplateMultiList .......................................51 201 Figure X: PSAMP SSRI, Template Record for interface............52 202 Figure Y: PSAMP SSRI, Template Record for linecard.............52 203 Figure Z: PSAMP SSRI, Template Record for linecard and interface 204 ...........................................................52 205 Figure ZA: Example of a PSAMP SSRI Data Record, Encoded using a 206 subTemplateMultiList .......................................53 207 Figure ZB: Common and Specific Properties Exported Together 208 [RFC5473] ..................................................55 209 Figure ZC: Common and Specific Properties Exported Separately 210 according to [RFC5473] .....................................55 212 Figure ZD: Common and Specific Properties Exported with Structured 213 Data Information Element ...................................55 214 Figure B0: Encoding IPS Alert, Template for Target.............72 215 Figure B1: Encoding IPS Alert, Template for Attacker...........72 216 Figure B2: Encoding IPS Alert, Template for Participant........73 217 Figure B3: Encoding IPS Alert, Template for IPS Alert..........73 218 Figure B4: Encoding IPS Alert, Data Set........................75 219 1. Overview 221 1.1. IPFIX Documents Overview 223 The IPFIX Protocol [RFC5101] provides network administrators with 224 access to IP Flow information. 226 The architecture for the export of measured IP Flow information 227 out of an IPFIX Exporting Process to a Collecting Process is 228 defined in the IPFIX Architecture [RFC5470], per the requirements 229 defined in RFC 3917 [RFC3917]. 231 The IPFIX Architecture [RFC5470] specifies how IPFIX Data Records 232 and Templates are carried via a congestion-aware transport 233 protocol from IPFIX Exporting Processes to IPFIX Collecting 234 Processes. 236 IPFIX has a formal description of IPFIX Information Elements, 237 their name, type and additional semantic information, as specified 238 in the IPFIX information model [RFC5102]. 240 In order to gain a level of confidence in the IPFIX 241 implementation, probe the conformity and robustness, and allow 242 interoperability, the Guidelines for IPFIX Testing [RFC5471] 243 presents a list of tests for implementers of compliant Exporting 244 Processes and Collecting Processes. 246 The Bidirectional Flow Export [RFC5103] specifies a method for 247 exporting bidirectional flow (biflow) information using the IP 248 Flow Information Export (IPFIX) protocol, representing each Biflow 249 using a single Flow Record. 251 The "Reducing Redundancy in IP Flow Information Export (IPFIX) and 252 Packet Sampling (PSAMP) Reports" [RFC5473] specifies a bandwidth 253 saving method for exporting Flow or packet information, by 254 separating information common to several Flow Records from 255 information specific to an individual Flow Record: common Flow 256 information is exported only once. 258 1.2. Relationship between IPFIX and PSAMP 260 The specification in this document applies to the IPFIX protocol 261 specifications [RFC5101]. All specifications from [RFC5101] apply 262 unless specified otherwise in this document. 264 The Packet Sampling (PSAMP) protocol [RFC5476] specifies the 265 export of packet information from a PSAMP Exporting Process to a 266 PSAMP Collecting Process. Like IPFIX, PSAMP has a formal 267 description of its information elements, their name, type and 268 additional semantic information. The PSAMP information model is 269 defined in [RFC5477]. 271 As the PSAMP protocol specifications [RFC5476] are based on the 272 IPFIX protocol specifications, the specifications in this document 273 are also valid for the PSAMP protocol. 275 Indeed, the major difference between IPFIX and PSAMP is that the 276 IPFIX protocol exports Flow Records while the PSAMP protocol 277 exports Packet Reports. From a pure export point of view, IPFIX 278 will not distinguish a Flow Record composed of several packets 279 aggregated together, from a Flow Record composed of a single 280 packet. So the PSAMP export can be seen as a special IPFIX Flow 281 Record containing information about a single packet. 283 2. Introduction 285 While collecting the interface counters every five minutes has 286 proven to be useful in the past, more and more granular 287 information is required from network elements for a series of 288 applications: performance assurance, capacity planning, security, 289 billing, or simply monitoring. However, the amount of information 290 has become so large that, when dealing with highly granular 291 information such as Flow information, a push mechanism (as opposed 292 to a pull mechanism, such as SNMP) is the only solution for 293 routers whose primary function is to route packets. Indeed, 294 polling short-lived Flows via SNMP is not an option: high end 295 routers can support hundreds of thousands of Flows simultaneously. 296 Furthermore, in order to reduce the export bandwidth requirements, 297 the network elements have to integrate mediation functions to 298 aggregate the collected information, both in space (typically from 299 different line cards or different Exporters) and in time. 301 Typically, it would be beneficial if access routers could export 302 Flow Records, composed of the counters before and after an 303 optimization mechanism on the egress interface, instead of 304 exporting two Flow Records with identical tuple information. 306 In terms of aggregation in time, let us imagine that, for 307 performance assurance, the network management application must 308 receive the performance metrics associated with a specific flow, 309 every millisecond. Since the performance metrics will be 310 constantly changing, there is a new dimension to the Flow 311 definition: we are not dealing anymore with a single Flow lasting 312 a few seconds or a few minutes, but with a multitude of one 313 millisecond sub flows for which the performance metrics are 314 reported. 316 Which current protocol is suitable for these requirements: push 317 mechanism, highly granular information, and huge number of similar 318 records? IPFIX, as specified in RFC5101 would give part of the 319 solution. 321 2.1. The IPFIX Track 323 The IPFIX working group has specified a protocol to export Flow 324 information [RFC5101]. This protocol is designed to export 325 information about IP traffic Flows and related measurement data, 326 where a Flow is defined by a set of key attributes (e.g. source 327 and destination IP address, source and destination port, etc.). 329 The IPFIX protocol specification [RFC5101] specifies that traffic 330 measurements for Flows are exported using a TLV (type, length, 331 value) format. The information is exported using a Template 332 Record that is sent once to export the {type, length} pairs that 333 define the data format for the Information Elements in a Flow. 334 The Data Records specify values for each Flow. 336 Based on the Requirements for IP Flow Information Export (IPFIX) 337 [RFC3917], the IPFIX protocol has been optimized to export Flow 338 related information. However, thanks to its Template mechanism, 339 the IPFIX protocol can export any type of information, as long as 340 the relevant Information Element is specified in the IPFIX 341 information model [RFC5102], registered with IANA [IANA-IPFIX], or 342 specified as an enterprise-specific Information Element. For each 343 Information Element, the IPFIX information model [RFC5102] defines 344 a numeric identifier, an abstract data type, an encoding mechanism 345 for the data type, and any semantic constraints. Only basic, 346 single-valued data types, e.g., numbers, strings, and network 347 addresses are currently supported. 349 2.2. The IPFIX Limitations 351 The IPFIX protocol specification [RFC5101] does not support the 352 encoding of hierarchical structured data and arbitrary-length 353 lists (sequences) of Information Elements as fields within a 354 Template Record. As it is currently specified, a Data Record is a 355 "flat" list of single-valued attributes. However, it is a common 356 data modeling requirement to compose complex hierarchies of data 357 types, with multiple occurrences, e.g., 0..* cardinality allowed 358 for instances of each Information Element in the hierarchy. 360 A typical example is the MPLS label stack entries model. An early 361 NetFlow implementation used two Information Elements to represent 362 the MPLS label stack entry: a "label stack entry position" 363 followed by a "label stack value". However, several drawbacks 364 were discovered. Firstly, the Information Elements in the 365 Template Record had to be imposed so that the position would 366 always precede the value. However, some encoding optimizations 367 are based on the permutation of Information Element order. 368 Secondly, a new semantic intelligence, not described in the 369 information model, had to be hardcoded in the Collecting Process: 370 the label value at the position "X" in the stack is contained in 371 the "label stack value" Information Element following by a "label 372 stack entry position" Information Element containing the value 373 "X". Therefore, this model was abandoned. 375 The selected solution in the IPFIX information model [RFC5102] is 376 a long series of Information Elements: mplsTopLabelStackSection, 377 mplsLabelStackSection2, mplsLabelStackSection3, 378 mplsLabelStackSection4, mplsLabelStackSection5, 379 mplsLabelStackSection6, mplsLabelStackSection7, 380 mplsLabelStackSection8, mplsLabelStackSection9, 381 mplsLabelStackSection10. While this model removes any ambiguity, 382 it overloads the IPFIX information model with repetitive 383 information. Furthermore, if mplsLabelStackSection11 is required, 384 IANA [IANA-IPFIX] will not be able to assign the new Information 385 Element next to the other ones in the registry, which might cause 386 some confusion. 388 2.3. Structured Data Use Cases 390 Clearly the MPLS label stack entries issue can best be solved by 391 using a real structured data type composed of ("label stack entry 392 position", "label stack value") pairs, potentially repeated 393 multiple times in Flow Records, since this would be the most 394 efficient from an information model point of view. 396 Some more examples enter the same category: how to encode the list 397 of output interfaces in a multicast Flow, how to encode the list 398 of BGP Autonomous Systems (AS) in a BGP Flow, how to encode the 399 BGP communities in a BGP Flow, etc? 401 The one-way delay passive measurement, which is described in the 402 IPFIX Applicability [RFC5472], is yet another example that would 403 benefit from a structured data encoding. Assuming synchronized 404 clocks, the Collector can deduce the one-way delay between two 405 Observation Points from the following two Information Elements, 406 collected from two different Observation Points: 407 - Packet arrival time: observationTimeMicroseconds [RFC5477] 408 - Packet ID: digestHashValue [RFC5477] 409 In practice, this implies that many pairs of 410 (observationTimeMicroseconds, digestHashValue) must be exported 411 for each Observation Point, even if Hash-Based Filtering [RFC5475] 412 is used. On top of that information, if the requirement is to 413 understand the one-way delay per application type, the 5-tuple 414 (source IP address, destination IP address, protocol, source port, 415 destination port) would need to be added to every Flow Record. 416 Instead of exporting this repetitive 5-tuple, as part of every 417 single Flow Record a Flow Record composed of a structured data 418 type such as the following would save a lot of bandwidth: 420 5-tuple 421 { observationTimeMicroseconds 1, digestHashValue 1 } 422 { observationTimeMicroseconds 2, digestHashValue 2 } 423 { observationTimeMicroseconds 3, digestHashValue 3 } 424 { ... , ... } 426 As a last example, here is a more complex case of hierarchical 427 structured data encoding. Consider the example scenario of an IPS 428 (Intrusion Prevention System) alert data structure containing 429 multiple participants, where each participant contains multiple 430 attackers and multiple targets, with each target potentially 431 composed of multiple applications, as depicted below: 433 alert 434 signatureId 435 protocolIdentifier 436 riskRating 437 participant 1 438 attacker 1 439 sourceIPv4Address 440 applicationId 442 ... 443 attacker N 444 sourceIPv4Address 445 applicationId 446 target 1 447 destinationIPv4Address 448 applicationId 1 449 ... 450 applicationId n 451 ... 452 target N 453 destinationIPv4Address 454 applicationId 1 455 ... 456 applicationId n 457 participant 2 458 ... 460 To export this information in IPFIX, the data would need to be 461 flattened (thus losing the hierarchical relationships) and a new 462 IPFIX Template created for each alert, according to the number of 463 applicationId elements in each target, the number of targets and 464 attackers in each participant, and the number of participants in 465 each alert. Clearly each Template will be unique to each alert, 466 and a large amount of CPU, memory and export bandwidth will be 467 wasted creating, exporting, maintaining, and withdrawing the 468 Templates. See Appendix B for a specific example related to this 469 case study. 471 2.4. Specifications Summary 473 This document specifies an IPFIX extension to support hierarchical 474 structured data and variable-length lists by defining three new 475 Information Elements and three corresponding new abstract data 476 types called basicList, subTemplateList, and subTemplateMultiList. 477 These are defined in Section 4.1. 479 The three Structured Data Information Elements carry some semantic 480 information so that the Collecting Process can understand the 481 relationship between the different list elements. The semantic in 482 the Structured Data Information Elements is provided in order to 483 express the relationship among the multiple top-level list 484 elements. As an example, if a list is composed of the elements 485 (A,B,C), the semantic expresses the relationship among A, B, and 486 C, regardless of whether A, B, and C, are individual elements or 487 list of elements. 489 It is important to note that whereas the Information Elements and 490 abstract data types defined in the IPFIX information model 491 [RFC5102] represent single values, these new abstract data types 492 are structural in nature and primarily contain references to other 493 Information Elements and to Templates. By referencing other 494 Information Elements and Templates from an Information Element's 495 data content, it is possible to define complex data structures 496 such as variable-length lists and to specify hierarchical 497 containment relationships between Templates. Therefore, this 498 document prefers the more generic "Data Record" term to the "Flow 499 Record" term. 501 This document specifies three new abstract data types, which are 502 basic blocks to represent structured data. However, this document 503 does not comment on all possible combinations of basicList, 504 subTemplateList, and subTemplateMultiList. Neither, does it limit 505 the possible combinations. 507 3. Terminology 509 IPFIX-specific terminology used in this document is defined in 510 Section 2 of the IPFIX protocol specification [RFC5101] and 511 Section 3 of PSAMP protocol specification [RFC5476]. As in 512 [RFC5101], these IPFIX-specific terms have the first letter of a 513 word capitalized when used in this document. 515 3.1. New Terminology 517 Structured Data Information Element 519 One of the Information Elements supporting structured data, 520 i.e., the basicList, subTemplateList, or subTemplateMultiList 521 Information Elements specified in section 4.3. 523 4. Linkage with the IPFIX Information Model 525 As in the IPFIX Protocol specification [RFC5101], the new 526 Information Elements specified in Section 4.3. below MUST be sent 527 in canonical format in network-byte order (also known as the big- 528 endian byte ordering). 530 4.1. New Abstract Data Types 532 This document specifies three new abstract data types, as 533 described below. 535 4.1.1. basicList 537 The type "basicList" represents a list of zero or more instances 538 of any Information Element, primarily used for single-valued data 539 types. For example, a list of port numbers, a list of interface 540 indexes, a list of AS in a BGP AS-PATH, etc. 542 4.1.2. subTemplateList 544 The type "subTemplateList" represents a list of zero or more 545 instances of a structured data type, where the data type of each 546 list element is the same and corresponds with a single Template 547 Record. For example, a structured data type composed of multiple 548 pairs of ("MPLS label stack entry position", "MPLS label stack 549 value"), a structured data type composed of performance metrics, a 550 structured data type composed of multiple pairs of IP address, 551 etc. 553 4.1.3. subTemplateMultiList 555 The type "subTemplateMultiList" represents a list of zero or more 556 instances of a structured data type, where the data type of each 557 list element can be different and corresponds with different 558 template definitions. For example, a structured data type 559 composed of multiple access-list entries, where entries can be 560 composed of different criteria types. 562 4.2. New Data Type Semantic 564 This document specifies a new data type semantic, in addition to 565 the ones specified in the section 3.2 of the IPFIX information 566 model [RFC5102], as described below. 568 4.2.1. List 570 A list represents an arbitrary-length sequence of zero or more 571 structured data Information Elements, either composed of regular 572 Information Elements or composed of data conforming to a Template 573 Record. 575 4.3. New Information Elements 577 This document specifies three new Information Elements, as 578 described below. 580 4.3.1. basicList 582 A basicList specifies a generic Information Element with a 583 basicList abstract data type as defined in Section 4.1.1. and list 584 semantics as defined in Section 4.2.1. For example, a list of 585 port numbers, a list of interface indexes, etc. 587 EDITOR'S NOTE: while waiting for IANA [IANA-IPFIX] to assign this 588 new Information Element identifier, the value XXX is used in all 589 the examples and in the XML in Appendix A. 591 4.3.2. subTemplateList 593 A subTemplateList specifies a generic Information Element with a 594 subTemplateList abstract data type as defined in Section 4.1.2. 595 and list semantics as defined in Section 4.2.1. 597 EDITOR'S NOTE: while waiting for IANA [IANA-IPFIX] to assign this 598 new Information Element identifier, the value YYY is used in all 599 the examples and in the XML in Appendix A. 601 4.3.3. subTemplateMultiList 603 A subTemplateMultiList specifies a generic Information Element 604 with a subTemplateMultiList abstract data type as defined in 605 Section 4.1.3. and list semantics as defined in Section 4.2.1. 607 EDITOR'S NOTE: while waiting for IANA [IANA-IPFIX] to assign this 608 new Information Element identifier, the value ZZZ is used in all 609 the examples and in the XML in Appendix A. 611 4.4. New Structured Data Type Semantics 613 Structured data type semantics are provided in order to express 614 the relationship among multiple list elements in a Structured Data 615 Information Element. These structured data type semantics require 616 a new IPFIX subregistry, as specified in the "IANA Considerations" 617 section. The semantics are specified in the next following 618 subsections. 620 4.4.1. undefined 622 The "undefined" structured data type semantic specifies that the 623 semantic of list elements is not specified, and that, if a 624 semantic exists, then it is up to the Collecting Process to draw 625 its own conclusions. The "undefined" structured data type 626 semantic, which is the default value, is used when none other 627 structured data type semantic applies. 629 For example, a mediator that wants to translate IPFIX [RFC5101] 630 into the export of structured data according to the specifications 631 in this document, doesn't know what the semantic is; it can only 632 guess, as the IPFIX specifications [RFC5101] does not contain any 633 semantic. Therefore, the mediator should use the "undefined" 634 semantic. 636 4.4.2. noneOf 638 The "noneOf" structured data type semantic specifies that none of 639 the elements are actual properties of the Data Record. 641 For example, a mediator might want to report to a Collector that a 642 specific Flow is suspicious, but that it checked already that this 643 Flow does not belong to the attack type 1, attack type 2, and 644 attack type 3. So this Flow might need some further inspection. 645 In such a case, the mediator would report the Flow Record with a 646 basicList composed of (attack type 1, attack type 2, attack type 647 3) and the respective structured data type semantic of "noneOf". 649 Another example is a router that monitors some specific BGP AS- 650 PATHs and reports if a Flow belongs to any of them. If the router 651 wants to export that a Flow does not belong to any of the 652 monitored BGP AS-PATHs, the router reports a Data Record with a 653 basicList composed of (BGP AS-PATH 1, BGP AS-PATH 2, BGP AS-PATH 654 3) and the respective structured data type semantic of "noneOf". 656 4.4.3. exactlyOneOf 658 The "exactlyOneOf" structured data type semantic specifies that 659 only a single element from the structured data is an actual 660 property of the Data Record. This is equivalent to a logical XOR 661 operation. 663 For example, if a Flow record contains a basicList of outgoing 664 interfaces with the "exactlyOneOf" semantic, then it implies that 665 the reported Flow only egressed from a single interface, although 666 the Flow Record lists all of the possible outgoing interfaces. 667 This is a typical example of a per destination load-balancing. 669 Another example is a mediator that must aggregate Data Records 670 from different Observation Points and report an aggregated 671 Observation Point. However, the different Observation Points can 672 be specified by different Information Element types depending on 673 the Exporter. For example: 675 Exporter1 Observation Point is characterized by the 676 exporterIPv4Address, so a specific Exporter can be represented. 678 Exporter2 Observation Point is characterized by the 679 exporterIPv4Address and a basicList of ingressInterface, so the 680 Exporting Process can express that the observations were made on a 681 series of input interfaces. 683 Exporter3 Observation Point is characterized by the 684 exporterIPv4Address and a specific lineCardId, so the Exporting 685 Process can express that the observation was made on a specific 686 line card. 688 If the mediator models the three different types of Observation 689 Points with the three Template Records below: 691 Template Record 1: exporterIPv4Address 692 Template Record 2: exporterIPv4Address, basicList of 693 ingressInterface 694 Template Record 3: exporterIPv4Address, lineCardId 696 then it can represent the aggregated Observation Point with a 697 subTemplateMultiList and the semantic "exactlyOneOf". The 698 aggregated Observation Point is modeled with the Data Records 699 corresponding to either Template Record 1, Template Record 2, or 700 Template Record 3 but not more than one of these. This implies 701 that the Flow was observed at exactly one of the Observation 702 Points reported. 704 4.4.4. oneOrMoreOf 706 The "oneOrMoreOf" structured data type semantic specifies that one 707 or more elements from the list in the structured data are actual 708 properties of the Data Record. This is equivalent to a logical OR 709 operation. 711 Consider an example where a mediator must report an aggregated 712 Flow (e.g. by aggregating IP addresses from IP prefixes), with an 713 aggregated Observation Point. However, the different Observation 714 Points can be specified by different Information Element types as 715 described in Section 4.4.2. 717 If the mediator models the three different types of Observation 718 Points with the three Template Records below: 720 Template Record 1: exporterIPv4Address 721 Template Record 2: exporterIPv4Address, basicList of 722 ingressInterface 723 Template Record 3: exporterIPv4Address, lineCardId 725 then it can represent the aggregated Observation Point with a 726 subTemplateMultiList and the semantic "oneOrMoreOf". The 727 aggregated Observation Point is modeled with the Data Records 728 corresponding to either Template Record 1, Template Record 2, or 729 Template Record 3. This implies that the Flow was observed on at 730 least one of the Observation Points reported, and potentially on 731 multiple Observation Points. 733 4.4.5. allOf 735 The "allOf" structured data type semantic specifies that all of 736 the list elements from the structured data are actual properties 737 of the Data Record. 739 For example, if a Record contains a basicList of outgoing 740 interfaces with the "allOf" semantic, then the observed Flow is 741 typically a multicast Flow where each packet in the Flow has been 742 replicated to each outgoing interface in the basicList. 744 4.4.6. ordered 746 The "ordered" structured data type semantic specifies that 747 elements from the list in the structured data are ordered. 749 For example, an Exporter might want to export the AS10 AS20 AS30 750 AS40 BGP AS-PATH. In such a case, the Exporter would report a 751 basicList composed of (AS10, AS20, AS30, AS40) and the respective 752 structured data type semantic of "ordered". 754 4.5. Encoding of IPFIX Data Types 756 The following subsections define the encoding of the abstract data 757 types defined in Section 4.1. above. These data types may be 758 encoded using either fixed or variable-length Information 759 Elements, as discussed in Section 5.1. . Like in the IPFIX 760 specifications [RFC5101], all length are specified in octets. 762 4.5.1. basicList 764 The basicList Information Element defined in Section 4.3.1. 765 represents a list of zero or more instances of an Information 766 Element and is encoded as follows: 768 0 1 2 3 769 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 770 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 771 | Semantic |0| Field ID | Element... | 772 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 773 | ...Length | basicList Content ... | 774 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 775 | ... | 776 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 777 | ... | 778 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 780 Figure A: basicList Encoding 782 Semantic 784 The Semantic field indicates the relationship among the 785 different Information Element values within this Structured 786 Data Information Element. Refer to IANA's IPFIX "structured 787 data types semantics registry. 789 Field ID 791 Field ID is the Information Element identifier of the 792 Information Element(s) contained in the list. 794 Element Length 796 Per Section 7 of [RFC5101], the Element Length field indicates 797 the length, in octets, of each list element specified by Field 798 ID, or contains the value 0xFFFF if the length is encoded as a 799 variable-length Information Element at the start of the 800 basicList Content. 802 The Element Length field is effectively part of the header, so 803 even in the case of a zero-element list, it MUST NOT be 804 omitted. 806 basicList Content 808 A Collecting Process decodes list elements from the basicList 809 Content until no further data remains. A field count is not 810 included but can be derived when the Information Element is 811 decoded. 813 Note that in the diagram above, Field ID is shown with the 814 Enterprise bit (most significant bit) set to 0. If instead the 815 Enterprise bit is set to 1, a four-byte Enterprise Number MUST be 816 encoded immediately after the Element Length as shown below. See 817 the "Field Specifier Format" section in the IPFIX Protocol 818 [RFC5101] for additional information. 820 0 1 2 3 821 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 822 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 823 | Semantic |1| Field ID | Element... | 824 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 825 | ...Length | Enterprise Number ... | 826 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 827 | ... | basicList Content ... | 828 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 829 | ... | 830 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 831 Figure B: basicList Encoding with Enterprise Number 833 Also note that, if a basicList has zero elements, the encoded data 834 contains the Semantic field, Field ID, the Element Length field 835 and the four-byte Enterprise Number (if present), while basicList 836 Content is empty. 838 If the basicList is encoded as a variable-length Information 839 Element in less than 255 octets, it MAY be encoded with the Length 840 field per Section 7 of [RFC5101] as shown in Figure C. However, 841 the three-byte length encoding, as shown Figure D, is RECOMMENDED 842 (see section 5.1. ). 844 0 1 2 3 845 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 846 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 847 | Length (< 255)| Semantic |0| Field ID | 848 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 849 | Element Length | basicList Content ... | 850 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 851 | ... | 852 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 853 | ... | 854 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 856 Figure C: Variable-Length basicList Encoding (Length < 255 857 octets) 859 If the basicList is encoded as a variable-length Information 860 Element in 255 or more octets, it MUST be encoded with the Length 861 field per Section 7 of [RFC5101] as follows: 863 0 1 2 3 864 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 865 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 866 | 255 | Length (0 to 65535) | Semantic | 867 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 868 |0| Field ID | Element Length | 869 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 870 | basicList Content ... | 871 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 872 | ... | 873 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 874 | ... | 875 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 877 Figure D: Variable-Length basicList Encoding (Length 0 to 65535 878 octets) 880 4.5.2. subTemplateList 882 The subTemplateList Information Element represents a list of zero 883 or more Data Records corresponding to a specific Template. 884 Because the Template Record referenced by a subTemplateList 885 Information Element can itself contain other subTemplateList 886 Information Elements, and because these Template Record references 887 are part of the Information Elements content in the Data Record, 888 it is possible to represent complex hierarchical data structures. 889 The following diagram shows how a subTemplateList Information 890 Element is encoded within a Data Record: 892 0 1 2 3 893 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 894 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 895 | Semantic | Template ID | ... | 896 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 897 | subTemplateList Content ... | 898 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 899 | ... | 900 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 902 Figure E: subTemplateList Encoding 904 Semantic 906 The Semantic field indicates the relationship among the 907 different Data Records within this Structured Data Information 908 Element. 910 Template ID 912 The Template ID field contains the ID of the template used to 913 encode and decode the subTemplateList Content. 915 subTemplateList Content 916 subTemplateList Content consists of zero or more instances of 917 Data Records corresponding to the Template ID specified in the 918 Template ID Field. A Collecting Process decodes the 919 subTemplateList Content until no further data remains. A 920 record count is not included but can be derived when the 921 subTemplateList is decoded. Encoding and decoding are 922 performed recursively if the specified Template itself 923 contains Structured Data Information Elements as described 924 here. 926 Note that, if a subTemplateList has zero elements, the encoded 927 data contains only the Semantic field and the Template ID field, 928 while subTemplateList Content is empty. 930 If the subTemplateList is encoded as a variable-length Information 931 Element in less than 255 octets, it MAY be encoded with the Length 932 field per Section 7 of [RFC5101] as shown in Figure F. However, 933 the three-byte length encoding, as shown Figure G, is RECOMMENDED 934 (see section 5.1. ). 936 0 1 2 3 937 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 938 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 939 | Length (< 255)| Semantic | Template ID | 940 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 941 | subTemplateList Content ... | 942 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 943 | ... | 944 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 946 Figure F: Variable-Length subTemplateList Encoding (Length < 255 947 octets) 949 If the subTemplateList is encoded as a variable-length Information 950 Element in 255 or more octets, it MUST be encoded with the Length 951 field per Section 7 of [RFC5101] as follows: 953 0 1 2 3 954 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 955 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 956 | 255 | Length (0 to 65535) | Semantic | 957 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 958 | Template ID | subTemplateList Content ... | 959 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 960 | ... | 961 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 963 Figure G: Variable-Length subTemplateList Encoding (Length 0 to 964 65535 octets) 966 4.5.3. subTemplateMultiList 968 Whereas each element in a subTemplateList Information Element 969 corresponds to a single Template, it is sometimes useful for a 970 list to contain elements corresponding to different Templates. To 971 support this case, each top-level element in a 972 subTemplateMultiList Information Element carries a Template ID, 973 Length and zero or more Data Records corresponding to the Template 974 ID. The following diagram shows how a subTemplateMultiList 975 Information Element is encoded within a Data Record. Note that 976 the encoding following the Semantic field is consistent with the 977 Set Header specified in [RFC5101]. 979 0 1 2 3 980 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 981 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 982 | Semantic | Template ID X |Data Records...| 983 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 984 | ... Length X | Data Record X.1 Content ... | 985 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 986 | ... | 987 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 988 | ... | Data Record X.2 Content ... | 989 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 990 | ... | 991 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 992 | ... | Data Record X.L Content ... | 993 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 994 | ... | 995 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 996 | ... | Template ID Y |Data Records...| 997 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 998 | ... Length Y | Data Record Y.1 Content ... | 999 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1000 | ... | 1001 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1002 | ... | Data Record Y.2 Content ... | 1003 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1004 | ... | 1005 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1006 | ... | Data Record Y.M Content ... | 1007 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1008 | ... | 1009 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1010 | ... | Template ID Z |Data Records...| 1011 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1012 | ... Length Z | Data Record Z.1 Content ... | 1013 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1014 | ... | 1015 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1016 | ... | Data Record Z.2 Content ... | 1017 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1018 | ... | 1019 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1020 | ... | Data Record Z.N Content ... | 1021 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1022 | ... | 1023 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1024 | ... | 1025 +-+-+-+-+-+-+-+-+ 1027 Figure H: subTemplateMultiList Encoding 1029 Semantic 1031 The Semantic field indicates the top-level relationship among 1032 the series of Data Records corresponding to the different 1033 Template Records within this Structured Data Information 1034 Element. 1036 Template ID 1038 Unlike the subTemplateList Information Element, each element 1039 of the subTemplateMultiList contains a Template ID which 1040 specifies the encoding of the following Data Records. 1042 Data Records Length 1044 The total length of the Data Records encoding for the Template 1045 ID previously specified, including the 2 bytes for the 1046 Template ID and the 2 bytes for the Data Records Length field 1047 itself. 1049 Data Record X.M 1050 The Data Record X.M consists of the Mth Data Record of the 1051 Template Record X. A Collecting Process decodes the Data 1052 Records according to Template Record X until no further data 1053 remains, according to the Data Records Length X. Further 1054 Template IDs and Data Records may then be decoded according to 1055 the overall subTemplateMultiList length. A record count is 1056 not included but can be derived when the Element Content is 1057 decoded. Encoding and decoding are performed recursively if 1058 the specified Template itself contains Structured Data 1059 Information Elements as described here. 1061 In the exceptional case of zero instances in the 1062 subTemplateMultiList, no data is encoded, only the Semantic field 1063 and Template ID field(s), and the Data Record Length field is set 1064 to zero. 1066 If the subTemplateMultiList is encoded as a variable-length 1067 Information Element in less than 255 octets, it MAY be encoded 1068 with the Length field per Section 7 of [RFC5101] as shown in 1069 Figure I. However, the three-byte length encoding, as shown 1070 Figure J, is RECOMMENDED (see section 5.1. ). 1072 0 1 2 3 1073 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1074 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1075 | Length (< 255)| Semantic | Template ID X | 1076 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1077 | Data Records Length X | Data Record X.1 Content ... | 1078 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1079 | ... | 1080 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1081 | ... | Data Record X.2 Content ... | 1082 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1083 | ... | 1084 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1085 | ... | Data Record X.L Content ... | 1086 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1087 | ... | 1088 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1089 | ... | Template ID Y | 1090 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1091 | Data RecordsLength Y | Data Record Y.1 Content ... | 1092 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1093 | ... | 1094 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1095 | ... | Data Record Y.2 Content ... | 1096 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1097 | ... | 1098 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1099 | ... | Data Record Y.M Content ... | 1100 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1101 | ... | 1102 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1103 | ... | Template ID Z | 1104 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1105 | Data Records Length Z | Data Record Z.1 Content ... | 1106 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1107 | ... | 1108 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1109 | ... | Data Record Z.2 Content ... | 1110 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1111 | ... | 1112 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1113 | ... | Data Record Z.N Content ... | 1114 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1115 | ... | 1116 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1117 | ... | 1118 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1120 Figure I: Variable-Length subTemplateMultiList Encoding (Length < 1121 255 octets) 1123 If the subTemplateMultiList is encoded as a variable-length 1124 Information Element in 255 or more octets, it MUST be encoded with 1125 the Length field per Section 7 of [RFC5101] as follows: 1127 0 1 2 3 1128 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1129 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1130 | 255 | Length (0 to 65535) | Semantic | 1131 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1132 | Template ID X | Data Records Length X | 1133 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1134 | Data Record X.1 Content ... | 1135 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1136 | ... | 1137 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1138 | Data Record X.2 Content ... | 1139 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1140 | ... | 1141 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1142 | Data Record X.L Content ... | 1143 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1144 | ... | 1145 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1146 | Template ID Y | Data Records Length Y | 1147 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1148 | Data Record Y.1 Content ... | 1149 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1150 | ... | 1151 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1152 | Data Record Y.2 Content ... | 1153 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1154 | ... | 1155 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1156 | Data Record Y.M Content ... | 1157 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1158 | ... | 1159 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1160 | Template ID Z | Data Records Length Z | 1161 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1162 | Data Record Z.1 Content ... | 1163 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1164 | ... | 1165 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1166 | Data Record Z.2 Content ... | 1167 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1168 | ... | 1169 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1170 | Data Record Z.N Content ... | 1171 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1172 | ... | 1173 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1175 Figure J: Variable-Length subTemplateMultiList Encoding (Length 1176 0 to 65535 octets) 1178 5. Structured Data Format 1179 5.1. Length Encoding Considerations 1181 The new Structured Data Information Elements represent a list that 1182 potentially carries complex hierarchical and repeated data. 1184 When the encoding of a Structured Data Information Element has a 1185 fixed length (because, for example, it contains the same number of 1186 fixed-length elements, or if the permutations of elements in the 1187 list always produces the same total length), the element length 1188 can be encoded in the corresponding Template Record. 1190 However, when representing variable-length data, hierarchical 1191 data, and repeated data with variable element counts, where the 1192 number and length of elements can vary from record to record, we 1193 RECOMMEND that the Information Elements are encoded using the 1194 variable-length encoding described in Section 7 of [RFC5101], with 1195 the length carried before the Structured Data Information Element 1196 encoding. 1198 Because of the complex and repeated nature of the data, it is 1199 potentially difficult for the Exporting Process to efficiently 1200 know in advance the exact encoding size. In this case, the 1201 Exporting Process may encode the available data starting at a 1202 fixed offset and fill in the final length afterwards. Therefore, 1203 the three-byte length encoding is RECOMMENDED for variable-length 1204 information elements in all Template Records containing a 1205 Structured Data Information Element, even if the encoded length 1206 can be less than 255 bytes, because the starting offset of the 1207 data is known in advance. 1209 When encoding such data, an Exporting Process MUST take care to 1210 not exceed the maximum allowed IPFIX message length of 65535 bytes 1211 as specified in [RFC5101]. 1213 5.2. Recursive Structured Data 1215 It is possible to define recursive relationships between IPFIX 1216 structured data instances, for example when representing a tree 1217 structure. The simplest case of this might be a basicList where 1218 each element is itself a basicList, or a subTemplateList where one 1219 of the fields of the referenced template is itself a 1220 subTemplateList referencing the same Template. Also, the 1221 Exporting Process MUST take care when encoding recursively-defined 1222 structured data, not to exceed the maximum allowed length of an 1223 IPFIX Message (as noted in Length Encoding Considerations). 1225 5.3. Structured Data Information Elements Applicability in Options 1226 Template Sets 1228 Structured Data Information Elements MAY be used in Options 1229 Template Sets. 1231 As an example, consider a mediation function that must aggregate 1232 Data Records from multiple Observation Point types: 1234 Router 1, (interface 1) 1235 Router 2, (line card A) 1236 Router 3, (line card B) 1237 Router 4, (line card C, interface 2) 1239 In order to encode the PSAMP Selection Sequence Report 1240 Interpretation [RFC5476], the mediation function must express this 1241 combination of Observation Points as a single new Observation 1242 Point. Recall from [RFC5476] that the PSAMP Selection Sequence 1243 Report Interpretation consists of the following fields: 1245 Scope: selectionSequenceId 1246 Non-Scope: one Information Element mapping the Observation Point 1247 selectorId (one or more) 1249 Without structured data, there is clearly no way to express the 1250 complex aggregated Observation Point as "one Information Element 1251 mapping the Observation Point". However, the desired result may 1252 be easily achieved using the structured data types. Refer to 1253 Section 9.5. for an encoding example related to this case study. 1255 Regarding the scope in the Options Template Record, the IPFIX 1256 specification [RFC5101] mentions that "The IPFIX protocol doesn't 1257 prevent the use of any Information Elements for scope". 1258 Therefore, a Structured Data Information Element MAY be used as 1259 scope in an Options Template Set. 1261 Extending the previous example, the mediation function could 1262 export a given name for this complex aggregated Observation Point: 1264 Scope: Aggregated Observation Point (structured data) 1265 Non-Scope: a new Information Element containing the name 1267 5.4. Usage Guidelines for Equivalent Data Representations 1269 Because basicList, subTemplateList, and subTemplateMultiList are 1270 all lists, in several cases there is more than one way to 1271 represent what is effectively the same data structure. However, 1272 in some cases, one approach has an advantage over the other e.g. 1273 more compact, uses fewer resources, etc., and is therefore 1274 preferred over an alternate representation. 1276 A subTemplateList can represent the same simple list of single- 1277 value Information Elements as a basicList, if the Template 1278 referenced by the subTemplateList contains only one single-valued 1279 Information Element. Although the encoding is more compact than a 1280 basicList by two bytes, using a subTemplateList in this case 1281 requires a new Template per Information Element. The basicList 1282 requires no additional Template and is therefore RECOMMENDED in 1283 this case. 1285 Although a subTemplateMultiList with one Element can represent the 1286 contents of a subTemplateList, the subTemplateMultiList carries 1287 two additional bytes (Element Length). It is also potentially 1288 useful to a Collecting Process to know in advance that a 1289 subTemplateList directly indicates that list element types are 1290 consistent. The subTemplateList Information Element is therefore 1291 RECOMMENDED in this case. 1293 The Semantic field in a subTemplateMultiList indicates the top- 1294 level relationship among the series of Data Records corresponding 1295 to the different Template Records, within this Structured Data 1296 Information Element. If a semantic is required to describe the 1297 relationship among the different Data Records corresponding to a 1298 single Template ID within the subTemplateMultiList, then an 1299 encoding based on a basicList of subTemplateLists should be used, 1300 refer to Section 5.6 for more information. Alternatively, if a 1301 semantic is required to describe the relationship among all Data 1302 Records within a subTemplateMultiList (regardless of the Template 1303 Record), an encoding based on a subTemplateMultiList with one Data 1304 Record corresponding to a single Template ID can be used. 1306 Note that the referenced Information Element(s) in the Structured 1307 Data Information Elements can be taken from the IPFIX information 1308 model [RFC5102], the PSAMP information model [RFC5477], any of the 1309 Information Elements defined in the IANA IPFIX registry [IANA- 1310 IPFIX] or enterprise-specific Information Elements. 1312 If a Template Record contains a subTemplateList as the only field, 1313 a Set encoding as specified in the IPFIX protocol specifications 1315 [RFC5101] should be considered, unless: 1316 - A relationship among multiple list elements must be exported, 1317 in which case, the semantic from the IPFIX Structured Data 1318 Information Element can convey this relationship. 1319 - The Exporting Process wants to convey the number of elements in 1320 the list, even in the special cases of zero or one element in the 1321 list. Indeed, the case of an empty list cannot be represented with 1322 the IPFIX protocol specifications [RFC5101]. In the case of a 1323 single element list, the Template Record specified in the IPFIX 1324 protocol specification [RFC5101] could be used. However, on the 1325 top of the Template Record with the subTemplateList to export 1326 multiple list elements, this supplementary Template would impose 1327 some extra management, both on the Exporting Process and on the 1328 Collecting Process, which might have to correlate the information 1329 from two Template Records. 1331 Similarly, if a Template Record contains a subTemplateMultiList as 1332 the only field, an IPFIX Message as described in the IPFIX 1333 protocol specification [RFC5101] should be considered, unless: 1334 - A relationship among top-level list elements must be exported, 1335 in which case, the semantic from the IPFIX Structured Data 1336 Information Element can convey this relationship. 1337 - The Exporting Process wants to convey the number of Data Records 1338 corresponding to every Template in the subTemplateMultiList. 1340 5.5. Padding 1342 The Exporting Process MAY insert some padding octets in structured 1343 data field values in a Data Record by including the 1344 'paddingOctets' Information Element as described in [RFC5101] 1345 Section 3.3.1. The paddingOctets Information Element can be 1346 included in a Template Record referenced by structured data 1347 Information Element for this purpose. 1349 5.6. Semantic 1351 Semantic interpretations of received Data Records at or beyond the 1352 Collecting Process remain explicitly undefined, unless that data 1353 is transmitted using this extension with explicit Structured Data 1354 type semantic information. 1356 It is not the Exporter's role to check the validity of the 1357 semantic representation of Data Records. 1359 More complex semantics can be expressed as a combination of the 1360 Semantic Data Information Elements specified in this document. 1362 For example, the export of the AS10 AS20 AS30 AS40 {AS50,AS60} BGP 1363 AS-PATH would be reported as a basicList of two elements, each 1364 element being a basicList of BGP AS, with the top level structured 1365 data type semantic of "ordered". The first element would contain 1366 a basicList composed of (AS10,AS20,AS30,AS40) and the respective 1367 structured data type semantic of "ordered", while the second 1368 element would contain a basicList composed of (AS50, AS60) and the 1369 respective structured data type semantic of "exactlyOneOf". A 1370 high level Data Record diagram would be represented as: 1372 BGP AS-PATH = (basicList, ordered, 1374 (basicList, ordered, AS10,AS20,AS30,AS40), 1376 (basicList, exactlyOneOf, AS50, AS60) 1378 ) 1380 If a semantic is required to describe the relationship among the 1381 different Data Records corresponding to a single Template ID 1382 within the subTemplateMultiList, then an encoding based on a 1383 basicList of subTemplateLists should be used, as shown in the next 1384 case study. 1386 Case study 1: 1388 In this example, an Exporter monitoring security attacks must 1389 export a list of security events consisting of attackers and 1390 targets. For the sake of the example, assume that the Collector 1391 can differentiate the attacker (which is expressed using source 1392 fields) from the target (which is expressed using destination 1393 fields). Imagine that attackers A1 or A2 may attack targets T1 1394 and T2. 1396 The first case uses a subTemplateMultiList composed of two 1397 Template Records, one representing the attacker and one 1398 representing the target, each of them containing an IP address and 1399 a port. 1401 Attacker Template Record = (src IP address, src port) 1403 Target Template Record = (dst IP address, dst port) 1405 A high level Data Record diagram would be represented as: 1407 Alert = (subTemplateMultiList, allOf, 1409 (Attacker Template Record, A1, A2), 1411 (Target Template Record, T1, T2) 1413 ) 1415 The Collecting Process can only conclude that the list of 1416 attackers (A1, A2) and the list of targets (T1, T2) are present, 1417 without knowing the relationship amongst attackers and targets. 1418 The Exporting Process would have to explicitly call out the 1419 relationship amongst attackers and targets as the top level 1420 semantic offered by the subTemplateMultiList isn't sufficient. 1422 The only proper encoding for the previous semantic (i.e. attacker 1423 A1 or A2 may attack target T1 and T2) uses a basicList of 1424 subTemplateLists and is represented as follows: 1426 Attacker Template Record = (src IP address, src port) 1428 Target Template Record = (dst IP address, dst port) 1430 Alert = (basicList, allof, 1432 (subTemplateList, exactlyOneOf, attacker A1, A2) 1434 (subTemplateList, allOf, target T1, T2) 1436 ) 1438 Case study 2: 1440 In this example, an Exporter monitoring security attacks must 1441 export a list of attackers and targets. For the sake of the 1442 example, assume that the Collector can differentiate the attacker 1443 (which is expressed using source fields) from the target (which is 1444 expressed using destination fields). Imagine that attackers A1 or 1445 A2 are attacking target T1, while attacker A3 is attacking targets 1446 T2 and T3. The first case uses a subTemplateMultiList that 1447 contains Data Records corresponding to two Template Records, one 1448 representing the attacker and one representing the target, each of 1449 them containing an IP address and a port. 1451 Attacker Template Record = (src IP address, src port) 1452 Target Template Record = (dst IP address, dst port) 1454 A high level Data Record diagram would be represented as: 1456 Alert = (subTemplateMultiList, allOf, 1458 (Attacker Template Record, A1, A2, A3), 1460 (Target Template Record, T1, T2, T3) 1462 ) 1464 The Collecting Process can only conclude that the list of 1465 attackers (A1, A2, A3) and the list of targets (T1, T2, T3) are 1466 present, without knowing the relationship amongst attackers and 1467 targets. 1469 The second case could use a Data Record definition composed of the 1470 following: 1472 Alert = (subTemplateMultiList, allOf, 1474 (Attacker Template Record, A1, A2), 1476 (Target Template Record, T1), 1478 (Attacker Template Record, A3), 1480 (Target Template Record, T2, T3) 1482 ) 1484 With the above representation, the Collecting Process can infer 1485 that the alert consists of the list of attackers (A1, A2), target 1486 (T1), attacker (A3) and list of targets (T2, T3). From the 1487 sequence in which attackers and targets are encoded, the Collector 1488 can possibly deduce that some relationship exists among (A1, A2, 1489 T1) and (A2, T1, T2) but cannot understand what it is exactly. 1490 So, there is a need for the Exporting Process to explicitly define 1491 the relationship between the attackers and targets and the top- 1492 level semantic of the subTemplateMultiList is not sufficient. 1494 The only proper encoding for the previous semantic (i.e. attacker 1495 A1 or A2 attack target T1, attacker A3 attacks targets T2 and T3) 1496 uses a basicList of subTemplateLists and is represented as 1497 follows: 1499 Participant P1 = 1501 (basicList, allOf, 1503 (subTemplateList, exactlyOneOf, attacker A1, A2) 1505 (subTemplateList, undefined, target T1) 1507 ) 1509 Participant P2 = 1511 (basicList, allOf, 1513 (subTemplateList, undefined, attacker A3, 1515 (subTemplateList, allOf, targets T2, T3) 1517 ) 1519 The security alert is represented as a subTemplateList of 1520 participants. 1522 Alert = 1524 (subTemplateList, allOf, Participant P1, Participant P2) 1526 Note that, in the particular case of a single element in a 1527 Structured Data Information Element, the semantic field is 1528 actually not very useful since it specifies the relationship among 1529 multiple elements. Any choice of allOf, exactlyOneOr, or 1530 OneOrMoreOf would provide the same result semantically. 1531 Therefore, in case of a single element in a Structured Data 1532 Information Element, the default "undefined" semantic SHOULD be 1533 used. 1535 6. Template Management 1537 This section introduces some more specific Template management and 1538 Template Withdrawal Message-related specifications compared to the 1539 IPFIX protocol specification [RFC5101]. 1541 First of all, the Template ID uniqueness is unchanged compared to 1542 [RFC5101]; the uniqueness is local to the Transport Session and 1543 Observation Domain that generated the Template ID. In other 1544 words, the Set ID used to export the Template Record does not 1545 influence the Template ID uniqueness. 1547 While [RFC5101] mentions that: "If an Information Element is 1548 required more than once in a Template, the different occurrences 1549 of this Information Element SHOULD follow the logical order of 1550 their treatments by the Metering Process.", this rule MAY be 1551 ignored within Structured Data Information Elements. 1553 As specified in [RFC5101], Templates that are not used anymore 1554 SHOULD be deleted. Deleting a Template implies that it MUST NOT 1555 be used within subTemplateList and subTemplateMultiList any more. 1556 Before reusing a Template ID, the Template MUST be deleted. In 1557 order to delete an allocated Template, the Template is withdrawn 1558 through the use of a Template Withdrawal Message. 1560 7. The Collecting Process's Side 1562 This section introduces some more specific specifications to the 1563 Collection Process compared to Section 9 in the IPFIX Protocol 1564 [RFC5101]. 1566 As opposed to the IPFIX specification in [RFC5101], IPFIX Messages 1567 with IPFIX Structured Data Information Elements change the IPFIX 1568 concept from the Collector's point of view as the data types are 1569 present in the Data Records rather than in the Template Records. 1570 For example, a basicList Information Element in a Template Record 1571 doesn't specify the list element data type, this information is 1572 contained in the Data Record. For example, in case of a 1573 subTemplateMultiList, the Collecting Process must refer to the 1574 included Template Records in the middle of the Data Record decode. 1576 As described in [RFC5101], a Collecting Process MUST note the 1577 Information Element identifier of any Information Element that it 1578 does not understand and MAY discard that Information Element from 1579 the Flow Record. Therefore a Collection Process that does not 1580 support the extension specified in this document can ignore the 1581 Structured Data Information Elements in a Data Record, or it can 1582 ignore Data Records containing these new Structured Data 1583 Information Elements while continuing to process other Data 1584 Records. 1586 If the structured data contains the "undefined" structured data 1587 type semantic, the Collecting Process MAY attempt to draw its own 1588 conclusion in terms of the semantic contained in the Data Record. 1590 8. Defining New Information Elements Based on the New Abstract Data 1591 Types 1593 This document specifies three new abstract data types: basicList, 1594 subTemplateList, and subTemplateMultiList. As specified in 1595 [RFC5102], the specification of new IPFIX Information Elements 1596 uses the template specified in Section 2.1 of [RFC5102]. This 1597 template mentioned existing and future the data types: "One of the 1598 types listed in Section 3.1 of this document or in a future 1599 extension of the information model." So new Information Elements 1600 can be specified based on the three new abstract data types. 1602 The authors anticipate the creation of both enterprise-specific 1603 and IANA Information Elements based on the IPFIX structured data 1604 types. For example, bgpPathList, bgpSequenceList and bgpSetList, 1605 of abstract types and semantics basicList/ordered, 1606 basicList/ordered, and basicList/exactlyOneOf respectively would 1607 define the complete semantic of the list. This specification 1608 doesn't specify any new Information Elements beyond the ones in 1609 Section 4.3. 1611 9. Structured Data Encoding Examples 1613 The following examples are created solely for the purpose of 1614 illustrating how the extensions proposed in this document are 1615 encoded. 1617 9.1. Encoding a Multicast Data Record with basicList 1619 Consider encoding a multicast Data Record containing the following 1620 data: 1622 --------------------------------------------------------------- 1623 Ingress If | Source IP | Destination IP | Egress Interfaces 1624 --------------------------------------------------------------- 1625 9 192.0.2.201 233.252.0.1 1, 4, 8 1626 --------------------------------------------------------------- 1628 Template Record for the multicast Flows, with the Template ID 256: 1630 0 1 2 3 1631 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1633 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1634 | Set ID = 2 | Length = 24 octets | 1635 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1636 | Template ID = 256 | Field Count = 4 | 1637 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1638 |0| ingressInterface = 10 | Field Length = 4 | 1639 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1640 |0| sourceIPv4Address = 8 | Field Length = 4 | 1641 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1642 |0| DestinationIPv4Address = 12 | Field Length = 4 | 1643 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1644 |0| basicList = XXX | Field Length = 0xFFFF | 1645 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1647 Figure K: Encoding basicList, Template Record 1649 The list of outgoing interfaces is represented as a basicList with 1650 semantic allOf, and the Length of the list is chosen to be encoded 1651 in three bytes even though it may be less than 255 octets. 1653 The Data Set is represented as follows: 1655 0 1 2 3 1656 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1657 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1658 | Set ID = 256 | Length = 36 | 1659 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1660 | ingressInterface = 9 | 1661 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1662 | sourceIPv4Address = 192.0.2.201 | 1663 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1664 | DestinationIPv4Address = 233.252.0.1 | 1665 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1666 | 255 | List Length = 17 | semantic=allOf| 1667 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1668 | egressInterface FieldId = 14 |egressInterface Field Length=4 | 1669 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1670 | egressInterface value 1 = 1 | 1671 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1672 | egressInterface value 2 = 4 | 1673 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1674 | egressInterface value 3 = 8 | 1675 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1676 Figure L: Encoding basicList, Data Record, Semantic allOf 1678 In the example above, the basicList contains fixed-length 1679 elements. To illustrate how variable-length elements would be 1680 encoded, the same example is shown below with variable-length 1681 interface names in the basicList instead: 1683 0 1 2 3 1684 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1685 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1686 | Set ID = 256 | Length = 44 | 1687 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1688 | ingressInterface = 9 | 1689 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1690 | sourceIPv4Address = 192.0.2.201 | 1691 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1692 | DestinationIPv4Address = 233.252.0.1 | 1693 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1694 | 255 | List Length = 25 | semantic=allOf| 1695 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1696 |0| InterfaceName FieldId = 82 | InterfaceName Field Len=0xFFFF| 1697 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1698 | Length = 5 | 'F' | 'E' | '0' | 1699 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1700 | '/' | '0' | Length = 7 | 'F' | 1701 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1702 | 'E' | '1' | '0' | '/' | 1703 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1704 | '1' | '0' | Length = 5 | 'F' | 1705 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1706 | 'E' | '2' | '/' | '2' | 1707 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1709 Figure M: Encoding basicList, Data Record with Variable-Length 1710 Elements, Semantic allOf 1712 9.2. Encoding a Load-balanced Data Record with a basicList 1714 Consider encoding a load-balanced Data Record containing the 1715 following data: 1717 --------------------------------------------------------------- 1718 Ingress If | Source IP | Destination IP | Egress Interfaces 1719 --------------------------------------------------------------- 1720 9 192.0.2.201 233.252.0.1 1, 4, 8 1721 --------------------------------------------------------------- 1723 So the Data Record egressed from either interface 1, 4, or 8. The 1724 Data Set is represented as follows: 1726 0 1 2 3 1727 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1728 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1729 | Set ID = 256 | Length = 36 | 1730 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1731 | ingressInterface = 9 | 1732 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1733 | sourceIPv4Address = 192.0.2.201 | 1734 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1735 | DestinationIPv4Address = 233.252.0.1 | 1736 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1737 | 255 | List Length = 17 |sem=exactlyOne | 1738 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1739 | egressInterface FieldId = 14 |egressInterface Field Length=4 | 1740 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1741 | egressInterface value 1 = 1 | 1742 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1743 | egressInterface value 2 = 4 | 1744 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1745 | egressInterface value 3 = 8 | 1746 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1748 Figure N: Encoding basicList, Data Record, Semantic ExactlyOneOf 1750 9.3. Encoding subTemplateList 1752 As explained in Section 2.2. , multiple pairs of 1753 (observationTimeMicroseconds, digestHashValue) must be collected 1754 from two different Observation Points to passively compute the 1755 one-way delay across the network. This data can be exported with 1756 an optimized Data Record that consists of the following 1757 attributes: 1759 5-tuple 1760 { observationTimeMicroseconds 1, digestHashValue 1 } 1761 { observationTimeMicroseconds 2, digestHashValue 2 } 1762 { observationTimeMicroseconds 3, digestHashValue 3 } 1763 { ... , ... } 1765 A subTemplateList is best suited for exporting the list of 1766 (observationTimeMicroseconds, digestHashValue). For illustration 1767 purposes, the number of elements in the list is 5; in practice, it 1768 could be more. 1770 ------------------------------------------------------------------ 1771 srcIP | dstIP | src | dst |proto| one-way delay 1772 | | Port | Port | | metrics 1773 ------------------------------------------------------------------ 1774 192.0.2.1 192.0.2.105 1025 80 6 Time1, 0x0x91230613 1775 Time2, 0x0x91230650 1776 Time3, 0x0x91230725 1777 Time4, 0x0x91230844 1778 Time5, 0x0x91230978 1779 ------------------------------------------------------------------ 1781 The following Template is defined for exporting the one-way delay 1782 metrics: 1784 0 1 2 3 1785 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1786 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1787 | Set ID = 2 | Length = 16 octets | 1788 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1789 | Template ID = 257 | Field Count = 2 | 1790 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1791 |0| observationTimeMicroSec=324 | Field Length = 8 | 1792 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1793 |0| digestHashValue = 326 | Field Length = 4 | 1794 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1796 Figure O: Encoding subTemplateList, Template for One-Way Delay 1797 Metrics 1799 The Template Record for the Optimized Data Record is as follows: 1801 0 1 2 3 1802 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1803 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1804 | Set ID = 2 | Length = 32 octets | 1805 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1806 | Template ID = 258 | Field Count = 6 | 1807 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1808 |0| sourceIPv4Address = 8 | Field Length = 4 | 1809 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1810 |0| destinationIPv4Address = 12 | Field Length = 4 | 1811 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1812 |0| sourceTransportPort = 7 | Field Length = 2 | 1813 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1814 |0| destinationTransportPort= 11| Field Length = 2 | 1815 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1816 |0| protocolIdentifier = 4 | Field Length = 1 | 1817 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1818 |0| subTemplateList = YYY | Field Length = 0xFFFF | 1819 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1821 Figure P: Encoding subTemplateList, Template Record 1823 The list of (observationTimeMicroseconds, digestHashValue) is 1824 exported as a subTemplateList with semantic allOf. The Length of 1825 the subTemplatelist is chosen to be encoded in three bytes even 1826 though it may be less than 255 octets. 1828 The Data Record is represented as follows: 1830 0 1 2 3 1831 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1832 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1833 | Set ID = 258 | Length = 83 octets | 1834 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1835 | sourceIPv4Address = 192.0.2.1 | 1836 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1837 | destinationIPV4Address = 192.0.2.105 | 1838 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1839 | sourceTransportPort = 1025 | destinationTransportPort = 80 | 1840 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1841 | Protocol = 6 | 255 | one-way metrics list len = 63 | 1842 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1843 | semantic=allOf| TemplateID = 257 | TimeValue1 | 1844 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1845 | ... octets 2-5 of TimeValue1 | 1846 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1847 | ... octets 6-8 of TimeValue1 |digestHashVal1=| 1848 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1849 | ... 0x0x91230613 | TimeValue2 | 1850 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1851 | ... octets 2-5 of TimeValue2 | 1852 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1853 | ... octets 6-8 of TimeValue2 |digestHashVal2=| 1854 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1855 | ... 0x0x91230650 | TimeValue3 | 1856 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1857 | ... octets 2-5 of TimeValue3 | 1858 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1859 | ... octets 6-8 of TimeValue3 |digestHashVal3=| 1860 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1861 | ... 0x0x91230725 | TimeValue4 | 1862 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1863 | ... octets 2-5 of TimeValue4 | 1864 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1865 | ... octets 6-8 of TimeValue4 |digestHashVal4=| 1866 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1867 | ... 0x0x91230844 | TimeValue5 | 1868 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1869 | ... octets 2-5 of TimeValue5 | 1870 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1871 | ... octets 6-8 of TimeValue5 |digestHashVal5=| 1872 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1873 | ... 0x0x91230978 | 1874 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1876 Figure Q: Encoding subTemplateList, Data Set 1878 9.4. Encoding subTemplateMultiList 1880 As explained in Section 4.5.3., a subTemplateMultiList is used to 1881 export a list of mixed-type content where each top level element 1882 corresponds to a different Template Record. 1884 To illustrate this, consider the Data Record with the following 1885 attributes: 1887 5-tuple (Flow Keys), octetCount, packetCount 1888 attributes for filtering 1889 selectorId, 1890 selectorAlgorithm 1891 attributes for sampling 1892 selectorId, 1893 selectorAlgorithm, 1894 samplingPacketInterval, 1895 samplingPacketSpace 1897 This example demonstrates that the Selector Report Interpretation 1898 [RFC5476] can be encoded with the subTemplateMultiList. More 1899 specifically, the example describes Property Match Filtering 1900 Selector Report Interpretation [RFC5476] used for filtering 1901 purposes, and the Systemic Count-Based Sampling as described in 1902 Section 6.5.2.1 of [RFC5476]. Some traffic will be filtered 1903 according to match properties configured, some will be sampled, 1904 some will be filtered and sampled, and some will not be filtered or 1905 be sampled. 1907 A subTemplateMultiList is best suited for exporting this variable 1908 data. A Template is defined for filtering attributes and another 1909 Template is defined for sampling attributes. A Data Record can 1910 contain data corresponding to either of the Templates, both of 1911 them, or neither of them. 1913 Consider the example below where the following Data Record contains 1914 both filtering and sampling attributes. 1916 Key attributes of the Data Record: 1918 ------------------------------------------------------------------ 1919 srcIP | dstIP | src | dst | proto | octetCount | packet 1920 | | Port | Port | | | Count 1921 ------------------------------------------------------------------ 1922 2001:DB8::1 2001:DB8::2 1025 80 6 108000 120 1923 ------------------------------------------------------------------ 1925 Filtering attributes: 1927 ------------------------------------------- 1928 selectorId | selectorAlgorithm 1929 ------------------------------------------- 1930 100 5 (Property Match Filtering) 1931 ------------------------------------------- 1933 Sampling attributes: 1935 For Systemic Count-Based Sampling as defined in Section 6.5.2.1 of 1936 [RFC5476] the required algorithm-specific Information Elements are: 1938 samplingPacketInterval: number of packets selected in a row 1939 samplingPacketSpace: number of packets between selections 1941 Example of a simple 1 out-of 100 systematic count-based Selector 1942 definition, where the samplingPacketInterval is 1 and the 1943 samplingPacketSpace is 99. 1945 -------------------------------------------------------------- 1946 selectorId | selectorAlgorithm | sampling | sampling 1947 | | Packet | Packet 1948 | | Interval | Space 1949 -------------------------------------------------------------- 1950 15 1 (Count-Based Sampling) 1 99 1951 -------------------------------------------------------------- 1953 To represent the Data Record, the following Template Records are 1954 defined: 1956 Template for filtering attributes: 259 1957 Template for sampling attributes: 260 1958 Template for Flow Record: 261 1960 Flow record (261) 1961 | (sourceIPv6Address) 1962 | (destinationIPv6Address) 1963 | (sourceTransportPort) 1964 | (destinationTransportPort) 1965 | (protocolIdentifier) 1966 | (octetTotalCount) 1967 | (packetTotalCount) 1968 | 1969 +------ filtering attributes (259) 1970 | (selectorId) 1971 | (selectorAlgorithm) 1972 | 1973 +------ sampling attributes (260) 1974 | (selectorId) 1975 | (selectorAlgorithm) 1976 | (samplingPacketInterval) 1977 | (samplingPacketSpace) 1979 The following Template Record is defined for filtering attributes: 1981 0 1 2 3 1982 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1983 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1984 | Set ID = 2 | Length = 16 | 1985 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1986 | Template ID = 259 | Field Count = 2 | 1987 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1988 |0| selectorId = 302 | Field Length = 4 | 1989 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1990 |0| selectorAlgorithm = 304 | Field Length = 1 | 1991 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1993 Figure R: Encoding subTemplateMultiList, Template for Filtering 1994 Attributes 1996 The Template for sampling attributes is defined as follows: 1998 0 1 2 3 1999 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2000 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2001 | Set ID = 2 | Length = 24 | 2002 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2003 | Template ID = 260 | Field Count = 4 | 2004 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2005 |0| selectorId = 302 | Field Length = 4 | 2006 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2007 |0| selectorAlgorithm = 304 | Field Length = 1 | 2008 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2009 |0| samplingPacketInteval = 305 | Field Length = 1 | 2010 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2011 |0| samplingPacketSpace = 306 | Field Length = 1 | 2012 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2014 Figure S: Encoding subTemplateMultiList, Template for Sampling 2015 Attributes 2017 Note that while selectorAlgorithm is defined as unsigned16, and 2018 samplingPacketInterval and samplingPacketSpace are defined as 2019 unsigned32, they are compressed down to 1 octet here as allowed 2020 by Reduced Size Encoding in Section 6.2 of the IPFIX protocol 2021 specifications [RFC5101]. 2023 Template for the Flow Record is defined as shown below: 2025 0 1 2 3 2026 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2027 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2028 | Set ID = 2 | Length = 40 | 2029 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2030 | Template ID = 261 | Field Count = 8 | 2031 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2032 |0| sourceIPv6Address = 27 | Field Length = 16 | 2033 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2034 |0| destinationIPv6Address = 28 | Field Length = 16 | 2035 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2036 |0| sourceTransportPort = 7 | Field Length = 2 | 2037 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2038 |0| destinationTransportPort=11 | Field Length = 2 | 2039 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2040 |0| protocolIdentifier = 4 | Field Length = 1 | 2041 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2042 |0| octetTotalCount = 85 | Field Length = 4 | 2043 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2044 |0| packetTotalCount = 86 | Field Length = 4 | 2045 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2046 |0| subTemplateMultiList = ZZZ | Field Length = 0XFFFF | 2047 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2049 Figure T: Encoding subTemplateMultiList, Template for Flow Record 2051 A subTemplateMultiList with semantic allOf is used to export the 2052 filtering and sampling attributes. The Length field of the 2053 subTemplateMultilist is chosen to be encoded in three bytes even 2054 though it may be less than 255 octets. 2056 The Data Record is encoded as follows: 2058 0 1 2 3 2059 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2060 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2061 | Set ID = 261 | Length = 73 | 2062 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2063 | sourceIPv6Address = ... | 2064 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2065 | ... | 2066 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2067 | ... | 2068 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2069 | 2001:DB8::1 | 2070 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2071 | destinationIPv6Address = ... | 2072 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2073 | ... | 2074 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2075 | ... | 2076 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2077 | 2001:DB8::2 | 2078 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2079 | sourceTransportPort = 1025 | destinationTransportPort = 80 | 2080 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2081 | protocol = 6 | octetTotalCount = 108000 | 2082 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2083 | ... | packetTotalCount = 120 | 2084 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2085 | ... | 255 | Attributes List Length = 21 | 2086 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2087 |semantic=allOf | Filtering Template ID = 259 |Filtering Attr | 2088 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2089 | ...Length = 9 | selectorId = ... | 2090 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2091 | ... 100 |selectorAlg = 5| Sampling Template ID = 260 | 2092 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2093 | Sampling Attributes Length=11 | selectorId = ... | 2094 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2095 | ... 15 |selectorAlg = 1| Interval = 1 | 2096 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2097 | Space = 99 | 2098 +-+-+-+-+-+-+-+-+ 2100 Figure U: Encoding subTemplateMultiList, Data Set 2102 9.5. Encoding an Options Template Set using Structured Data 2104 As described in Section 5.3. , consider a mediation function that 2105 must aggregate Data Records from different Observation Points. 2107 Say Observation Point 1 consists of one or more interfaces, 2108 Observation Points 2 and 3 consist of one or more line cards, and 2109 Observation Point 4 consists of one or more interfaces and one or 2110 more line cards. Without structured data, a template would have 2111 to be defined for every possible combination to interpret the data 2112 corresponding to each of the Observation Points. However, with 2113 structured data, a basicList can be used to encode the list of 2114 interfaces and another basicList can be used to encode the list of 2115 line cards. 2117 For the sake of simplicity, each Observation Point shown below has 2118 the IP address corresponding to the Router and an or 2119 or . This can very well be 2120 extended to include a list of interfaces and a list of linecards 2121 using basicLists as explained above. 2123 Observation Point 1: Router 1, (interface 1) 2124 Observation Point 2: Router 2, (line card A) 2125 Observation Point 3: Router 3, (line card B) 2126 Observation Point 4: Router 4, (line card C, interface 2) 2128 The mediation function wishes to express this as a single 2129 Observation Point, in order to encode the PSAMP Selection 2130 Sequence Report Interpretation (SSRI). Recall from [RFC5476] 2131 that the PSAMP Selection Sequence Report Interpretation 2132 consists of the following fields: 2134 Scope: selectionSequenceId 2135 Non-Scope: one Information Element mapping the 2136 Observation Point 2137 selectorId (one or more) 2139 For example, the Observation Point detailed above may be 2140 encoded in a PSAMP Selection Sequence Report Interpretation as 2141 shown below: 2143 Selection Sequence 7 (Filter->Sampling): 2144 Observation Point: subTemplateMultiList. 2145 Router 1 (IP address = 192.0.2.11), (interface 1) 2146 Router 2 (IP address = 192.0.2.12), (line card A) 2147 Router 3 (IP address = 192.0.2.13), (line card B) 2148 Router 4 (IP address = 192.0.2.14), (line card C, interface 2) 2149 selectorId: 5 (Filter, match IPV4SourceAddress 192.0.2.1) 2150 selectorId: 10 (Sampler, Random 1 out-of ten) 2152 The following Templates are defined to represent the PSAMP SSRI: 2153 Template for representing PSAMP SSRI: 262 2154 Template for representing interface: 263 2155 Template for representing linecard: 264 2156 Template for representing linecard and interface: 265 2158 PSAMP SSRI (262) 2159 | (SelectionSequenceId) 2160 | 2161 +--- Observation Point 1 (263) 2162 | (exporterIPv4Address) 2163 | (Interface Id) 2164 | 2165 +--- Observation Point 2 and 3 (264) 2166 | (exporterIPv4Address) 2167 | (line card) 2168 | 2169 +--- Observation Point 4 (265) 2170 | (exporterIPv4Address) 2171 | (line card) 2172 | (Interface Id) 2173 | 2174 | (selectorId 1) 2175 | (selectorId 2) 2177 Note that the example could further be improved with a basicList 2178 of selectorId if many Selector IDs have to be reported. 2180 Figure V: PSAMP SSRI to be encoded 2182 0 1 2 3 2183 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2184 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2185 | Set ID = 3 | Length = 26 | 2186 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2187 | Template ID = 262 | Field Count = 4 | 2188 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2189 | Scope Field Count = 1 |0| selectionSequenceId = 301 | 2190 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2191 | Scope 1 Length = 4 |0| subTemplateMultiList = ZZZ | 2192 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2193 | Field Length = 0xFFFF |0| selectorId = 302 | 2194 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2195 | Field Length = 4 |0| selectorId = 302 | 2196 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2197 | Field Length = 4 | 2198 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2200 Figure W: Options Template Record for PSAMP SSRI using 2201 subTemplateMultiList 2203 A subTemplateMultiList with semantic allOf is used to encode the 2204 list of Observation Points. 2206 0 1 2 3 2207 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2208 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2209 | Set ID = 2 | Length = 16 | 2210 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2211 | Template ID = 263 | Field Count = 2 | 2212 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2213 |0| exporterIPv4Address = 8 | Field Length = 4 | 2214 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2215 |0| ingressInterface = 10 | Field Length = 4 | 2216 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2218 Figure X: PSAMP SSRI, Template Record for interface 2220 0 1 2 3 2221 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2222 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2223 | Set ID = 2 | Length = 16 | 2224 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2225 | Template ID = 264 | Field Count = 2 | 2226 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2227 |0| exporterIPv4Address = 8 | Field Length = 4 | 2228 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2229 |0| lineCardId = 141 | Field Length = 4 | 2230 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2232 Figure Y: PSAMP SSRI, Template Record for linecard 2234 0 1 2 3 2235 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2236 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2237 | Set ID = 2 | Length = 20 | 2238 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2239 | Template ID = 265 | Field Count = 3 | 2240 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2241 |0| exporterIPv4Address = 8 | Field Length = 4 | 2242 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2243 |0| lineCardId = 141 | Field Length = 4 | 2244 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2245 |0| ingressInterface = 10 | Field Length = 4 | 2246 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2248 Figure Z: PSAMP SSRI, Template Record for linecard and interface 2250 The PSAMP SSRI Data Set is represented as follows: 2252 0 1 2 3 2253 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2254 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2255 | Set ID = 262 | Length = 68 | 2256 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2257 | selectionSequenceId = 7 | 2258 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2259 | 255 | Observation Point List Len=49 |semantic=allOf | 2260 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2261 | OP1 Template ID = 263 | OP1 Length = 12 | 2262 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2263 | Router 1 exporterIPv4Address = 192.0.2.11 | 2264 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2265 | OP1 ingressInterface = 1 | 2266 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2267 | OP2&OP3 Template ID = 264 | OP2 & OP3 Length = 20 | 2268 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2269 | Router 2 exporterIPv4Address = 192.0.2.12 | 2270 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2271 | OP2 lineCardId = A | 2272 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2273 | Router 3 exporterIPv4Address = 192.0.2.13 | 2274 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2275 | OP3 lineCardId = B | 2276 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2277 | OP4 Template ID = 265 | OP4 Length = 16 | 2278 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2279 | Router 4 exporterIPv4Address = 192.0.2.14 | 2280 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2281 | OP4 lineCardId = C | 2282 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2283 | OP4 ingressInterface = 2 | 2284 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2285 | selectorId = 5 | 2286 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2287 | selectorId = 10 | 2288 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2290 Figure ZA: Example of a PSAMP SSRI Data Record, Encoded using a 2291 subTemplateMultiList 2293 Note that the Data Record above contains multiple instances of 2294 Template 264 to represent Observation Point 2 (Router2, line card 2295 A) and Observation Point 3 (Router3, line card B). Instead, if a 2296 single Observation Point had both line card A and line card B, a 2297 basicList would be used to represent the list of line cards. 2299 10. Relationship with the Other IFPIX Documents 2301 10.1. Relationship with Reducing Redundancy 2303 "Reducing Redundancy in IP Flow Information Export (IPFIX) and 2304 Packet Sampling (PSAMP) Reports" [RFC5473] describes a bandwidth 2305 saving method for exporting Flow or packet information using the 2306 IP Flow Information eXport (IPFIX) protocol. 2308 It defines the commonPropertiesID Information Element for 2309 exporting Common Properties. 2311 10.1.1. Encoding Structured Data Element using Common Properties. 2313 When Structured Data Information Elements contain repeated 2314 elements, these elements may be replaced with a 2315 commonPropertiesID Information Element as specified in 2316 [RFC5473]. The replaced elements may include the basicList, 2317 subTemplateList and subTemplateMultiList Information Elements. 2319 This technique might help reducing the bandwidth requirements 2320 for the export. However, a detailed analysis of the gain has 2321 not been done; refer to Section 8.3 of [RFC5473] for further 2322 considerations. 2324 10.1.2. Encoding Common Properties elements With Structured Data 2325 Information Element. 2327 Structured Data Information Element MAY be used to define a list 2328 of commonPropertiesID, as a replacement for the specifications 2329 in [RFC5473]. 2331 Indeed, the example in figures 1 and 2 of [RFC5473] can be 2332 encoded with the specifications in this document. 2334 +----------------+-------------+---------------------------+ 2335 | sourceAddressA | sourcePortA | | 2336 +----------------+-------------+---------------------------+ 2337 | sourceAddressA | sourcePortA | | 2338 +----------------+-------------+---------------------------+ 2339 | sourceAddressA | sourcePortA | | 2340 +----------------+-------------+---------------------------+ 2341 | sourceAddressA | sourcePortA | | 2342 +----------------+-------------+---------------------------+ 2343 | ... | ... | ... | 2344 +----------------+-------------+---------------------------+ 2346 Figure ZB: Common and Specific Properties Exported Together 2347 [RFC5473] 2349 +------------------------+-----------------+-------------+ 2350 | index for properties A | sourceAddressA | sourcePortA | 2351 +------------------------+-----------------+-------------+ 2352 | ... | ... | ... | 2353 +------------------------+-----------------+-------------+ 2355 +------------------------+---------------------------+ 2356 | index for properties A | | 2357 +------------------------+---------------------------+ 2358 | index for properties A | | 2359 +------------------------+---------------------------+ 2360 | index for properties A | | 2361 +------------------------+---------------------------+ 2362 | index for properties A | | 2363 +------------------------+---------------------------+ 2365 Figure ZC: Common and Specific Properties Exported Separately 2366 according to [RFC5473] 2368 +----------------+-------------+---------------------------+ 2369 | sourceAddressA | sourcePortA | | 2370 +----------------+-------------+---------------------------+ 2371 | | 2372 +---------------------------+ 2373 | | 2374 +---------------------------+ 2375 | | 2376 +---------------------------+ 2377 | ... | 2378 +---------------------------+ 2380 Figure ZD: Common and Specific Properties Exported with 2381 Structured Data Information Element 2383 The example in figure ZD could be encoded with a basicList if 2384 the represents a single Information Element, 2385 with a subTemplateList if the represents a 2386 Template Record, or with a subTemplateMultiList if the is composed of different Template Records. 2389 Using Structured Data Information Elements as a replacement for 2390 the techniques specified in "Reducing Redundancy in IP Flow 2391 Information Export (IPFIX) and Packet Sampling (PSAMP) Reports" 2392 [RFC5473] offers the advantage that a single Template Record is 2393 defined. Hence the Collectors job is simplified in terms of 2394 Template management and combining Template/Options Template 2395 Records. 2397 However, it must be noted that using Structured Data Information 2398 Elements as a replacement for the techniques specified in 2399 "Reducing Redundancy in IP Flow Information Export (IPFIX) and 2400 Packet Sampling (PSAMP) Reports" only applies to simplified 2401 cases. For example, the "Multiple Data Reduction" (Section 7.1 2402 [RFC5473]) might be too complex to encode with Structured Data 2403 Information Elements. 2405 10.2. Relationship with Guidelines for IPFIX Testing 2407 [RFC5471] presents a list of tests for implementers of IP Flow 2408 Information eXport (IPFIX) compliant Exporting Processes and 2409 Collecting Processes. 2411 Although [RFC5471] doesn't define any structured data element 2412 specific tests, the Structured Data Information Elements can be 2413 used in many of the [RFC5471] tests. 2415 The [RFC5471] series of test could be useful because the 2416 document specifies that every Information Element type should be 2417 tested. However, not all cases from this document are tested in 2418 [RFC5471]. 2420 The following sections are especially noteworthy: 2422 . 3.2.1. Transmission of Template with fixed size 2423 Information Elements 2425 - each data type should be used in at least one test. 2426 The new data types specified in Section 4.1. should 2427 be included in this test. 2429 . 3.2.2. Transmission of Template with variable length 2430 Information Elements 2432 - this test should be expanded to include Data Records 2433 containing variable length basicList, 2434 subTemplateList, and subTemplateMultiList Information 2435 Elements. 2437 . 3.3.1. Enterprise-specific Information Elements 2439 - this test should include the export of basicList, 2440 subTemplateList, and subTemplateMultiList Information 2441 Elements containing Enterprise-specific Information 2442 Elements. e.g., see the example in figure B. 2444 . 3.3.3. Multiple instances of the same Information Element 2445 in one Template 2447 - this test should verify that multiple instances of the 2448 basicList, subTemplateList and subTemplateMultiList 2449 Information Elements are accepted. 2451 . 3.5 Stress/Load tests 2453 - since the structured data types defined here allow 2454 modeling of complex data structures, they may be 2455 useful for stress testing both Exporting Processes 2456 and Collecting Processes. 2458 10.3. Relationship with IPFIX Mediation Function 2460 The Structured Data Information Elements would be beneficial for 2461 the export of aggregated Data Records in mediation function, as 2462 was demonstrated with the example of the aggregated Observation 2463 Point in Section 5.3. 2465 11. IANA Considerations 2467 This document specifies several new IPFIX abstract data types, a 2468 new IPFIX Data Type Semantic, and several new Information 2469 Elements. 2471 These require the creation of two new IPFIX registries and 2472 updating the existing IPFIX Information Element registry as 2473 detailed below. 2475 11.1. New Abstract Data Types 2477 Section 4.1. of this document specifies several new IPFIX abstract 2478 data types. Per Section 6 of the IPFIX information model 2479 [RFC5102], new abstract data types can be added to the IPFIX 2480 information model, in the IPFIX Information Element Data Types 2481 registry. 2483 Abstract data types to be added to the IPFIX "Information Element 2484 Data Types" registry are listed below. 2486 EDITOR'S NOTE: IANA, please pick the number three values in the 2487 http://www.iana.org/assignments/ipfix/ipfix.xml#informationElement 2488 DataTypes for the basicList, subTemplateList, and 2489 subTemplateMultiList. 2491 11.1.1. basicList 2493 The type "basicList" represents a list of any Information Element 2494 used for single-valued data types. 2496 11.1.2. subTemplateList 2498 The type "subTemplateList" represents a list of a structured data 2499 type, where the data type of each list element is the same and 2500 corresponds with a single Template Record. 2502 11.1.3. subTemplateMultiList 2504 The type "subTemplateMultiList" represents a list of structured 2505 data types, where the data types of the list elements can be 2506 different and correspond with different template definitions. 2508 11.2. New Data Type Semantics 2510 Section 4.2. of this document specifies a new IPFIX Data Type 2511 Semantic. Per Section 3.2 of the IPFIX information model 2512 [RFC5102], new data type semantics can be added to the IPFIX 2513 information model. Therefore, the IANA IPFIX 2514 informationElementSemantics registry [IANA-IPFIX], which contains 2515 all the data type semantics from Section 3.2 of [RFC5102], must be 2516 augmented with the "list" value below. 2518 11.2.1. list 2520 A list is a structured data type, being composed of a sequence of 2521 elements e.g. Information Element, Template Record, etc. 2523 11.3. New Information Elements 2525 Section 4.3. of this document specifies several new Information 2526 Elements which are to be created in the IPFIX Information Element 2527 registry [IANA-IPFIX]. 2529 New Information Elements to be added to the IPFIX Information 2530 Element registry are listed below. 2532 EDITOR'S NOTE: the XML specification in Appendix A must be updated 2533 with the elementID values allocated below. 2535 11.3.1. basicList 2537 Name: basicList 2538 Description: 2539 Specifies a generic Information Element with a basicList abstract 2540 data type. For example, a list of port numbers, a list of 2541 interface indexes, etc. 2542 Abstract Data Type: basicList 2543 Data Type Semantics: list 2544 ElementId: XXX (to be specified by IANA) 2545 Status: current 2547 11.3.2. subTemplateList 2549 Name: subTemplateList 2550 Description: 2551 Specifies a generic Information Element with a subTemplateList 2552 abstract data type. 2553 Abstract Data Type: subTemplateList 2554 Data Type Semantics: list 2555 ElementId: YYY (to be specified by IANA) 2556 Status: current 2558 11.3.3. subTemplateMultiList 2560 Name: subTemplateMultiList 2561 Description: 2562 Specifies a generic Information Element with a 2563 subTemplateMultiList abstract data type. 2564 Abstract Data Type: subTemplateMultiList 2565 Data Type Semantics: list 2566 ElementId: ZZZ (to be specified by IANA) 2567 Status: current 2569 11.4. New Structured Data Semantics 2571 Section 4.4. of this document specifies a series of new IPFIX 2572 structured data type semantics, which is expressed as an 8-bit 2573 value. This requires the creation of a new IPFIX "structured data 2574 types semantics" IPFIX subregistry [IANA-IPFIX]. 2576 Entries may be added to this subregistry subject to a Standards 2577 Action [RFC5226]. Initially, this registry should include all the 2578 structured data type semantics listed below. 2580 11.4.1. undefined 2582 Name: undefined 2584 Description: The "undefined" structured data type semantic 2585 specifies that the semantic of list elements is not specified, and 2586 that, if a semantic exists, then it is up to the Collecting 2587 Process to draw its own conclusions. The "undefined" structured 2588 data type semantic is the default structured data type semantic. 2590 Value: 0xFF 2592 Reference: 2594 11.4.2. noneOf 2596 Name: noneOf 2598 Description: The "noneOf" structured data type semantic specifies 2599 that none of the elements are actual properties of the Data 2600 Record. 2602 Value: 0x00 2603 Reference: 2605 11.4.3. exactlyOneOf 2607 Name: exactlyOneOf 2609 Description: The "exactlyOneOf" structured data type semantic 2610 specifies that only a single element from the structured data is 2611 an actual property of the Data Record. This is equivalent to a 2612 logical XOR operation. 2614 Value: 0x01 2616 Reference: 2618 11.4.4. oneOrMoreOf 2620 Name: oneOrMoreOf 2622 Description: The "oneOrMoreOf" structured data type semantic 2623 specifies that one or more elements from the list in the 2624 structured data are actual properties of the Data Record. This is 2625 equivalent to a logical OR operation. 2627 Value: 0x02 2629 Reference: 2631 11.4.5. allOf 2633 Name: allOf 2635 Description: The "allOf" structured data type semantic specifies 2636 that all of the list elements from the structured data are actual 2637 properties of the Data Record. 2639 Value: 0x03 2641 Reference: 2643 11.4.6. ordered 2645 Name: ordered 2646 Description: The "ordered" structured data type semantic specifies 2647 that elements from the list in the structured data are ordered. 2649 Value: 0x04 2651 Reference: 2653 12. Security Considerations 2655 The addition of complex data types necessarily complicates the 2656 implementation of the Collector. This could easily result in new 2657 security vulnerabilities (e.g., buffer overflows); this creates 2658 additional risk in cases where either DTLS is not used, or if the 2659 Observation Point and Collector belong to different trust domains. 2660 Otherwise, the same security considerations as for the IPFIX 2661 Protocol [RFC5101] and the IPFIX information model [RFC5102] 2662 apply. 2664 13. References 2666 13.1. Normative References 2668 [RFC2119] S. Bradner, Key words for use in RFCs to Indicate 2669 Requirement Levels, BCP 14, RFC 2119, March 1997. 2671 [RFC5101] Claise, B., Ed., "Specification of the IP Flow 2672 Information Export (IPFIX) Protocol for the Exchange of 2673 IP Traffic Flow Information", RFC 5101, January 2008. 2675 [RFC5102] Quittek, J., Bryant, S., Claise, B., Aitken, P., and 2676 J. Meyer, "Information Model for IP Flow Information 2677 Export", RFC 5102, January 2008. 2679 [RFC5226] T. Narten, T., Alverstrand, H. , "Guidelines for 2680 Writing an IANA Considerations Section in RFCs", 2681 RFC5226, May 2008. 2683 13.2. Informative References 2685 [RFC3917] Quittek, J., Zseby, T., Claise, B., and S. Zander, 2686 Requirements for IP Flow Information Export, RFC 3917, 2687 October 2004. 2689 [RFC5103] Trammell, B., and E. Boschi, "Bidirectional Flow 2690 Export Using IP Flow Information Export (IPFIX)", RFC 2691 5103, January 2008. 2693 [RFC5470] Sadasivan, G., Brownlee, N., Claise, B., and J. 2694 Quittek, "Architecture for IP Flow Information Export", 2695 RFC 5470, March 2009. 2697 [RFC5471] Schmoll, C., Aitken, P., and B. Claise, "Guidelines 2698 for IP Flow Information Export (IPFIX) Testing", RFC 2699 5471, March 2009. 2701 [RFC5472] Zseby, T., Boschi, E., Brownlee, N., and B. Claise, 2702 "IP Flow Information Export (IPFIX) Applicability", RFC 2703 5472, March 2009. 2705 [RFC5473] Boschi, E., Mark, L., and B. Claise, "Reducing 2706 Redundancy in IP Flow Information Export (IPFIX) and 2707 Packet Sampling (PSAMP) Reports", RFC 5473, March 2009. 2709 [RFC5475] Zseby, T., Molina, M., Duffield, N., Niccolini, S., 2710 and F. Raspall, "Sampling and Filtering Techniques for 2711 IP Packet Selection", RFC 5475, March 2009. 2713 [RFC5476] Claise, B., Ed., "Packet Sampling (PSAMP) Protocol 2714 Specifications", RFC 5476, March 2009. 2716 [RFC5477] Dietz, T., Claise, B., Aitken, P., Dressler, F., and 2717 G. Carle, "Information Model for Packet Sampling 2718 Exports", RFC 5477, March 2009. 2720 [IANA-IPFIX] http://www.iana.org/assignments/ipfix/ipfix.xhtml 2722 14. Acknowledgement 2724 The authors would like to thank Zhipu Jin, Nagaraj Varadharajan, 2725 Brian Trammel, Atsushi Kobayashi, Rahul Patel for their feedback, 2726 and Gerhard Muenz, for proof reading the document. 2728 15. Authors' Addresses 2730 Benoit Claise 2731 Cisco Systems, Inc. 2732 De Kleetlaan 6a b1 2733 Diegem 1813 2734 Belgium 2736 Phone: +32 2 704 5622 2737 EMail: bclaise@cisco.com 2739 Gowri Dhandapani 2740 Cisco Systems, Inc. 2741 13615 Dulles Technology Drive 2742 Herndon, Virigina 20171 2743 United States 2745 Phone: +1 408 853 0480 2746 EMail: gowri@cisco.com 2748 Stan Yates 2749 Cisco Systems, Inc. 2750 7100-8 Kit Creek Road 2751 PO Box 14987 2752 Research Triangle Park 2753 North Carolina, 27709-4987 2754 United States 2756 Phone: +1 919 392 8044 2757 EMail: syates@cisco.com 2759 Paul Aitken 2760 Cisco Systems, Inc. 2761 96 Commercial Quay 2762 Commercial Street 2763 Edinburgh, EH6 6LX, United Kingdom 2765 Phone: +44 131 561 3616 2766 EMail: paitken@cisco.com 2767 Appendix A. Additions to XML Specification of IPFIX Information 2768 Elements and Abstract Data Types 2770 This appendix contains additions to the machine-readable 2771 description of the IPFIX information model coded in XML in 2772 Appendix A and Appendix B in [RFC5102]. Note that this appendix 2773 is of informational nature, while the text in section 4. 2774 (generated from this appendix) is normative. 2776 The following field definitions are appended to the IPFIX 2777 information model in Appendix A of [RFC5102]. 2779 2784 2785 2786 Represents a list of zero or more instances of 2787 any Information Element, primarily used for 2788 single-valued data types. For example, a list of port 2789 numbers, list of interface indexes, list of AS in a 2790 BGP AS-PATH, etc. 2791 2792 2793 2795 2800 2801 2802 Represents a list of zero or more instances of a 2803 structured data type, where the data type of each list 2804 element is the same and corresponds with a single 2805 Template Record. For example, a structured data type 2806 composed of multiple pairs of ("MPLS label stack entry 2807 position", "MPLS label stack value"), a structured data 2808 type composed of performance metrics, a structured data 2809 type composed of multiple pairs of IP address, etc. 2810 2811 2813 2815 2820 2821 2822 Represents a list of zero or more instances of 2823 structured data types, where the data type of each list 2824 element can be different and corresponds with 2825 different template definitions. For example, a 2826 structured data type composed of multiple access-list 2827 entries, where entries can be composed of different 2828 criteria types. 2829 2830 2831 2833 The following structured data type semantic definitions are 2834 appended to the the IPFIX information model in Appendix A of 2835 [RFC5102]. 2837 2838 2839 2840 2841 The "undefined" structured data type semantic specifies 2842 that the semantic of list elements is not specified, and 2843 that, if a semantic exists, then it is up to the 2844 Collecting Process to draw its own conclusions. The 2845 "undefined" structured data type semantic is the default 2846 structured data type semantic. 2847 2848 2849 2851 2852 2853 2854 The "noneOf" structured data type semantic specifies 2855 that none of the elements are actual properties of the 2856 Data Record. 2857 2859 2860 2862 2863 2864 2865 The "exactlyOneOf" structured data type semantic 2866 specifies that only a single element from the structured 2867 data is an actual property of the Data Record. This is 2868 equivalent to a logical XOR operation. 2869 2870 2871 2873 2874 2875 2876 The "oneOrMoreOf" structured data type semantic 2877 specifies that one or more elements from the list in the 2878 structured data are actual properties of the Data 2879 Record. This is equivalent to a logical OR operation. 2880 2881 2882 2884 2885 2886 2887 The "allOf" structured data type semantic specifies that 2888 all of the list elements from the structured data are 2889 actual properties of the Data Record. 2890 2891 2892 2894 2895 2896 2897 The "ordered" structured data type semantic specifies 2898 that elements from the list in the structured data are 2899 ordered. 2900 2901 2902 2903 2905 The following schema definitions are appended to the abstract data 2906 types defined in Appendix B of [RFC5102]. This schema and its 2907 namespace are registered by IANA at 2908 http://www.iana.org/assignments/xml-registry/schema/ipfix.xsd 2910 2911 2912 2913 2914 2915 Represents a list of zero or more instances of 2916 any Information Element, primarily used for 2917 single-valued data types. For example, a list of port 2918 numbers, list of interface indexes, list of AS in a 2919 BGP AS-PATH, etc. 2920 2921 2922 2923 2924 2925 2926 Represents a list of zero or more instances of a 2927 structured data type, where the data type of each list 2928 element is the same and corresponds with a single 2929 Template Record. For example, a structured data type 2930 composed of multiple pairs of ("MPLS label stack entry 2931 position", "MPLS label stack value"), a structured 2932 data type composed of performance metrics, a 2933 structured data type composed of multiple pairs of IP 2934 address, etc. 2935 2936 2937 2938 2939 2940 2941 Represents a list of zero or more instances of 2942 structured data types, where the data type of each 2943 list element can be different and corresponds with 2944 different template definitions. For example, a 2945 structured data type composed of multiple 2946 access-list entries, where entries can be 2947 composed of different criteria types. 2948 2949 2951 2952 2953 2955 2956 2957 2958 2959 2960 Represents an arbitrary-length sequence of structured 2961 data elements, either composed of regular Information 2962 Elements or composed of data conforming to a Template 2963 Record. 2964 2965 2966 2967 2968 2970 2971 2972 2974 2975 2976 2977 2978 2979 2981 2982 2983 2984 2986 2988 2989 2990 structured data type semantics express the relationship 2991 among multiple list elements in a structured data 2992 Information Element. 2993 2994 2995 2997 Appendix B. Encoding IPS Alert using Structured Data Information 2998 Elements 3000 In this section, an IPS alert example is used to demonstrate how 3001 complex data and multiple levels of hierarchy can be encoded using 3002 Structured Data Information Elements. Also, this example 3003 demonstrates how a basicList of subTemplateLists can be used to 3004 represent semantics at multiple levels in the hierarchy. 3006 An IPS alert consists of the following mandatory attributes: 3007 signatureId, protocolIdentifier and riskRating. It can also 3008 contain zero or more participants, each participant can contain 3009 zero or more attackers and zero or more targets. An attacker 3010 contains the attributes sourceIPv4Address and applicationId, and 3011 a target contains the attributes destinationIPv4Address and 3012 applicationId. 3014 Note that the signatureId and riskRating Information Element 3015 fields are created for these examples only; the Field IDs are 3016 shown as N/A. The signatureId helps to uniquely identify the IPS 3017 signature that triggered the alert. The riskRating identifies the 3018 potential risk, on a scale of 0-100 (100 being most serious), of 3019 the traffic that triggered the alert. 3021 Consider the example described in case study 2 of Section 5.6. The 3022 IPS alert contains participants encoded as a subTemplateList with 3023 semantic allOf. Each participant uses a basicList of 3024 subTemplateLists to represent attackers and targets. For the sake 3025 of simplicity, the alert has two participants P1 and P2. In 3026 participant P1, attacker A1 or A2 attack target T1. In 3027 participant P2, attacker A3 attacks targets T2 and T3. 3029 Participant P1: 3031 (basicList, allof, 3033 (subTemplateList, exactlyOneOf, attacker A1, A2) 3035 (subTemplateList, undefined, target T1) 3037 ) 3039 Participant P2: 3041 (basicList, allOf, 3043 (subTemplateList, undefined, attacker A3, 3044 (subTemplateList, allOf, targets T2, T3) 3046 ) 3048 Alert : 3050 (subTemplateList, allOf, Participant P1, Participant P2) 3052 ------------------------------------------------------------------ 3053 | | | participant 3054 sigId |protocol| risk | attacker | target 3055 | Id | Rating | IP | appId | IP | appId 3056 ------------------------------------------------------------------ 3057 1003 17 10 192.0.2.3 103 192.0.2.103 3001 3058 192.0.2.4 104 3060 192.0.2.5 105 192.0.2.104 4001 3061 192.0.2.105 5001 3062 ------------------------------------------------------------------ 3064 Participant P1 contains: 3065 Attacker A1: (IP, appID)=(192.0.2.3, 103) 3066 Attacker A2: (IP, appID)=(192.0.2.4, 104) 3067 Target T1: (IP, appID)= (192.0.2.103, 3001) 3069 Participant P2 contains: 3070 Attacker A3: (IP, appID) = (192.0.2.5, 105) 3071 Target T2: (IP, appID)= (192.0.2.104, 4001) 3072 Target T3: (IP, appID)= (192.0.2.105, 5001) 3074 To represent an alert, the following Templates are defined: 3075 Template for target (268) 3076 Template for attacker (269) 3077 Template for participant (270) 3078 Template for alert (271) 3080 alert (271) 3081 | (signatureId) 3082 | (protocolIdentifier) 3083 | (riskRating) 3084 | 3085 +------- participant (270) 3086 | 3087 +------- attacker (269) 3088 | (sourceIPv4Address) 3089 | (applicationId) 3090 | 3091 +------- target (268) 3092 | (destinationIPv4Address) 3093 | (applicationId) 3095 Note that the attackers are always composed of a single 3096 applicationId, while the targets typically have multiple 3097 applicationId, for the sake of simplicity this example shows only 3098 one applicationId in the target. 3100 Template Record for target, with the Template ID 268: 3102 0 1 2 3 3103 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 3104 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3105 | Set ID = 2 | Length = 16 octets | 3106 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3107 | Template ID = 268 | Field Count = 2 | 3108 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3109 |0| destinationIPv4Address = 12 | Field Length = 4 | 3110 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3111 |0| applicationId = 95 | Field Length = 4 | 3112 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3114 Figure B0: Encoding IPS Alert, Template for Target 3116 Template Record for attacker, with the Template ID 269: 3118 0 1 2 3 3119 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 3120 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3121 | Set ID = 2 | Length = 16 octets | 3122 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3123 | Template ID = 269 | Field Count = 2 | 3124 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3125 |0| sourceIPv4Address = 8 | Field Length = 4 | 3126 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3127 |0| applicationId = 95 | Field Length = 4 | 3128 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3130 Figure B1: Encoding IPS Alert, Template for Attacker 3132 Template Record for participant, with the Template ID 270: 3134 0 1 2 3 3135 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 3136 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3137 | Set ID = 2 | Length = 12 octets | 3138 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3139 | Template ID = 270 | Field Count = 1 | 3140 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3141 |0| basicList = XXX | Field Length = 0xFFFF | 3142 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3144 Figure B2: Encoding IPS Alert, Template for Participant 3146 The Template Record for the participant has one basicList 3147 Information Element, which is a list of subTemplateLists of 3148 attackers and targets. 3150 Template Record for IPS alert, with the Template ID 271: 3152 0 1 2 3 3153 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 3154 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3155 | Set ID = 2 | Length = 24 octets | 3156 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3157 | Template ID = 271 | Field Count = 4 | 3158 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3159 |0| signatureId = N/A | Field Length = 2 | 3160 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3161 |0| protocolIdentifier = 4 | Field Length = 1 | 3162 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3163 |0| riskRating = N/A | Field Length = 1 | 3164 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3165 |0| subTemplateList = YYY | Field Length = 0xFFFF | 3166 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3168 Figure B3: Encoding IPS Alert, Template for IPS Alert 3170 The subTemplateList in the alert Template Record contains a list 3171 of participants. 3173 The Length of basicList and subTemplateList are encoded in three 3174 bytes even though they may be less than 255 octets. 3176 The Data Set is represented as follows: 3178 0 1 2 3 3179 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 3180 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3181 | Set ID = 271 | Length = 102 | 3182 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3183 | signatureId = 1003 | protocolId=17 | riskRating=10 | 3184 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3185 | 255 |participant List Length = 91 |semantic=allOf | 3186 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3187 | participant Template ID = 270 | 255 | P1 List Len = | 3188 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3189 | 41 | semantic=allOf| P1 List Field ID = YYY | 3190 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3191 | P1 List Field ID Len = 0xFFFF | 255 |P1 attacker ...| 3192 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3193 | List Len = 19 |sem=exactlyOne | P1 attacker Template ID = 269 | 3194 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3195 | P1 attacker A1 sourceIPv4Address = 192.0.2.3 | 3196 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3197 | P1 attacker A1 applicationId = 103 | 3198 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3199 | P1 attacker A2 sourceIPv4Address = 192.0.2.4 | 3200 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3201 | P1 attacker A2 applicationId = 104 | 3202 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3203 | 255 | P1 target List Len = 11 | sem=undefined | 3204 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3205 | P1 target Template ID = 268 | P1 target T1 destinationIPv4 | 3206 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3207 | ... Address = 192.0.2.103 |P1 target T1 applicationId =...| 3208 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3209 | ... 3001 | 255 | P2 List Len = | 3210 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3211 | ... 41 | semantic=allOf| P2 List Field ID = YYY | 3212 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3213 | P2 List Field ID Len = 0xFFFF | 255 |P2 attacker ...| 3214 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3215 | List Len = 11 | sem=undefined | P2 attacker Template ID = 269 | 3216 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3217 | P2 attacker A3 sourceIPv4Address = 192.0.2.5 | 3218 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3219 | P2 attacker A3 applicationId = 105 | 3220 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3221 | 255 | P2 target List Len = 19 |semantic=allOf | 3222 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3223 | P2 target Template ID = 268 | P2 target T2 destinationIPv4 | 3224 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3225 | ... Address = 192.0.2.104 |P2 target T2 applicationId =...| 3226 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3227 | ... 4001 | P2 target T3 destinationIPv4 | 3228 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3229 | ... Address = 192.0.2.105 |P2 target T3 applicationId =...| 3230 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3231 | ... 5001 | 3232 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3234 Figure B4: Encoding IPS Alert, Data Set