idnits 2.17.1 draft-ietf-ipfix-export-per-sctp-stream-08.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (May 31, 2010) is 5072 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Obsolete normative reference: RFC 4960 (Obsoleted by RFC 9260) ** Obsolete normative reference: RFC 5101 (Obsoleted by RFC 7011) ** Obsolete normative reference: RFC 5102 (Obsoleted by RFC 7012) == Outdated reference: A later version (-13) exists of draft-ietf-tsvwg-sctp-strrst-04 Summary: 3 errors (**), 0 flaws (~~), 2 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 IPFIX Working Group B. Claise 3 Internet-Draft P. Aitken 4 Intended Status: Standards Track A. Johnson 5 Expires: November 31, 2010 Cisco Systems, Inc. 6 G. Muenz 7 TU Muenchen 8 May 31, 2010 10 IPFIX Export per SCTP Stream 11 draft-ietf-ipfix-export-per-sctp-stream-08 13 Abstract 15 This document specifies an extension to the specifications 16 in RFC5101, IP Flow Information Export (IPFIX), when using 17 the Partial Reliability extension of SCTP (PR-SCTP, Partial 18 Reliability Stream Control Transmission Protocol). 19 When implemented at both the Exporting and Collecting Processes, 20 this method offers several advantages such as the ability to 21 calculate Data Record losses for PR-SCTP, immediate export of 22 Template Withdrawal Messages, immediate reuse of Template IDs 23 within an SCTP stream, reduced likelihood of Data Record loss, 24 and reduced demands on the Collecting Process. When implemented 25 in only the Collecting or Exporting Process then normal IPFIX 26 behavior will be seen without these additional benefits. 28 Status of this Memo 30 This Internet-Draft is submitted to IETF in full conformance 31 with the provisions of BCP 78 and BCP 79. 33 Internet-Drafts are working documents of the Internet 34 Engineering Task Force (IETF), its areas, and its working 35 groups. Note that other groups may also distribute working 36 documents as Internet-Drafts. 38 Internet-Drafts are draft documents valid for a maximum of six 39 months and may be updated, replaced, or obsoleted by other 40 documents at any time. It is inappropriate to use Internet- 41 Drafts as reference material or to cite them other than as "work 42 in progress." 44 The list of current Internet-Drafts can be accessed at 45 http://www.ietf.org/ietf/1id-abstracts.txt 47 The list of Internet-Draft Shadow Directories can be accessed at 48 http://www.ietf.org/shadow.html 50 This Internet-Draft will expire on May, 2010. 52 Copyright Notice 53 Copyright (c) 2010 IETF Trust and the persons identified as the 54 document authors. All rights reserved. 56 This document is subject to BCP 78 and the IETF Trust's Legal 57 Provisions Relating to IETF Documents 58 (http://trustee.ietf.org/license-info) in effect on the date of 59 publication of this document. Please review these documents 60 carefully, as they describe your rights and restrictions with 61 respect to this document. Code Components extracted from this 62 document must include Simplified BSD License text as described 63 in Section 4.e of the Trust Legal Provisions and are provided 64 without warranty as described in the Simplified BSD License. 66 Conventions used in this document 68 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", 69 "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", 70 and "OPTIONAL" in this document are to be interpreted as 71 described in RFC 2119 [RFC2119]. 73 Table of Contents 75 1. Introduction............................................... 3 76 1.1. Relationship with IPFIX and PSAMP..................... 4 77 1.2. Applicability......................................... 4 78 1.3. Limitations........................................... 5 79 2. Terminology................................................ 5 80 2.1. IPFIX Documents Overview.............................. 6 81 2.2. PSAMP Documents Overview.............................. 6 82 3. IPFIX Protocol Specifications Limitations and 83 Improvements............................................... 7 84 3.1. Data Record Loss per Template......................... 7 85 3.1.1. IPFIX Protocol Specifications Limitation......... 7 86 3.1.2. IPFIX Export per SCTP Stream Advantage........... 8 87 3.2. Transmission Order within an SCTP stream.............. 8 88 3.2.1. IPFIX Protocol Specifications Limitation......... 8 89 3.2.2. IPFIX Export per SCTP Stream Advantages.......... 9 90 3.3. No Transmission Order across SCTP Streams............. 9 91 3.3.1. IPFIX Protocol Specifications Limitation......... 9 92 3.3.2. IPFIX Export per SCTP Stream Advantages......... 10 93 4. Specifications............................................ 10 94 4.1. New Information Element.............................. 10 95 4.2. Template Management.................................. 11 96 4.3. SCTP................................................. 13 97 4.4. Template Withdrawal Message.......................... 13 98 4.5. The Collecting Process's Side........................ 14 99 5. Performance Impact........................................ 16 100 6. Guidelines for IPFIX per-SCTP-stream Extension Testing.... 16 101 7. Examples.................................................. 17 102 8. IANA Considerations....................................... 21 103 9. Security Considerations................................... 21 104 10. References............................................... 21 105 10.1. Normative References................................ 21 106 10.2. Informative References.............................. 22 107 11. Acknowledgements......................................... 22 108 12. Author's Addresses....................................... 23 110 1. Introduction 112 The IPFIX protocol [RFC5101] has the goal of exporting IP 113 Flow information. This protocol is designed to export 114 information about IP traffic Flows and related measurement 115 data, where a Flow is defined by a set of key attributes 116 (e.g., source and destination IP address, source and 117 destination port, etc.). However, thanks to its Template 118 mechanism, the IPFIX protocol can export any type of 119 information, as long as the relevant Information Element is 120 specified in the IPFIX Information Model [RFC5102], 121 registered with IANA, or specified as an enterprise-specific 122 Information Element. 124 The IPFIX protocol [RFC5101] specifies that IP traffic 125 measurements for Flows are exported using a TLV (type, 126 length, value) format. The information is exported using a 127 Template Record which is sent once to export the {type, 128 length} pairs that define the data format for the Information 129 Elements in a Flow. The Data Records specify values for each 130 Flow. 132 The IPFIX protocol [RFC5101] is flexible: it foresees the usage 133 of multiple SCTP streams per association; it allows the 134 transmission of Data Sets, Template Sets, and/or Options 135 Template Sets on any SCTP stream; it offers full and partially 136 reliable export of Data Sets; it proposes ordered or out-of- 137 order delivery of Data Sets. However, due to bandwidth 138 restrictions and packet losses in the network as well as 139 resource constraints on the Exporter and Collector (e.g., 140 limited buffer sizes), it is not always possible to export all 141 Data Sets in a reliable way. 143 This document specifies a method for exporting a Template Record 144 and its associated Data Sets in a single SCTP stream, limiting 145 each Template ID to a single SCTP stream if possible, and 146 imposing in-order transmission. 148 This method offers several advantages over IPFIX export as 149 specified in [RFC5101] such as the ability to calculate Data 150 Record losses for PR-SCTP, immediate export of Template 151 Withdrawal Messages, immediate reuse of Template IDs within an 152 SCTP stream, reduced likelihood of Data Record loss, and reduced 153 demands on the Collecting Process. 155 1.1. Relationship with IPFIX and PSAMP 157 The specification in this document applies to the IPFIX 158 protocol specifications [RFC5101]. However, it only applies 159 to the SCTP transport protocol [RFC4960] option of the IPFIX 160 protocol specifications, specifically in the case of the 161 partial reliability extension [RFC3758]. All specifications 162 from [RFC5101] apply unless specified otherwise in this 163 document. 165 As the Packet Sampling (PSAMP) protocol specifications 166 [RFC5476] are based on the IPFIX protocol specifications, the 167 specifications in this document are also valid for the PSAMP 168 protocol. 170 1.2. Applicability 172 The specifications contained in this document are applicable to 173 cases where application requirements include knowing how many 174 data records of a certain type (i.e., from a certain Template) 175 were lost. A typical example is a router exporting billing 176 records. Furthermore, they apply in cases where the Exporter 177 can not afford to export all the Flow Records reliably, due to 178 the limited resources to buffer the huge amount of Flow Records. 179 Such situations may occur if Data Sets are generated at a higher 180 rate at the Exporter than can be transferred to the Collector 181 because of bandwidth limitations in the network or slow 182 reception at the Collector. 184 To be more precise, the specification applicability is the case 185 where multiple Templates are simultaneously active within a 186 single SCTP Transport Session and the calculation of the Data 187 Record loss for a particular Template is required. Indeed, with 188 the current IPFIX specifications [RFC5101], if an IPFIX Message 189 is lost (UDP or SCTP partially reliable), it is not possible to 190 determine to which Template the lost Data Records belong to. 192 Exporting Processes following this specification will 193 interoperate with existing Collecting Processes that comply with 194 [RFC5101]; no changes are required at the Collecting Process to 195 receive data from an Exporting Process compliant with this 196 method. However, Collecting Processes may implement additional 197 support for per-stream export specified in this document in 198 order to realize all the benefits of the approach specified 199 herein. 201 1.3. Limitations 203 When multiple Templates are required, this method requires 204 multiple SCTP streams in the association between the Exporting 205 and Collecting Process, ideally one per Template. To properly 206 handle the transmission of additional Templates during the 207 Transport Session, additional SCTP streams are sometimes 208 required. These SCTP streams can only be added within the 209 existing SCTP association if the specifications in [SCTP-RESET] 210 are supported. 212 2. Terminology 214 IPFIX-specific terminology used in this document is defined 215 in section 2 of [RFC5101]. As in [RFC5101], these IPFIX- 216 specific terms have the first letter of a word capitalized 217 when used in this document. 219 Note that, in this document, "(Options) Template" is used to 220 refer to Templates and Options Templates. Unless otherwise 221 specified, "Template" alone refers to Templates exclusive of 222 Options Templates. 224 Template Reuse Delay 226 The time the Exporting Process needs to wait after sending 227 the last Data Set described by a given Template before 228 sending a Template Withdrawal Message for the Template. 229 [RFC5101] specifies a default value of 5 seconds. 231 2.1. IPFIX Documents Overview 233 The IPFIX Protocol [RFC5101] provides network administrators 234 with access to IP Flow information. 236 The architecture for the export of measured IP Flow 237 information out of an IPFIX Exporting Process to a Collecting 238 Process is defined in the IPFIX Architecture [RFC5470], per 239 the requirements defined in [RFC3917]. 241 The IPFIX Architecture [RFC5470] specifies how IPFIX Data 242 Records and Templates are carried via a congestion-aware 243 transport protocol from IPFIX Exporting Processes to IPFIX 244 Collecting Processes. 246 IPFIX has a formal description of IPFIX Information Elements, 247 their names, types and additional semantic information, as 248 specified in the IPFIX Information Model [RFC5102]. 250 Finally the IPFIX Applicability Statement [RFC5472] describes 251 what type of applications can use the IPFIX protocol and how 252 they can use the information provided. It furthermore shows 253 how the IPFIX framework relates to other architectures and 254 frameworks. 256 2.2. PSAMP Documents Overview 258 The document "A Framework for Packet Selection and Reporting" 259 [RFC5474], describes the Packet Sampling (PSAMP) framework 260 for network elements to select subsets of packets by 261 statistical and other methods, and to export a stream of 262 reports on the selected packets to a collector. 264 The set of packet selection techniques (sampling, filtering, 265 and hashing) supported by PSAMP are described in "Sampling 266 and Filtering Techniques for IP Packet Selection" [RFC5475]. 268 The PSAMP protocol [RFC5476] specifies the export of packet 269 information from a PSAMP Exporting Process to a PSAMP 270 Collecting Process. Like IPFIX, PSAMP has a formal 271 description of its Information Elements, their names, types 272 and additional semantic information. The PSAMP information 273 model is defined in [RFC5477]. 275 Finally [PSAMP-MIB] describes the PSAMP Management 276 Information Base. 278 3. IPFIX Protocol Specifications Limitations and Improvements 280 For three specific topics ("Data Record Loss per Template", 281 "Transmission Order within an SCTP stream", "No Transmission 282 Order across SCTP Streams"), this section explains the IPFIX 283 protocol specifications limitations on the one hand, and the 284 advantages of the method specified in this document on the other 285 hand. 287 3.1. Data Record Loss per Template 289 3.1.1. IPFIX Protocol Specifications Limitation 291 Section 6.3.2 of the "Requirements for IP Flow Information 292 Export" [RFC3917] discusses the data transfer reliability 293 issues: "Loss of flow records during the data transfer from 294 the Exporting Process to the Collecting Process must be 295 indicated at the Collecting Process." 297 However, in some cases, it may be important to know how many 298 Data Records of a certain type were lost (e.g., in the case 299 of billing), and IPFIX does not conventionally provide this 300 information. 302 A Collector can detect out-of-sequence, dropped, or duplicate 303 IPFIX Messages by tracking the Sequence Number [RFC5101]. 304 Note that the Sequence Number field in the IPFIX Message 305 header increases with the number of IPFIX Data Records within 306 the SCTP stream, so loss will be detected per stream. 308 The IPFIX protocol specification [RFC5101] specifies that Data 309 Records defined by any Template may be sent on any SCTP stream. 310 As such, if there is more than one Template defined within the 311 whole SCTP association, then there is no way of knowing which 312 Template any lost Data Record is associated with. This is true, 313 no matter what convention the Exporting Process uses to send 314 Data Records on different SCTP streams, as the protocol makes no 315 guarantees. 317 Note that a workaround allowed by the IPFIX specifications 318 [RFC5101] is to use only one Template Record per SCTP Transport 319 Session, at the cost of multiplying the number of SCTP Transport 320 Sessions when multiple Template Records are required. 322 3.1.2. IPFIX Export per SCTP Stream Advantage 324 Using the specification in this document, it is guaranteed that 325 any lost Data Records will be associated only with the Templates 326 that are defined on that SCTP stream. By defining only one 327 Template per SCTP stream, it is ensured that any loss is 328 associated with that single Template. So, by exporting each 329 Template and the corresponding Data Records within a different 330 SCTP stream, the loss pertaining to each specific Template can 331 be deduced from the Sequence Number field in the IPFIX Message 332 headers. 334 3.2. Transmission Order within an SCTP stream 336 3.2.1. IPFIX Protocol Specifications Limitation 338 A Collecting Process must have received the Template Record 339 associated with the Data Records to be able to decode the 340 information in the Data Records. [RFC5101] specifies: 342 "The Exporting Process SHOULD transmit the Template Set 343 and Options Template Set in advance of any Data Sets that 344 use that (Options) Template ID, to help ensure that the 345 Collector has the Template Record before receiving the 346 first Data Record." 348 The fact that the Collecting Process cannot decode the Data 349 Records without the corresponding Template Record may result in 350 Data Records being discarded by the Collector, as specified in 351 [RFC5101]: 353 "The Collecting Process normally receives Template Records 354 from the Exporting Process before receiving Data Records. 355 The Data Records are then decoded and stored by the 356 Collector. If the Template Records have not been received 357 at the time Data Records are received, the Collecting 358 Process MAY store the Data Records for a short period of 359 time and decode them after the Template Records are 360 received." 362 3.2.2. IPFIX Export per SCTP Stream Advantages 364 By exporting each Template Record and the corresponding Data 365 Records within a single SCTP stream and imposing in-order 366 transmission, the Template Record will always arrive before 367 the associated Data Records. Therefore, there is no risk 368 that the Collecting Process discards Data Records while 369 waiting for the Template Record to arrive. 371 Furthermore, when reusing a Template ID within an SCTP 372 stream, the Template Withdrawal Message will be guaranteed to 373 arrive before the new definition of the Template and 374 therefore the Template Record may be sent directly after the 375 Template Withdrawal Message. In other words, the Template 376 Reuse Delay restriction (by default, 5 seconds, as specified 377 in [RFC5101] is removed for Template ID reuse within the same 378 SCTP stream. 380 Another advantage of the new specifications in this document 381 is a reduced load on the Collecting Process. Indeed, the 382 Collecting Process doesn't have to store the Data Records 383 while waiting for the Template Record, as the transmission 384 order is always guaranteed. This way, extra reliability of 385 the Data Records is achieved without extra burden on the 386 Collecting Process. 388 3.3. No Transmission Order across SCTP Streams 390 3.3.1. IPFIX Protocol Specifications Limitation 392 The fact that the protocol specifications [RFC5101] are 393 flexible in terms of SCTP stream(s) on which the Template 394 Set, Options Template Set, and corresponding Data Sets are 395 exported, implies that the (Options) Template Record might be 396 exported on a different SCTP stream than the corresponding 397 Data Records. This might cause Data Record loss in the 398 Collecting Process as ordered transmission across SCTP 399 streams is not guaranteed. 401 For example, a Template Record may be blocked pending 402 reliable transmission on one SCTP stream while the 403 corresponding Data Records may be transmitted immediately in 404 another SCTP stream. Also, due to different SCTP stream 405 congestion, it is possible that even if the Template Record 406 and corresponding Data Records are sent reliably, Data 407 Records sent on a different SCTP stream than the Template 408 Record might still arrive before the Template Record. 410 3.3.2. IPFIX Export per SCTP Stream Advantages 412 By exporting each Template Record and all corresponding Data 413 Records within a single SCTP stream, and imposing in-order 414 transmission, the issue of ordered transmission across 415 multiple SCTP streams is avoided. 417 By exporting all corresponding Data Records within the same 418 ordered SCTP stream as the Template Record, each SCTP stream is 419 independent and self-contained and the interaction between SCTP 420 streams is limited to that of Options Template and associated 421 Data Records sent in different streams. This has several 422 advantageous consequences, including the order preservation that 423 does not result in the blocking of unrelated data and load 424 reduction on the Collecting Process (as the Template Records are 425 guaranteed to be delivered before the associated Data Records, 426 there is no need for the buffering of Data Sets which correspond 427 with Templates that are missing). 429 4. Specifications 431 This section specifies Exporting Process and Collecting Process 432 behavior different from that in [RFC5101] in order to realize 433 the benefits of per-stream export. Note that Exporting Processes 434 following these specifications will interoperate with [RFC5101]- 435 compliant Collecting Processes, but that Collecting Processes 436 will have to follow additional non-interoperable specifications 437 to realize the full benefits of the technique. These new 438 specifications, which add to those in [RFC5101], are described 439 with the key words described in [RFC2119]. 441 4.1. New Information Element 443 dataRecordsReliability 445 Description: 446 The export reliability of Data Records, within this SCTP 447 stream, for the element(s) in the Options Template 448 scope. A typical example of element for which the 449 export reliability must be reported is the templateID, 450 as a specified in the Data Record Reliability Options 451 Template. A value of 'true' means that the Exporting 452 Process MUST send any Data Records associated with the 453 element(s) reliably within this SCTP stream. A value of 454 'false' means that the Exporting Process MAY send any 455 Data Records associated with the element(s) unreliably 456 within this SCTP stream. 458 Abstract Data Type: boolean 459 Data Type Semantics: identifier 460 ElementId: XXX 461 Status: current 463 IANA NOTE: IANA should replace XXX with the assigned value 465 4.2. Template Management 467 To take advantage of per-stream export, Exporting Processes MUST 468 follow the specification in this section in addition to Section 8, 469 Template Management, of [RFC5101]. 470 As specified in [RFC5101], Template Sets and Options Template 471 Sets MUST be sent reliably. 473 Any Data Sets associated with a Template Record MUST be sent on 474 the same SCTP stream on which the Template Record was sent. 476 The Data Record Reliability Options Template is used to 477 explicitly inform the Collecting Process which Templates will be 478 used in each SCTP stream and whether each set of associated Data 479 Records will be sent reliably or unreliably. Before sending any 480 Data Records on an SCTP stream, the Exporting Process MUST 481 notify the Collecting Process of its intention to send those 482 Data Records reliably or unreliably within that SCTP stream. It 483 does this by sending a Data Record defined by the Data Record 484 Reliability Options Template for the Template associated with 485 Data Records to be sent. The one exception to this rule is that 486 the Data Records associated with the Data Record Reliability 487 Options Template don't require an explicit notification as these 488 MUST always be sent reliably. 490 The Data Record Reliability Options Template MUST contain the 491 following Information Elements: 493 Scope: Template ID 494 Non-scope: dataRecordsReliability 496 A value of 'true' for the dataRecordsReliability Element means 497 that the Exporting Process MUST send any Data Records associated 498 with the Template ID reliably within this SCTP stream. A value 499 of 'false' for the dataRecordsReliability Element means that the 500 Exporting Process MAY send any Data Records associated with the 501 Template ID unreliably within this SCTP stream. 503 If the Exporter wants to change the export reliability value 504 (from reliable to unreliable, or vice-versa) for Data Records on 505 an SCTP stream, the Template MUST be withdrawn, and a new 506 Template MUST be used. 508 The Data Record Reliability Options Template MAY contain other 509 non-scope Information Elements associated with the (Options) 510 Template. 512 When an Options Template, including the Data Record Reliability 513 Options Template, and associated Data Records are sent in the 514 same SCTP stream, the first associated Data Record can follow 515 the Options Template immediately. When the Options Template and 516 associated Data Records are sent in different SCTP streams, the 517 Exporting Process SHOULD transmit the Options Template in 518 advance of any Data Sets that use it, to help ensure that the 519 Collector has received the Options Template Record before 520 receiving the first associated Data Record. 522 It is RECOMMENDED that the Exporter only sends a single Template 523 and corresponding Data Sets within a single SCTP stream in order 524 to enable calculation of the potential Data Record loss for this 525 Template. The Exporter MAY group related (Options) Templates 526 and their associated Data Records within a single SCTP stream so 527 that loss statistics are calculated for the group of Templates 528 that are being sent unreliably within the SCTP stream. This is 529 suitable in cases where there are only slight variations among 530 the Templates in a group (e.g., the omission of unavailable 531 fields for export efficiency) and may be necessary if the SCTP 532 association does not support enough SCTP streams to export each 533 Template in its own SCTP stream. 535 If an SCTP stream contains a mixture of Data Records defined by 536 Template Records and Options Template Records, the Data Records 537 defined by the Options Template Records SHOULD be sent reliably 538 so that the Collector does not consider any loss to be 539 associated with the options Data Records. 541 4.3. SCTP 543 To take advantage of per-stream export, Exporting Processes MUST 544 manage SCTP streams according to the specification in this 545 section, in addition to Section 10.2.4.3, Stream, of [RFC5101]. 547 PR-SCTP [RFC3758] MUST be implemented by all compliant 548 implementations. 550 All IPFIX Messages in an SCTP stream MUST be sent in order. 551 As specified in [RFC5101], depending on the requirements of the 552 application, the Exporting Process may send Data Sets with full 553 or partial reliability. 555 If the Exporting Process is required to export a new Template 556 Record but there are no more free SCTP streams available, it 557 SHOULD attempt to increase the number of outbound SCTP streams 558 it is able to send to, per [SCTP-RESET]. Alternatively, the 559 Exporting Process MAY add the Template Set and Data Records to 560 an existing SCTP stream at the cost of diluting the granularity 561 of Data Records loss. An alternative, which may result in the 562 loss of Flow Records (for example, due to lack of buffering on 563 the Exporter), is to restart the SCTP association with an 564 increased number of SCTP streams. 566 4.4. Template Withdrawal Message 568 To take advantage of per-stream export, Exporting Processes MUST 569 send Template Withdrawal Messages according to the specification 570 in this section, in addition to Section 8, Template Management, 571 of [RFC5101]. 573 As specified in [RFC5101], Templates which are not used anymore 574 SHOULD be deleted. Before reusing a Template ID, the Template 575 MUST be deleted. In order to delete an allocated Template, the 576 Template is withdrawn through the use of a Template Withdrawal 577 Message. 579 The Template Withdrawal Message MUST be sent on the same SCTP 580 stream as the associated Template Record. 582 As the Template Withdrawal Message MUST be sent reliably, using 583 SCTP-ordered delivery per [RFC5101], and as all IPFIX Messages 584 are sent in order within an SCTP stream (per the specifications 585 in this document), the IPFIX Message containing the Template 586 Withdrawal Message will not arrive at the Collecting Process 587 before any associated and previously sent Data Record. As a 588 consequence, no Data Records will be lost due to delayed arrival 589 at the Collector. 591 The Template ID from a withdrawn Template MAY be reused on the 592 same SCTP stream immediately after the Template Withdrawal 593 Message is sent. This case is equivalent to the use of a 594 Template Reuse Delay value of 0. 596 After reusing the Template ID, the Exporting Process MUST send a 597 Data Record associated with the Data Record Reliability Options 598 Template to specify the reliability level of the Data Records 599 associated with the new Template. 601 If the Template ID is to be reused on a different SCTP stream, 602 the new Template Record MUST NOT be sent before the Template 603 Reuse Delay. 605 A Template Withdrawal Message to withdraw all Templates for the 606 Observation Domain ID specified in the IPFIX Message header MUST 607 NOT be used. 609 Multiple Template IDs MAY be withdrawn with a single Template 610 Withdrawal Message under the condition that all the Template IDs 611 in the Template Withdrawal Message are used on the same SCTP 612 stream as the Template Withdrawal Message. 614 4.5. The Collecting Process's Side 616 Collecting Processes must operate slightly contrary to [RFC5101] 617 in order to realize the full benefits of per-stream export. 618 However, the specification in this section contains a mechanism 619 which allows per-stream-capable Collecting Processes to 620 selectively enable per-stream export, in order to ensure 621 interoperability of per-stream-capable Collecting Processes with 622 Exporting Processes which do not implement per-stream export. 624 As specified in [RFC5101], the Collecting Process SHOULD listen 625 for a new association request from the Exporting Process. The 626 Exporting Process will request a number of SCTP streams to use 627 for export. 629 A Collecting Process SHOULD support the procedure for the 630 addition of an SCTP stream [SCTP-RESET]. 632 In IPFIX, there is no explicit notification of the Exporting 633 Process's capabilities. There is also no return channel for the 634 Collecting Process to communicate its capabilities. 636 In the case where the Exporting Process uses the per-SCTP-stream 637 extension, the first Data Record received by the Collecting 638 Process MUST be associated with the Data Records Reliability 639 Options Template. If the first Data Record is associated with 640 any other (Options) Template, the Collecting Process MUST 641 disable the extension for the specific Exporter on the 642 Collecting side. 644 The Collecting Process MUST accept other non-scope Information 645 Elements in the Data Record Reliability Options Template. 647 As specified in [RFC5101], the IPFIX protocol has a Sequence 648 Number field in the IPFIX Message header that increases with the 649 number of IPFIX Data Records in the IPFIX Message. A Collector 650 may detect out-of-sequence, dropped, or duplicate IPFIX Messages 651 by tracking the Sequence Number. 653 When one or more sequential IPFIX Messages are considered lost, 654 the number of lost Data Records is equal to the Sequence Number 655 of the first IPFIX Message Header following the lost packets 656 (S2) minus the Sequence Number of the first lost IPFIX Message 657 (S1). The Sequence Number of the first lost IPFIX Message can 658 be calculated as the Sequence Number of the last IPFIX Message 659 before the sequence of lost IPFIX Messages (S0) plus the number 660 of Data Records in that IPFIX Message (N0). 662 S1 = S0 + N0 663 loss = (S2 - S1) (mod 2^32) 664 = (S2 - (S0 + N0)) (mod 2^32) 666 Note that molulo 2^32 arithmetic is required since the Sequence 667 Number may wrap once or multiple times in the series of lost 668 IPFIX Messages. If less than 2^32 Data Records are lost in a 669 sequence (which can be assumed in practice), the above equation 670 returns the exact number of lost Data Records. 672 Note that using a unsigned32 type for the loss would 673 automatically take care of the mod(2^32) operation. 675 As this Sequence Number is incremented per SCTP stream, the loss 676 of Data Records sent in that SCTP stream can be calculated in 677 case of partially-reliable export. This loss can be attributed 678 to the Data Records sent for the (Options) Template(s) whose 679 records are being sent unreliably within that SCTP stream. 681 Once the Collecting Process receives a Data Record Reliability 682 Options Data Record for a particular Template, if the Collecting 683 Process receives a Data Record or a Template Withdrawal Message 684 for the same Template on a different SCTP stream, then the 685 Collecting Process SHOULD log an error message and 'disable' 686 this extension for the SCTP association. 688 5. Performance Impact 690 Although adding the new SCTP streams requires a message 691 exchange, it is more lightweight to set up additional SCTP 692 streams than to set up a new SCTP association since the only 693 overhead of adding SCTP stream(s) to an existing SCTP 694 association is the addition of 16-24 more bytes (allocated in 695 the SCTP association, a single time), whereas setting up a new 696 SCTP association implies more overhead. 698 In terms of throughput impact, the fact that these 699 specifications discourage multiplexing Templates and Data 700 Records of different Template IDs may lead to a slightly larger 701 IPFIX Message overhead. If the Data Record rate is low for a 702 specific Template (hence a specific SCTP stream), the Exporting 703 Process might not be able to fill the IPFIX Messages with Data 704 Records associated with other Templates. In such a situation, 705 there is a potential overhead due to additional IPFIX Message 706 headers and SCTP chunk headers. 708 Finally, with respect to the processing overhead on the 709 Exporter, a lot of state information must be stored when a large 710 number of SCTP streams are used within an SCTP association. 711 However, no comparison of the performance impact of multiple 712 streams within an SCTP association versus opening the same 713 number of independent SCTP associations is available. 715 6. Guidelines for IPFIX per-SCTP-stream Extension Testing 717 This section specifies guidelines for a series of tests that can 718 be run on the Collecting Process in order to probe the 719 conformity and robustness of the IPFIX per-SCTP-stream extension 720 protocol implementations. 722 For example, nothing prevents an implementation that does not 723 meet the specification of the per-SCTP-stream extension from 724 sending a Template that looks like a dataRecordsReliability 725 Options Template. Therefore, a Collecting Process MUST detect 726 if the Exporter fails to meet the specification fully. If any 727 of the conditions below is met, the Exporting Process does not 728 properly use the per-SCTP-stream extension, and the Collecting 729 Process MUST report an error message: 731 1. A Data Record is received before the appropriate Data 732 Record associated with the Data Records Reliability Options 733 Template has been received on the same SCTP stream (see 734 section 4.1). 736 2. A Data Record associated with a Data Record 737 ReliabilityOptions Template is received on an SCTP stream 738 for a (non-Options) Template that was defined on a 739 different SCTP stream. 741 3. Loss of Data Records is detected within a stream where 742 there has not been received a Data Record associated with 743 the Data Record Reliability Options Template indicating 744 unreliable transmission for any template. 746 4. A message is received with the SCTP U(nordered) flag set 747 to 1, (i.e., the message was sent unordered) even if it 748 isprocessed in order. 750 7. Examples 752 Figure 1 shows an example where SCTP stream 10 carries a 753 Template Record with the Template ID 256 transmitted with full 754 reliability (FR), together with associated Data Records 755 transmitted with partial reliability (PR). The Data Record 756 Reliability Options Template with Template ID 257 is transmitted 757 with full reliability (FR). Its corresponding Data Set contains 758 one Data Record. 759 Record 1: 761 o Scope: Template ID = 256 762 o Non-scope: dataRecordsReliability = False 764 +--------+ +---------+ +--------+ 765 | | | | | | 766 stream 10 ----| Data | . . . | Data |---| Data |---... 768 | 256 | | 256 | | 257 | 769 | PR| | PR| | FR| 770 +--------+ +---------+ +--------+ 772 +----------+ +------------+ 773 | | | Options | 774 | | | Reliability| 775 ...---| Template |-------| Template |------> 776 | 256 | | 257 | 777 | FR| | FR| 778 +----------+ +------------+ 780 Figure 1 782 Note that Template 256 will always be processed before the Data 783 Records by the Collecting Process because all IPFIX Messages are 784 sent in order within an SCTP stream. Therefore, the Collecting 785 Process job is simplified. Furthermore, the Data Record loss 786 for the Template 256 can easily be calculated on the Collecting 787 Process. 789 If an Options Template is necessary to understand the content of 790 a Data Record (i.e., the scope in the Options Template Record is 791 an Information Element contained in the Data Record or 792 associated with the Data Record), the Options Template Record 793 should be sent in the same SCTP stream, as displayed in figure 794 2. 796 +--------+ +--------+ +--------+ 797 | | | | | | 798 stream 20 ----| Data |...| Data |-----| Data |--- ... 799 | 260 | | 260 | | 259 | 800 | PR| | PR| | FR| 801 +--------+ +--------+ +--------+ 803 +--------+ +----------+ 804 | | | | 805 ...---| Data |-------| Template |---... 806 | 258 | | 260 | 807 | FR| | FR| 808 +--------+ +----------+ 810 +----------+ +-------------+ 811 | Options | | Options | 812 | Template | | Reliability | 813 ...---| |-------| Template |------> 814 | 259 | | 258 | 815 | FR| | FR| 816 +----------+ +-------------+ 818 Figure 2 820 Figure 2 shows an example where SCTP stream 20 carries: 821 - a Data Record Reliability Options Template with Template ID 822 258, transmitted with full reliability (FR) 823 - an Options Template Record with Template ID 259 transmitted 824 with full reliability. This Options Template Record contains 825 additional information related to the subsequent Data Records 826 based on Template ID 260. Typical examples are the Common 827 Properties information [RFC5473] or a Selector Report 828 Interpretation [RFC5476]. 829 - a Template Record with Template ID 260, transmitted with full 830 reliability. 831 - a Data Set specified by the Reliability Options Template with 832 Template ID 258 transmitted with full reliability. 833 The Data Set contains three Data Records. 834 Record 1: 835 o Scope: Template ID = 258 836 o Non-scope: dataRecordsReliability = True 837 Record 2: 838 o Scope: Template ID = 259 839 o Non-scope: dataRecordsReliability = True 840 Record 3: 841 o Scope: Template ID = 260 842 o Non-scope: dataRecordsReliability = False 843 These Data Records inform the Collector that the Data Records 844 for Template ID 258 and 259 are sent reliably, while the Data 845 Records for Template ID 260 are not. 846 - a Data Record specified by Template ID 259, transmitted with 847 full reliability 848 - a Data Record specified by Template ID 260, transmitted with 849 partial reliability 851 If the Collector observes some Data Record loss using the 852 Sequence Number, the loss can only stem from the Data Records 853 associated with Template ID 260, as these are the only Data 854 Records not exported reliably. Therefore, the calculation of 855 loss per Template ID 260 is possible. 857 Note that the Options Templates 258, 259, and 260 will always 858 arrive before their associated Data Records, respectively, 859 because all IPFIX Messages must be sent in order within an SCTP 860 stream. 862 Figure 3 shows an example where SCTP stream 30 carries a 863 Template Record with Template ID 262 transmitted with full 864 reliability (FR), an associated Data Record transmitted with 865 full reliability (FR), a Template Withdrawal Message, followed 866 by a redefinition of the Template ID 262, and finally the Data 867 Record associated with the new Template transmitted with partial 868 reliability. The Template Withdrawal Message and the new 869 definition of the Template ID 262 are sent immediately, without 870 waiting for the Template Reuse Delay. 872 +--------+ +----------+ +----------+ 873 | | |Data | | | 874 stream 30 ... ---| Data |...| 261 |-----| Template |--- 875 | 262 | |tmpID: 262| | 262 | 876 | PR| |dRR: false| | FR| 877 +--------+ +----------+ +----------+ 879 +----------+ +--------+ +----------+ 880 | Template | | | | Data | 881 ...| Withdraw |-----| Data |-------| 261 |---... 882 | 262 | | 262 | |tmpID: 262| 883 | FR| | FR| |dRR: true| 884 +----------+ +--------+ +----------+ 886 +----------+ +-------------+ 887 | | | Options | 888 | Template | | Reliability | 889 ...---| |-------| Template |------> 890 | 262 | | 261 | 891 | FR| | FR| 892 +----------+ +-------------+ 894 Figure 3 896 The second Data Record associated with the Data Record 897 Reliability Options Template shows that the Data Records 898 associated with the newly specified Template ID 262, will be 899 sent unreliably. 901 8. IANA Considerations 903 According to the process defined in [RFC5102], IANA will 904 allocate the dataRecordsReliability Information Element defined 905 in Section 4.1. in the IANA IPFIX Information Elements 906 registry. 908 9. Security Considerations 910 The same security considerations as for the IPFIX Protocol 911 [RFC5101] apply. 913 10. References 915 10.1. Normative References 917 [RFC2119] S. Bradner, Key words for use in RFCs to Indicate 918 Requirement Levels, BCP 14, RFC 2119, March 1997 920 [RFC3758] Stewart, R., Ramalho, M, Xie, Q., Tuexen, M., and P. 921 Conrad, "Stream Control Transmission Protocol (SCTP), 922 Partial Reliability Extension", May 2004 924 [RFC4960] Stewart, R., Ed., "Stream Control Transmission 925 Protocol", RFC 4960, September 2007. 927 [RFC5101] Claise, B., Ed., "Specification of the IP Flow 928 Information Export (IPFIX) Protocol for the Exchange of 929 IP Traffic Flow Information", RFC 5101, January 2008. 931 [RFC5102] Quittek, J., Bryant, S., Claise, B., Aitken, P., and 932 J. Meyer, "Information Model for IP Flow Information 933 Export", RFC 5102, January 2008. 935 [RFC5475] Zseby, T., Molina, M., Duffield, N., Niccolini S., and 936 F. Raspall, "Sampling and Filtering Techniques for IP 937 Packet Selection", RFC5475, March 2009 939 [SCTP-RESET] Stewart, R., Lei, P., Tuexen, M, "Stream Control 940 Transmission Protocol (SCTP) Stream Reconfiguration", 941 draft-ietf-tsvwg-sctp-strrst-04, Internet-Draft work in 942 progress, February 2010 944 10.2. Informative References 946 [RFC3917] Quittek, J., Zseby, T., Claise, B., and S. Zander, 947 Requirements for IP Flow Information Export, RFC 3917, 948 October 2004 950 [RFC5470] Sadasivan, G., Brownlee, N., Claise, B., and J. 951 Quittek, "Architecture Model for IP Flow Information 952 Export", RFC5470, March 2009 954 [RFC5472] Zseby, T., Boschi, E., Brownlee, N., and B. Claise, 955 "IP Flow Information Export (IPFIX) Applicability", RFC 956 5472, March 2009 958 [RFC5477] T. Dietz, F. Dressler, G. Carle, and B. Claise, 959 "Information Model for Packet Sampling Exports", RFC 960 5477, March 2009 962 [RFC5476] Claise, B., Quittek, J., and A. Johnson, "Packet 963 Sampling (PSAMP) Protocol Specifications", RFC 5476, 964 March 2009. 966 [RFC5474] Chiou, D., Claise, B., Duffield, N., Greenberg, A., 967 Grossglauser, M., Marimuthu, P., Rexford, J., and G. 968 Sadasivan, RFC 5474, March 2009 970 [RFC5473] Boschi, E., Mark, L., and B. Claise, " Reducing 971 Redundancy in IP Flow Information Export (IPFIX) and 972 Packet Sampling (PSAMP) Reports", RFC 5473, March 2009 974 [PSAMP-MIB] Dietz, T., and B. Claise, "Definitions of Managed 975 Objects for Packet Sampling", Internet-Draft work in 976 progress, June 2006 978 11. Acknowledgements 980 The authors would like to thank Brian Trammell for his expert 981 feedback and continuous effort to improve the specifications, 982 Elisa Boschi for her thorough reading, and Randall Stewart, 983 Peter Lei, Michael Tuexen for their SCTP-related feedback and 984 expertise, and Tobias Limmer. 986 12. Author's Addresses 988 Benoit Claise 989 Cisco Systems Inc. 990 De Kleetlaan 6a b1 991 Diegem 1813 992 Belgium 994 Phone: +32 2 704 5622 995 Email: bclaise@cisco.com 997 Paul Aitken 998 Cisco Systems (Scotland) Ltd. 999 96 Commercial Quay 1000 Commercial Street 1001 Edinburgh, EH6 6LX, United Kingdom 1003 Phone: +44 131 561 3616 1004 Email: paitken@cisco.com 1006 Andrew Johnson 1007 Cisco Systems (Scotland) Ltd. 1008 96 Commercial Quay 1009 Commercial Street 1010 Edinburgh, EH6 6LX, United Kingdom 1012 Phone: +44 131 561 3641 1013 Email: andrjohn@cisco.com 1015 Gerhard Muenz 1016 Technische Universitaet Muenchen 1017 Departement of Informatics - I8 1018 Boltzmannstr. 3 1019 Garching D-85748 1020 DE 1022 Phone: +49 89 289-18008 1023 Email: muenz@net.in.tum.de 1024 URI: http://www.net.in.tum.de/~muenz