idnits 2.17.1 draft-pohl-perpktinfo-02.txt: -(51): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(52): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(53): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(54): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(55): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(56): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(57): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(58): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(59): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(60): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(61): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(62): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(63): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(64): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(65): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(66): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(67): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** It looks like you're using RFC 3978 boilerplate. You should update this to the boilerplate described in the IETF Trust License Policy document (see https://trustee.ietf.org/license-info), which is required now. -- Found old boilerplate from RFC 3978, Section 5.1.a on line 19. -- Found old boilerplate from RFC 3978, Section 5.5 on line 487. -- Found old boilerplate from RFC 3979, Section 5, paragraph 1 on line 457. -- Found old boilerplate from RFC 3979, Section 5, paragraph 3 on line 469. ** The document seems to lack an RFC 3978 Section 5.1 IPR Disclosure Acknowledgement. ** This document has an original RFC 3978 Section 5.4 Copyright Line, instead of the newer IETF Trust Copyright according to RFC 4748. ** This document has an original RFC 3978 Section 5.5 Disclaimer, instead of the newer disclaimer which includes the IETF Trust according to RFC 4748. ** The document seems to lack an RFC 3979 Section 5, para. 2 IPR Disclosure Acknowledgement -- however, there's a paragraph with a matching beginning. Boilerplate error? ** The document uses RFC 3667 boilerplate or RFC 3978-like boilerplate instead of verbatim RFC 3978 boilerplate. After 6 May 2005, submission of drafts without verbatim RFC 3978 boilerplate is not accepted. The following non-3978 patterns matched text found in the document. That text should be removed or replaced: This document is an Internet-Draft and is subject to all provisions of Section 3 of RFC 3667. By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == There are 17 instances of lines with non-ascii characters in the document. == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 8 longer pages, the longest (page 2) being 60 lines Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack a both a reference to RFC 2119 and the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. RFC 2119 keyword, line 210: '...erties templates SHOULD be sent before...' Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (February 2005) is 7003 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: 'DuGG03' is defined on line 395, but no explicit reference was found in the text == Unused Reference: 'DuGr00' is defined on line 399, but no explicit reference was found in the text == Unused Reference: 'QuZC04' is defined on line 404, but no explicit reference was found in the text == Unused Reference: 'GrDM98' is defined on line 408, but no explicit reference was found in the text == Unused Reference: 'TPBC03' is defined on line 414, but no explicit reference was found in the text -- Possible downref: Non-RFC (?) normative reference: ref. 'Cla104' -- Possible downref: Non-RFC (?) normative reference: ref. 'Cla204' Summary: 7 errors (**), 0 flaws (~~), 9 warnings (==), 8 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Draft G. Pohl 3 Document: Fraunhofer FOKUS 4 Expires: August 2005 L. Mark 5 Fraunhofer FOKUS 6 E. Boschi 7 Fraunhofer FOKUS 8 February 2005 10 Use of IPFIX for Export of Per-Packet Information 12 Status of this Memo 14 This document is an Internet-Draft and is subject to all 15 provisions of section 3 of RFC 3667. By submitting this 16 Internet-Draft, each author represents that any applicable 17 patent or other IPR claims of which he or she is aware have been 18 or will be disclosed, and any of which he or she become aware 19 will be disclosed, in accordance with RFC 3668. 21 Internet-Drafts are working documents of the Internet 22 Engineering Task Force (IETF), its areas, and its working 23 groups. Note that other groups may also distribute working 24 documents as Internet-Drafts. 25 Internet-Drafts are draft documents valid for a maximum of six 26 months and may be updated, replaced, or obsoleted by other 27 documents at any time. It is inappropriate to use 28 Internet-Drafts as reference material or to cite them other than 29 as "work in progress." 31 The list of current Internet-Drafts can be accessed at 32 http://www.ietf.org/ietf/1id-abstracts.txt. 34 The list of Internet-Draft Shadow Directories can be accessed at 35 http://www.ietf.org/shadow.html. 37 Abstract 39 This document describes the usage of the IP Flow Information 40 Export (IPFIX) protocol for the case of exporting and processing 41 per-packet information. 42 The main idea is to export two types of records per flow: the 43 first one contains the usual flow information plus a unique flow 44 identifier while the other one consists of the actual per-packet 45 information plus a flow identifier that refers to the flow the 46 specific packet belongs to -- that means the flow identifier is 47 used to associate the packet data to its corresponding flow. 49 Table of Contents 51 1. Introduction��������������������������������������������2 52 2. Terminology���������������������������������������������2 53 3. General Problem Statement�������������������������������3 54 4. Export Per-Packet Information���������������������������3 55 5. Flow ID Management��������������������������������������5 56 6. Example of Per-Packet Information Export����������������5 57 7. IPFIX for per-packet information export and PSAMP�������6 58 8. Export and evaluation considerations��������������������6 59 9. IANA Consideration��������������������������������������7 60 10. Security Considerations���������������������������������7 61 11. References����������������������������������������������7 62 11.1 Normative References����������������������������������7 63 11.2 Informative References��������������������������������7 64 12. Author's Addresses��������������������������������������8 65 13. Intellectual Property Statement�������������������������8 66 14. Copyright Statement�������������������������������������9 67 15. Disclaimer����������������������������������������������9 69 1. Introduction 71 In the scope of passive QoS Measurements, there is often the 72 need to exchange and export measurement data in a finer 73 granularity then per flows. One typical application is passive 74 One-Way-Delay measurement; this draft takes it as example when 75 demonstrating the need for information export on a per-packet 76 basis. 78 The IPFIX protocol however, has been designed to export flow 79 records. A possible approach to export packet records using 80 IPFIX could be exporting flow records containing information 81 about single packets. This method has been proposed by the PSAMP 82 working group in [Cla204]. Exporting flow related information 83 per-packet introduces a high degree of redundancy. This draft 84 shows how packet information and flow information can be 85 efficiently exported and related using IPFIX. 87 2. Terminology 89 Collecting Process 90 The collecting process receives records of flow or packet 91 information. The data is stored for later processing (by 92 Exporting Process 93 The exporting processes send flow and packet records to the 94 collecting processes. The records are generated by the 95 measurement process. 97 Filtering 98 Filtering selects a subset of packets by applying 99 deterministic functions on parts of the packet content like 100 header fields or parts of the payload. A filtering process 101 needs to process the packet (look at packet header and/or 102 payload) in order to make the selection decision. 104 Measurement Device 105 A measurement device has access to at least one observation 106 point. It is hosting at least one measurement process and 107 one export process. 109 Metering/Measurement Process 110 The measurement process generates records of packet and 111 flow information. Packets passing the observation point are 112 captured, time stamped, filtered and classified. The 113 measurement process calculates the packet Ids. 115 Passive One-Way-Delay Measurement 116 Abbreviated: POWD Measurement 118 3. General Problem Statement 120 In [Cla104] the IPFIX working group has defined a protocol to 121 transport measurement data containing flow information. 123 The main purpose of the protocol is to exchange information 124 about IP traffic flows. In this scope a flow is defined by a set 125 of key attributes (source/destination address, 126 source/destination port, Layer3 Protocol Type, TOS/DSCP byte, 127 interface of the flow exporting network element). As such, a 128 flow is a collection of packets that share a set of common 129 attributes. 131 However, for a number of metrics there is a need to export 132 per-packet data. 134 Undoubtedly a single packet could be considered a special case 135 of a flow and thus, per-packet information could be exported 136 using flow records. Doing this though would have consequences on 137 the efficiency of the exporting procedure, as it would mean 138 additional overhead. Packets belonging to the same flow share 139 common attributes, i.e. source address, destination address, 140 etc. Exporting these attributes on a per-packet basis, each time 141 with a different packet ID, would be redundant information. 143 There are cases however, where it is desirable to keep flow 144 information along with the per-packet information, that is, when 145 analyzing packet characteristics while observing flows. This 146 document proposes a solution that reduces the overhead caused by 147 the flow properties while keeping a link to flow information. 149 The proposed method does not need any changes to the IPFIX 150 protocol. 152 4. Export Per-Packet Information 153 packet that belongs to flow B, respectively. It shows export 154 records containing packet information plus flow information 155 (source and destination address). Undoubtedly, the flow 156 information introduces a huge amount of redundancy, as it is 157 repeated for every packet in every record. Minimizing the 158 redundancy is a common problem in relational data base design 159 and we apply here similar solutions to those proposed in that 160 area. 162 In Figure 2 we separate flow from packet information. In order 163 to maintain the relation between Packet Properties and Flow 164 Properties we introduce indices (idxA and idxB) for the Flow 165 Properties that are unique for all Flow Property entries. The 166 purpose of the indices is to serve as "primary key" that 167 identifies rows of the Flow Properties. The rows are then 168 referenced by the Packet Properties by using the appropriate 169 value for the flow identifier. 171 One-packet flows 172 +------+------+------------+---------+ 173 | srcA | dstA | packet info| ... | 174 +------+------+------------+---------+ 175 | srcA | dstA | packet info| ... | 176 +------+------+------------+---------+ 177 | srcB | dstB | packet info| ... | 178 +------+------+------------+---------+ 179 | srcA | dstA | packet info| ... | 180 +------+------+------------+---------+ 182 Figure 1: Flow and packet information represented in one-packet 183 flows 185 Packet Properties 186 +-----+------------+---------+ 187 Flow Properties >idxA | packet info| ... | 188 +------+------+-----+ +-----+------------+---------+ 189 | srcA | dstA |idxA < >idxA | packet info| ... | 190 +------+------+-----+ +-----+------------+---------+ 191 | srcB | dstB |idxB <-------->idxB | packet info| ... | 192 +------+------+-----+ +-----+------------+---------+ 193 >idxB | packet info| ... | 194 +-----+------------+---------+ 196 here, the linkage of one packet and 197 flow B (srcB, dstB, idxB) is explicitly drawn 199 Figure 2: Flow information and packet information 201 The IPFIX protocol is template based like NetFlow version 9. For 202 a complete description of features of IPFIX refer to [Fehler! 203 Verweisquelle konnte nicht gefunden werden.]. 204 Templates define the structure of data to be exported, including 205 describing data fields to be exported together with their type 206 and meaning. 208 For Measurement Process Results we define two different template 209 records, namely Flow Properties and Packet Properties. The Flow 210 Properties templates SHOULD be sent before the Packet Properties 211 Templates. 213 In Figure 3, the Flow Properties template defines the 214 attributes for a flow; e.g. IP source and destination address 215 and the flow identifier. The flow identifier is a unique 216 identifier for flow definitions; this field allows packet 217 this template define the actual flows. 219 The format for the information related to single packets is 220 defined in the Packet Properties template. This information is 221 packet specific and normally not shared between many packets. 222 Otherwise one would rather consider the information as flow 223 related and therefore it needs not be exported in every record. 225 +------------------+ +-------------------+ 226 | Template Set | | Template Set | 227 | | | | Description of 228 | Flow Properties | | Packet Properties | exported data 229 +----------+-------+ +----------+--------+ 230 ...........|.........................|......................... 231 | | 232 +----------v-------+ +----------v--------+ 233 | Data Set | | Data Set | exported data 234 | <------+ | with references 235 | Flow Properties | | Packet Properties | by means of 236 +------------------+ +-------------------+ flow identfier 238 Figure 3: Template FlowSet and Data FlowSet dependencies 240 5. Flow ID Management 242 Like template IDs the flow IDs have to be unique per observation 243 domain (source identifier in the IPFIX header). Using 32 bit 244 flow IDs allows the export of 2**32 active flows in parallel. 246 If it is desired, one can optionally use option templates to 247 specify the mapping between two flow and packet properties 248 templates. In this case, the flow ID only has to be unique 249 within this association. 251 6. Example of Per-Packet Information Export 253 To demonstrate how to use IPFIX efficiently to export per-packet 254 information, this section proposes how to use the IPFIX protocol 255 for exporting flow information and per-packet information (in 256 this case related to a long-lived flow) for OWD computation. 258 In order to acquire a One-Way path delay information, two 259 measurement points with synchronized clocks must exist, one at 260 each end of the path under examination. Both measurement points 261 will capture packets, assign them timestamps and generate an 262 identifier for a packet passing that point. Each measurement 263 point will export its measurement data to a collecting process 264 where the data are correlated based on the packet identifiers 265 and timestamps and then the delay is calculated as a difference 266 of two timestamps of a packet pair. 268 The templates that would be needed for exporting measurement 269 data of this kind are illustrated in Figure 4. 270 The upper part of the figure shows the template containing the 271 information concerning flows. The lower part displays the 272 template with the packet properties. 274 For passive One-Way-Delay measurement, the Packet Properties 275 template consists of at least Timestamp and Packet ID. 276 Additionally, this template contains a flow identifier field. In 277 unique indices of the flow records exported before. 279 +-------------------+-------------------+-----------------------+.. 280 | flowID | sourceAddressV4 | destinationAddressV4 | 281 | | | | 282 | unsigned32/vendor | ipv4Address/ID 8 | ipv4Address/ID 12 | 283 +-------------------+-------------------+-----------------------+.. 285 ..+------------------+--------------------+---------------------+.. 286 | classOfServiceV4 | protocolIdentifier | transportSourcePort | 287 | | | | 288 | octet/ID 5 | octet/ID 4 | unsigned16/ID 7 | 289 ..+------------------+--------------------+---------------------+.. 291 ..+--------------------------+ 292 | transportDestinationPort | 293 | | 294 | unsigned16/ID 11 | 295 ..+--------------------------+ 296 FlowPropTemplate 297 =================================================================== 298 PacketPropTemplate 299 +-------------------+-------------------+-------------------+.. 300 | packetTimestamp | packetID | packetLength | 301 | | | | 302 | unsigned64/vendor | unsigned32/vendor | unsigned32/vendor | 303 +-------------------+-------------------+-------------------+.. 305 ..+-------------------+ 306 | flowID | 307 | | 308 | unsigned32/vendor | 309 ..+-------------------+ 311 Figure 4: Example Templates for Flow and Packet Properties 313 The delay is derived by a calculation step: At the collection 314 point packet records of two measurement points are gathered and 315 correlated by means of the packet ID. The resulting delay data 316 records are exported in a similar manner as the packet data have 317 been. Especially, the linkage between delay data and flow 318 information is represented with the discussed flow identifier 319 fields. The OWD properties contain the Packet Pair ID (which is 320 the packet ID matching that of the two contributing packet 321 records), a timestamp (which is the timestamp of the packet 322 passing the reference monitor point) in order to reconstruct a 323 time series, the calculated delay value, and finally a flow 324 identifier. 326 7. IPFIX for per-packet information export and PSAMP 328 In [Cla204] the PSAMP working group proposes to use IPFIX to 329 export packet information from a PSAMP Exporting Process to a 330 PSAMP Collecting Process. Even though no new version of the 331 draft has been produced the solution seems to be accepted from 332 the group. 333 While IPFIX is well suited for the purpose due to the good match 334 IPFIX satisfies PSAMP requirements, the described approach has a 335 high degree of redundancy. It proposes to treat packets as flows 336 and export per-packet information using flow records. 337 We propose to use the solution described in this draft to 338 efficiently export PSAMP packet information. 340 8. Export and evaluation considerations 341 The main advantage of this proposed method to export per-packet 342 information is the reduced amount of measurement data that has 343 to be transferred from the exporter to the collector. In 344 addition there is less storage capacity needed at the collector 345 side. 347 On the other hand there is some extra processing power needed on 348 the exporter side to manage flow information and to assign 349 packets to flows. The collector has to process records of two 350 templates instead of just one but has to read and write less 351 data. Additional effort is needed when post processing the 352 measurement data, because now the correlation of flow and packet 353 information is needed. 355 In the above example (see Figure 4) using IPFIX to export the 356 measurement data for each received packet 28 bytes have to be 357 transferred (sourceAddressV4=4, destinationAddressV4=4, 358 classOfServiceV4=1, protocolIdentifier=1, transportSourcePort=2, 359 transportDestionationPort=2, packetTimestamp=8, packetID=4, 360 packetLength=2). Disregarding the IPFIX protocol overhead a flow 361 of 1000 packets produces 28000 bytes of measurement data. Using 362 the proposed optimization each packet produces an export of only 363 16 bytes (packetTimestamp=8, packetID=4, packetLength=2, 364 flowID=2). The export of the flow information produces 16 bytes 365 (sourceAddressV4=4, destinationAddressV4=4, classOfServiceV4=1, 366 protocolIdentifier=1, transportSourcePort=2, 367 transportDestionationPort=2, flowID =2). For a flow of 1000 368 packet this sums up to 16016 bytes. This is a decrease of more 369 than 40 percent. 371 9. IANA Consideration 373 This document does not imply any IANA action. 375 10. Security Considerations 377 For the proposed use of the IPFIX protocol for export of 378 per-packet information the security considerations as for the 379 IPFIX protocol apply. 381 11. References 383 11.1 Normative References 385 [Cla104] Benoit Claise: IPFIX Protocol Specification, 386 IETF draft work in progress 387 , August 2004 389 [Cla204] Benoit Claise: PSAMP Protocol Specification, 390 Internet Draft , 391 February 2004 393 11.2 Informative References 395 [DuGG03] Nick Duffield, et Al.: A Framework for Packet 396 Selection and reporting, Internet Draft 397 , September 2004 399 [DuGr00] Nick Duffield, Matthias Grossglauser: Trajectory 400 Sampling for Direct Traffic Observation, 401 Proceedings of ACM SIGCOMM 2000, Stockholm, Sweden, 402 August 28 - September 1, 2000 404 [QuZC04] J. Quittek, et Al.: Requirements for IP Flow 405 Information Export, Internet Draft 406 , June 2004 408 [GrDM98] Ian D. GRAHAM, Stephen F. DONNELLY, Stele MARTIN, 409 Jed MARTENS, John G. CLEARY: Nonintrusive and 410 Accurate Measurement of Unidirectional Delay and 411 Delay Variation on the Internet, INET'98, Geneva, 412 Switzerland, July 21-24, 1998 414 [TPBC03] T. Zseby, E. Boschi, R. Penno, N. Brownlee, B. 415 Claise: IPFIX Applicability, Internet Draft 416 , July 2004 418 12. Author's Addresses 420 Elisa Boschi 421 Fraunhofer Institute for Open Communication Systems 422 Kaiserin-Augusta-Allee 31 423 10589 Berlin 424 Germany 425 Phone: +49-30-34 63 7366 426 Fax: +49-30-34 53 8366 427 Email: boschi@fokus.fraunhofer.de 429 Lutz Mark 430 Fraunhofer Institute for Open Communication Systems 431 Kaiserin-Augusta-Allee 31 432 10589 Berlin 433 Germany 434 Phone: +49-30-34 63 7306 435 Fax: +49-30-34 53 8306 436 Email: mark@fokus.fraunhofer.de 438 Guido Pohl 439 Fraunhofer Institute for Open Communication Systems 440 Kaiserin-Augusta-Allee 31 441 10589 Berlin 442 Germany 443 Phone: +49-30-34 63 7164 444 Fax: +49-30-34 53 8164 445 Email: pohl@fokus.fraunhofer.de 447 13. Intellectual Property Statement 449 The IETF takes no position regarding the validity or scope of 450 any Intellectual Property Rights or other rights that might be 451 claimed to pertain to the implementation or use of the 452 technology described in this document or the extent to which any 453 license under such rights might or might not be available; nor 454 does it represent that it has made any independent effort to 455 identify any such rights. Information on the procedures with 456 respect to rights in RFC documents can be found in BCP 78 and 457 BCP 79. 459 Copies of IPR disclosures made to the IETF Secretariat and any 460 assurances of licenses to be made available, or the result of an 461 attempt made to obtain a general license or permission for the 462 use of such proprietary rights by implementers or users of this 463 specification can be obtained from the IETF on-line IPR 465 The IETF invites any interested party to bring to its attention 466 any copyrights, patents or patent applications, or other 467 proprietary rights that may cover technology that may be 468 required to implement this standard. Please address the 469 information to the IETF at ietf-ipr@ietf.org. 471 14. Copyright Statement 473 Copyright (C) The Internet Society (2005). This document is 474 subject to the rights, licenses and restrictions contained in 475 BCP 78, and except as set forth therein, the authors retain all 476 their rights. 478 15. Disclaimer 480 This document and the information contained herein are provided 481 on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE 482 REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND 483 THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, 484 EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY 485 THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY 486 RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS 487 FOR A PARTICULAR PURPOSE.