idnits 2.17.1 draft-tuexen-opsawg-pcapng-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'MUST not' in this paragraph: A given option may have a fixed length, in which case all instances of that option have a length that is equal to the specified fixed length, or a variable length, in which case the option has a minimum length and all instances of that option must have a length equal to or greater than the specified minimum length. The length of fixed-length options, and the minimum length of variable-length options, is specified in the description of the option; if the minimum length of a variable-length option is not specified, a zero-length option is valid. Software that reads these files SHOULD report options that have an invalid length as errors; the software MAY stop processing the file if it sees an option that has invalid length, or MAY ignore the option and continue processing it. Software that writes these files MUST not write files with options that have invalid lengths. -- The document date (March 27, 2020) is 1484 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- -- Looks like a reference, but probably isn't: '1' on line 2229 -- Looks like a reference, but probably isn't: '2' on line 2231 -- Looks like a reference, but probably isn't: '3' on line 2233 -- Looks like a reference, but probably isn't: '4' on line 2235 -- Looks like a reference, but probably isn't: '5' on line 2237 -- Looks like a reference, but probably isn't: '6' on line 2239 -- Looks like a reference, but probably isn't: '7' on line 2241 -- Looks like a reference, but probably isn't: '8' on line 2243 -- Looks like a reference, but probably isn't: '9' on line 2245 -- Looks like a reference, but probably isn't: '10' on line 2248 -- Looks like a reference, but probably isn't: '11' on line 2250 -- Looks like a reference, but probably isn't: '12' on line 2252 -- Looks like a reference, but probably isn't: '13' on line 2255 -- Looks like a reference, but probably isn't: '14' on line 2257 -- Looks like a reference, but probably isn't: '15' on line 2259 -- Looks like a reference, but probably isn't: '16' on line 2261 -- Looks like a reference, but probably isn't: '17' on line 2263 -- Looks like a reference, but probably isn't: '18' on line 2266 -- Looks like a reference, but probably isn't: '19' on line 2268 -- Looks like a reference, but probably isn't: '20' on line 2270 -- Looks like a reference, but probably isn't: '21' on line 2273 -- Looks like a reference, but probably isn't: '22' on line 2276 -- Looks like a reference, but probably isn't: '23' on line 2278 -- Looks like a reference, but probably isn't: '24' on line 2281 -- Looks like a reference, but probably isn't: '25' on line 2284 -- Looks like a reference, but probably isn't: '26' on line 2286 -- Looks like a reference, but probably isn't: '27' on line 2288 -- Looks like a reference, but probably isn't: '28' on line 2290 -- Looks like a reference, but probably isn't: '29' on line 2292 -- Looks like a reference, but probably isn't: '30' on line 2294 -- Looks like a reference, but probably isn't: '31' on line 2296 -- Looks like a reference, but probably isn't: '32' on line 2298 -- Looks like a reference, but probably isn't: '33' on line 2300 -- Looks like a reference, but probably isn't: '34' on line 2302 -- Looks like a reference, but probably isn't: '35' on line 2304 -- Looks like a reference, but probably isn't: '36' on line 2306 -- Looks like a reference, but probably isn't: '37' on line 2308 -- Looks like a reference, but probably isn't: '38' on line 2395 Summary: 0 errors (**), 0 flaws (~~), 2 warnings (==), 39 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group M. Tuexen, Ed. 3 Internet-Draft Muenster Univ. of Appl. Sciences 4 Intended status: Informational F. Risso 5 Expires: September 28, 2020 Politecnico di Torino 6 J. Bongertz 7 Airbus DS CyberSecurity 8 G. Combs 9 Wireshark 10 G. Harris 12 M. Richardson 13 Sandelman 14 March 27, 2020 16 PCAP Next Generation (pcapng) Capture File Format 17 draft-tuexen-opsawg-pcapng-01 19 Abstract 21 This document describes a format to record captured packets to a 22 file. This format is extensible; Wireshark can currently read and 23 write it, and libpcap can currently read some pcapng files. 25 Status of This Memo 27 This Internet-Draft is submitted in full conformance with the 28 provisions of BCP 78 and BCP 79. 30 Internet-Drafts are working documents of the Internet Engineering 31 Task Force (IETF). Note that other groups may also distribute 32 working documents as Internet-Drafts. The list of current Internet- 33 Drafts is at https://datatracker.ietf.org/drafts/current/. 35 Internet-Drafts are draft documents valid for a maximum of six months 36 and may be updated, replaced, or obsoleted by other documents at any 37 time. It is inappropriate to use Internet-Drafts as reference 38 material or to cite them other than as "work in progress." 40 This Internet-Draft will expire on September 28, 2020. 42 Copyright Notice 44 Copyright (c) 2020 IETF Trust and the persons identified as the 45 document authors. All rights reserved. 47 This document is subject to BCP 78 and the IETF Trust's Legal 48 Provisions Relating to IETF Documents 49 (https://trustee.ietf.org/license-info) in effect on the date of 50 publication of this document. Please review these documents 51 carefully, as they describe your rights and restrictions with respect 52 to this document. Code Components extracted from this document must 53 include Simplified BSD License text as described in Section 4.e of 54 the Trust Legal Provisions and are provided without warranty as 55 described in the Simplified BSD License. 57 Table of Contents 59 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 60 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 61 2.1. Acronyms . . . . . . . . . . . . . . . . . . . . . . . . 4 62 3. General File Structure . . . . . . . . . . . . . . . . . . . 4 63 3.1. General Block Structure . . . . . . . . . . . . . . . . . 4 64 3.2. Block Types . . . . . . . . . . . . . . . . . . . . . . . 5 65 3.3. Logical Block Hierarchy . . . . . . . . . . . . . . . . . 7 66 3.4. Physical File Layout . . . . . . . . . . . . . . . . . . 7 67 3.5. Options . . . . . . . . . . . . . . . . . . . . . . . . . 8 68 3.5.1. Custom Options . . . . . . . . . . . . . . . . . . . 11 69 3.6. Data format . . . . . . . . . . . . . . . . . . . . . . . 13 70 3.6.1. Endianness . . . . . . . . . . . . . . . . . . . . . 13 71 3.6.2. Alignment . . . . . . . . . . . . . . . . . . . . . . 13 72 4. Block Definition . . . . . . . . . . . . . . . . . . . . . . 14 73 4.1. Section Header Block . . . . . . . . . . . . . . . . . . 14 74 4.2. Interface Description Block . . . . . . . . . . . . . . . 17 75 4.3. Enhanced Packet Block . . . . . . . . . . . . . . . . . . 22 76 4.3.1. Enhanced Packet Block Flags Word . . . . . . . . . . 26 77 4.4. Simple Packet Block . . . . . . . . . . . . . . . . . . . 27 78 4.5. Name Resolution Block . . . . . . . . . . . . . . . . . . 28 79 4.6. Interface Statistics Block . . . . . . . . . . . . . . . 32 80 4.7. systemd Journal Export Block . . . . . . . . . . . . . . 35 81 4.8. Decryption Secrets Block . . . . . . . . . . . . . . . . 36 82 4.9. Custom Block . . . . . . . . . . . . . . . . . . . . . . 38 83 5. Experimental Blocks (deserve further investigation) . . . . . 40 84 5.1. Alternative Packet Blocks (experimental) . . . . . . . . 40 85 5.2. Compression Block (experimental) . . . . . . . . . . . . 40 86 5.3. Encryption Block (experimental) . . . . . . . . . . . . . 41 87 5.4. Fixed Length Block (experimental) . . . . . . . . . . . . 42 88 5.5. Directory Block (experimental) . . . . . . . . . . . . . 43 89 5.6. Traffic Statistics and Monitoring Blocks (experimental) . 43 90 5.7. Event/Security Block (experimental) . . . . . . . . . . . 43 91 6. Vendor-Specific Custom Extensions . . . . . . . . . . . . . . 43 92 6.1. Supported Use-Cases . . . . . . . . . . . . . . . . . . . 44 93 6.2. Controlling Copy Behavior . . . . . . . . . . . . . . . . 44 94 6.3. Strings vs. Octets . . . . . . . . . . . . . . . . . . . 45 95 6.4. Endianness Issues . . . . . . . . . . . . . . . . . . . . 45 96 7. Recommended File Name Extension: .pcapng . . . . . . . . . . 45 97 8. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . 46 98 9. Implementations . . . . . . . . . . . . . . . . . . . . . . . 46 99 10. Security Considerations . . . . . . . . . . . . . . . . . . . 46 100 11. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 46 101 11.1. Standardized Block Type Codes . . . . . . . . . . . . . 46 102 12. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 49 103 13. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 49 104 14. References . . . . . . . . . . . . . . . . . . . . . . . . . 49 105 14.1. Normative References . . . . . . . . . . . . . . . . . . 49 106 14.2. URIs . . . . . . . . . . . . . . . . . . . . . . . . . . 49 107 Appendix A. Packet Block (obsolete!) . . . . . . . . . . . . . . 51 108 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 54 110 1. Introduction 112 The problem of exchanging packet traces becomes more and more 113 critical every day; unfortunately, no standard solutions exist for 114 this task right now. One of the most accepted packet interchange 115 formats is the one defined by libpcap, which is rather old and is 116 lacking in functionality for more modern applications particularly 117 from the extensibility point of view. 119 This document proposes a new format for recording packet traces. The 120 following goals are being pursued: 122 Extensibility: It should be possible to add new standard 123 capabilities to the file format over time, and third parties 124 should be able to enrich the information embedded in the file with 125 proprietary extensions, with tools unaware of newer extensions 126 being able to ignore them. 128 Portability: A capture trace must contain all the information needed 129 to read data independently from network, hardware and operating 130 system of the machine that made the capture. 132 Merge/Append data: It should be possible to add data at the end of a 133 given file, and the resulting file must still be readable. 135 2. Terminology 137 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 138 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 139 document are to be interpreted as described in [RFC2119]. 141 2.1. Acronyms 143 The following acronyms are used throughout this document: 145 SHB: Section Header Block 147 IDB: Interface Description Block 149 ISB: Interface Statistics Block 151 EPB: Enhanced Packet Block 153 SPB: Simple Packet Block 155 NRB: Name Resolution Block 157 CB: Custom Block 159 3. General File Structure 161 3.1. General Block Structure 163 A capture file is organized in blocks, that are appended one to 164 another to form the file. All the blocks share a common format, 165 which is shown in Figure 1. 167 0 1 2 3 168 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 169 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 170 | Block Type | 171 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 172 | Block Total Length | 173 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 174 / Block Body / 175 / variable length, padded to 32 bits / 176 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 177 | Block Total Length | 178 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 180 Figure 1: Basic block structure. 182 The fields have the following meaning: 184 o Block Type (32 bits): a unique unsigned value that identifies the 185 block. Values whose Most Significant Bit (MSB) is equal to 1 are 186 reserved for local use. They can be used to make extensions to 187 the file format to save private data to the file. The list of 188 currently defined types can be found in Section 11.1. 190 o Block Total Length (32 bits): an unsigned value giving the total 191 size of this block, in octets. For instance, the length of a 192 block that does not have a body is 12 octets: 4 octets for the 193 Block Type, 4 octets for the initial Block Total Length and 4 194 octets for the trailing Block Total Length. This value MUST be a 195 multiple of 4. 197 o Block Body: content of the block. 199 o Block Total Length: total size of this block, in octets. This 200 field is duplicated to permit backward file navigation. 202 This structure, shared among all blocks, makes it easy to process a 203 file and to skip unneeded or unknown blocks. Some blocks can contain 204 other blocks inside (nested blocks). Some of the blocks are 205 mandatory, i.e. a capture file is not valid if they are not present, 206 other are optional. 208 The General Block Structure allows defining other blocks if needed. 209 A parser that does not understand them can simply ignore their 210 content. 212 3.2. Block Types 214 The currently standardized Block Type codes are specified in 215 Section 11.1; they have been grouped in the following four 216 categories: 218 The following MANDATORY block MUST appear at least once in each file: 220 o Section Header Block (Section 4.1): it defines the most important 221 characteristics of the capture file. 223 The following OPTIONAL blocks MAY appear in a file: 225 o Interface Description Block (Section 4.2): it defines the most 226 important characteristics of the interface(s) used for capturing 227 traffic. This block is required in certain cases, as described 228 later. 230 o Enhanced Packet Block (Section 4.3): it contains a single captured 231 packet, or a portion of it. It represents an evolution of the 232 original, now obsolete, Packet Block (Appendix A). If this 233 appears in a file, an Interface Description Block is also 234 required, before this block. 236 o Simple Packet Block (Section 4.4): it contains a single captured 237 packet, or a portion of it, with only a minimal set of information 238 about it. If this appears in a file, an Interface Description 239 Block is also required, before this block. 241 o Name Resolution Block (Section 4.5): it defines the mapping from 242 numeric addresses present in the packet capture and the canonical 243 name counterpart. 245 o Interface Statistics Block (Section 4.6): it defines how to store 246 some statistical data (e.g. packet dropped, etc) which can be 247 useful to understand the conditions in which the capture has been 248 made. If this appears in a file, an Interface Description Block 249 is also required, before this block. 251 o Custom Block (Section 4.9): it contains vendor-specific data in a 252 portable fashion. 254 The following OBSOLETE block SHOULD NOT appear in newly written files 255 (but is documented in the Appendix for reference): 257 o Packet Block (Appendix A): it contains a single captured packet, 258 or a portion of it. It is OBSOLETE, and superseded by the 259 Enhanced Packet Block (Section 4.3). 261 The following EXPERIMENTAL blocks are considered interesting but the 262 authors believe that they deserve more in-depth discussion before 263 being defined: 265 o Alternative Packet Blocks 267 o Compression Block 269 o Encryption Block 271 o Fixed Length Block 273 o Directory Block 275 o Traffic Statistics and Monitoring Blocks 277 o Event/Security Blocks 279 Requests for new standardized Block Type codes should be sent to the 280 pcap-ng-format mailing list [1]. 282 3.3. Logical Block Hierarchy 284 The blocks build a logical hierarchy as they refer to each other. 285 Figure 2 shows the logical hierarchy of the currently defined blocks 286 in the form of a "tree view": 288 Section Header 289 | 290 +- Interface Description 291 | +- Simple Packet 292 | +- Enhanced Packet 293 | +- Interface Statistics 294 | 295 +- Name Resolution 297 Figure 2: Logical Block Hierarchy of a pcapng File 299 For example: each captured packet refers to a specific capture 300 interface, the interface itself refers to a specific section. 302 3.4. Physical File Layout 304 The file MUST begin with a Section Header Block. However, more than 305 one Section Header Block can be present in the capture file, each one 306 covering the data following it until the next one (or the end of 307 file). A Section includes the data delimited by two Section Header 308 Blocks (or by a Section Header Block and the end of the file), 309 including the first Section Header Block. 311 In case an application cannot read a Section because of different 312 version number, it MUST skip everything until the next Section Header 313 Block. Note that, in order to properly skip the blocks until the 314 next section, all blocks MUST have the fields Type and Length at the 315 beginning. In order to properly skip blocks in the backward 316 direction, all blocks MUST have the Length repeated at the end of the 317 block. These are mandatory requirements that MUST be maintained in 318 future versions of the block format. 320 Figure 3 shows a typical file layout, with a single Section Header 321 that covers the whole file. 323 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 324 | SHB v1.0 | Data | 325 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 327 Figure 3: File structure example: Typical layout with a single 328 Section Header Block 330 Figure 4 shows a file that contains three headers, and is normally 331 the result of file concatenation. An application that understands 332 only version 1.0 of the file format skips the intermediate section 333 and restart processing the packets after the third Section Header. 335 |-- 1st Section --|-- 2nd Section --|-- 3rd Section --| 336 | | 337 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 338 | SHB v1.0 | Data | SHB V1.1 | Data | SHB V1.0 | Data | 339 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 341 Figure 4: File structure example: three Section Header Blocks in a 342 single file 344 Figure 5 shows a file comparable to a "classic libpcap" file - the 345 minimum for a useful capture file. It contains a single 346 Section Header Block (SHB), a single Interface Description Block 347 (IDB) and a few Enhanced Packet Blocks (EPB). 349 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 350 | SHB | IDB | EPB | EPB | ... | EPB | 351 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 353 Figure 5: File structure example: a pcapng file similar to a 354 classical libpcap file 356 Figure 6 shows a complex example file. In addition to the minimum 357 file above, it contains packets captured from three interfaces, 358 capturing on the third of which begins after packets have arrived on 359 other interfaces, and also includes some Name Resolution Blocks (NRB) 360 and an Interface Statistics Block (ISB). 362 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 363 | SHB | IDB | IDB | EPB | NRB |...| IDB | EPB | ISB | NRB | EPB | 364 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 366 Figure 6: File structure example: complex pcapng file 368 The last example should make it obvious that the block structure 369 makes the file format very flexible compared to the classical libpcap 370 format. 372 3.5. Options 374 All the block bodies MAY embed optional fields. Optional fields can 375 be used to insert some information that may be useful when reading 376 data, but that is not really needed for packet processing. 378 Therefore, each tool can either read the content of the optional 379 fields (if any), or skip some of them or even all at once. 381 A block that may contain options must be structured so that the 382 number of octets of data in the Block Body that precede the options 383 can be determined from that data; that allows the beginning of the 384 options to be found. That is true for all standard blocks that 385 support options; for Custom Blocks that support options, the Custom 386 Data must be structured in such a fashion. This means that the Block 387 Length field (present in the General Block Structure, see 388 Section 3.1) can be used to determine how many octets of optional 389 fields, if any, are present in the block. That number can be used to 390 determine whether the block has optional fields (if it is zero, there 391 are no optional fields), to check, when processing optional fields, 392 whether any optional fields remain, and to skip all the optional 393 fields at once. 395 Options are a list of Type - Length - Value fields, each one 396 containing a single value: 398 o Option Type (16 bits): an unsigned value that contains the code 399 that specifies the type of the current TLV record. Option types 400 whose Most Significant Bit is equal to one are reserved for local 401 use; therefore, there is no guarantee that the code used is unique 402 among all capture files (generated by other applications), and is 403 most certainly not portable. For cross-platform globally unique 404 vendor-specific extensions, the Custom Option MUST be used 405 instead, as defined in Section 3.5.1). 407 o Option Length (16 bits): an unsigned value that contains the 408 actual length of the following 'Option Value' field without the 409 padding octets. 411 o Option Value (variable length): the value of the given option, 412 padded to a 32-bit boundary. The actual length of this field 413 (i.e. without the padding octets) is specified by the Option 414 Length field. 416 Requests for new standardized option codes for a given block should 417 be sent to the pcap-ng-format mailing list [2]. 419 A given option may have a fixed length, in which case all instances 420 of that option have a length that is equal to the specified fixed 421 length, or a variable length, in which case the option has a minimum 422 length and all instances of that option must have a length equal to 423 or greater than the specified minimum length. The length of fixed- 424 length options, and the minimum length of variable-length options, is 425 specified in the description of the option; if the minimum length of 426 a variable-length option is not specified, a zero-length option is 427 valid. Software that reads these files SHOULD report options that 428 have an invalid length as errors; the software MAY stop processing 429 the file if it sees an option that has invalid length, or MAY ignore 430 the option and continue processing it. Software that writes these 431 files MUST not write files with options that have invalid lengths. 433 If an option's value is a string, the value is not necessarily zero- 434 terminated. Software that reads these files MUST NOT assume that 435 strings are zero-terminated, and MUST treat a zero-value octet as a 436 string terminator. 438 Some options may be repeated several times; for example, a block can 439 have multiple comments, and an Interface Description Block can give 440 multiple IPv4 or IPv6 addresses for the interface if it has multiple 441 IPv4 or IPv6 addresses assigned to it. Other options may appear at 442 most once in a given block. 444 The option list is terminated by a option which uses the special 'End 445 of Option' code (opt_endofopt). Code that writes pcapng files MUST 446 put an opt_endofopt option at the end of an option list. Code that 447 reads pcapng files MUST NOT assume an option list will have an 448 opt_endofopt option at the end; it MUST also check for the end of the 449 block, and SHOULD treat blocks where the option list has no 450 opt_endofopt option as if the option list had an opt_endofopt option 451 at the end. 453 The format of the optional fields is shown in Figure 7. 455 0 1 2 3 456 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 457 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 458 | Option Code | Option Length | 459 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 460 / Option Value / 461 / variable length, padded to 32 bits / 462 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 463 / / 464 / . . . other options . . . / 465 / / 466 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 467 | Option Code == opt_endofopt | Option Length == 0 | 468 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 470 Figure 7: Options Format 472 The following codes can always be present in any optional field: 474 +--------------+-----------------------+--------------+-------------+ 475 | Name | Code | Length | Multiple | 476 | | | | allowed? | 477 +--------------+-----------------------+--------------+-------------+ 478 | opt_endofopt | 0 | 0 | no | 479 | opt_comment | 1 | variable | yes | 480 | opt_custom | 2988/2989/19372/19373 | variable, | yes | 481 | | | minimum 4 | | 482 +--------------+-----------------------+--------------+-------------+ 484 Table 1: Common Options 486 opt_endofopt: 487 The opt_endofopt option delimits the end of the optional 488 fields. This option MUST NOT be repeated within a given list 489 of options. 491 opt_comment: 492 The opt_comment option is a UTF-8 string containing human- 493 readable comment text that is associated to the current 494 block. Line separators SHOULD be a carriage-return + 495 linefeed ('\r\n') or just linefeed ('\n'); either form may 496 appear and be considered a line separator. The string is not 497 zero-terminated. 499 Examples: "This packet is the beginning of all of our 500 problems", "Packets 17-23 showing a bogus TCP 501 retransmission!\r\n This is reported in bugzilla entry 502 1486.\nIt will be fixed in the future.". 504 opt_custom: 505 This option is described in detail in Section 3.5.1. 507 3.5.1. Custom Options 509 Customs Options are used for portable, vendor-specific data related 510 to the block they're in. A Custom Option can be in any block type 511 that can have options, can be repeated any number of times in a 512 block, and may come before or after other option types - except the 513 opt_endofopt which is always the last option. Different Custom 514 Options, of different type codes and/or different Private Enterprise 515 Numbers, may be used in the same pcapng file. See Section 6 for 516 additional details. 518 0 1 2 3 519 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 520 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 521 | Custom Option Code | Option Length | 522 +---------------------------------------------------------------+ 523 | Private Enterprise Number (PEN) | 524 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 525 / Custom Data / 526 / variable length, padded to 32 bits / 527 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 529 Figure 8: Custom Options Format 531 The Custom Option has the following fields: 533 o Custom Option Code: The code number for the Custom Option, which 534 can be one of the following decimal numbers: 536 2988: 537 This option code identifies a Custom Option containing a 538 UTF-8 string in the Custom Data portion. The string is 539 not zero-terminated. This Custom Option can be safely 540 copied to a new file if the pcapng file is manipulated by 541 an application; otherwise 19372 should be used instead. 542 See Section 6.2 for details. 544 2989: 545 This option code identifies a Custom Option containing 546 binary octets in the Custom Data portion. This Custom 547 Option can be safely copied to a new file if the pcapng 548 file is manipulated by an application; otherwise 19372 549 should be used instead. See Section 6.2 for details. 551 19372: 552 This option code identifies a Custom Option containing a 553 UTF-8 string in the Custom Data portion. The string is 554 not zero-terminated. This Custom Option should not be 555 copied to a new file if the pcapng file is manipulated by 556 an application. See Section 6.2 for details. 558 19373: 559 This option code identifies a Custom Option containing 560 binary octets in the Custom Data portion. This Custom 561 Option should not be copied to a new file if the pcapng 562 file is manipulated by an application. See Section 6.2 563 for details. 565 o Option Length: as described in Section 3.1, this contains the 566 length of the option's value, which includes the 4-octet Private 567 Enterprise Number and variable-length Custom Data fields, without 568 the padding octets. 570 o Private Enterprise Number: An IANA-assigned Private Enterprise 571 Number identifying the organization which defined the Custom 572 Option. See Section 6.1 for details. The PEN number MUST be 573 encoded using the same endianness as the Section Header Block it 574 is within the scope of. 576 o Custom Data: the custom data, padded to a 32 bit boundary. 578 3.6. Data format 580 3.6.1. Endianness 582 Data contained in each section will always be saved according to the 583 characteristics (little endian / big endian) of the capturing 584 machine. This refers to all the fields that are saved as numbers and 585 that span over two or more octets. 587 The approach of having each section saved in the native format of the 588 generating host is more efficient because it avoids translation of 589 data when reading / writing on the host itself, which is the most 590 common case when generating/processing capture captures. 592 Please note: The endianness is indicated by the Section Header Block 593 (Section 4.1). Since this block can appear several times in a pcapng 594 file, a single file can contain both endianness variants. 596 3.6.2. Alignment 598 All fields of this specification use proper alignment for 16- and 599 32-bit values. This makes it easier and faster to read/write file 600 contents if using techniques like memory mapped files. 602 The alignment octets (marked in this document e.g. with "padded to 32 603 bits") MUST be filled with zeroes. 605 Please note: 64-bit values are not aligned to 64-bit boundaries. 606 This is because the file is naturally aligned to 32-bit boundaries 607 only. Special care MUST be taken when reading and writing such 608 values. (Note also that some 64-bit values are represented as a 609 64-bit integer in the endianness of the machine that wrote the file, 610 and others are represented as 2 32-bit values, one containing the 611 upper 32 bits of the value and one containing the lower 32 bits of 612 the value, each written as 32-bit integers in the endianness of the 613 machine that wrote the file. Neither of these formats guarantee 614 64-bit alignment.) 616 4. Block Definition 618 This section details the format of the blocks currently defined. 620 4.1. Section Header Block 622 The Section Header Block (SHB) is mandatory. It identifies the 623 beginning of a section of the capture capture file. The 624 Section Header Block does not contain data but it rather identifies a 625 list of blocks (interfaces, packets) that are logically correlated. 626 Its format is shown in Figure 9. 628 0 1 2 3 629 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 630 +---------------------------------------------------------------+ 631 0 | Block Type = 0x0A0D0D0A | 632 +---------------------------------------------------------------+ 633 4 | Block Total Length | 634 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 635 8 | Byte-Order Magic | 636 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 637 12 | Major Version | Minor Version | 638 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 639 16 | | 640 | Section Length | 641 | | 642 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 643 24 / / 644 / Options (variable) / 645 / / 646 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 647 | Block Total Length | 648 +---------------------------------------------------------------+ 650 Figure 9: Section Header Block Format 652 The meaning of the fields is: 654 o Block Type: The block type of the Section Header Block is the 655 integer corresponding to the 4-char string "\n\r\r\n" 656 (0x0A0D0D0A). This particular value is used for 2 reasons: 658 1. This number is used to detect if a file has been transferred 659 via FTP or HTTP from a machine to another with an 660 inappropriate ASCII conversion. In this case, the value of 661 this field will differ from the standard one ("\n\r\r\n") and 662 the reader can detect a possibly corrupted file. 664 2. This value is palindromic, so that the reader is able to 665 recognize the Section Header Block regardless of the 666 endianness of the section. The endianness is recognized by 667 reading the Byte Order Magic, that is located 8 octets after 668 the Block Type. 670 o Block Total Length: total size of this block, as described in 671 Section 3.1. 673 o Byte-Order Magic (32 bits): an unsigned magic number, whose value 674 is the hexadecimal number 0x1A2B3C4D. This number can be used to 675 distinguish sections that have been saved on little-endian 676 machines from the ones saved on big-endian machines, and to 677 heuristically identify pcapng files. 679 o Major Version (16 bits): an unsigned value, giving the number of 680 the current major version of the format. The value for the 681 current version of the format is 1. This value should change if 682 the format changes in such a way that code that reads the new 683 format could not read the old format (i.e., code to read both 684 formats would have to check the version number and use different 685 code paths for the two formats) and code that reads the old format 686 could not read the new format. Note that adding a new block type 687 or a new option is NOT such a change. 689 o Minor Version (16 bits): an unsigned value, giving the number of 690 the current minor version of the format. The value is for the 691 current version of the format is 0. This value should change if 692 the format changes in such a way that code that reads the new 693 format could read the old format without checking the version 694 number but code that reads the old format could not read all files 695 in the new format. Note that adding a new block type or a new 696 option is NOT such a change. 698 o Section Length (64 bits): a signed value specifying the length in 699 octets of the following section, excluding the Section Header 700 Block itself. This field can be used to skip the section, for 701 faster navigation inside large files. If the Section Length is -1 702 (0xFFFFFFFFFFFFFFFF), this means that the size of the section is 703 not specified, and the only way to skip the section is to parse 704 the blocks that it contains. Please note that if this field is 705 valid (i.e. not negative), its value is always a multiple of 4, 706 as all the blocks are aligned to and padded to 32-bit (4 octet) 707 boundaries. Also, special care should be taken in accessing this 708 field: since the alignment of all the blocks in the file is 709 32-bits, this field is not guaranteed to be aligned to a 64-bit 710 boundary. This could be a problem on 64-bit processors. 712 o Options: optionally, a list of options (formatted according to the 713 rules defined in Section 3.5) can be present. 715 Adding new block types or options would not necessarily require that 716 either Major or Minor numbers be changed, as code that does not know 717 about the block type or option should just skip it; only if skipping 718 a block or option does not work should the minor version number be 719 changed. 721 Aside from the options defined in Section 3.5, the following options 722 are valid within this block: 724 +--------------+------+----------+-------------------+ 725 | Name | Code | Length | Multiple allowed? | 726 +--------------+------+----------+-------------------+ 727 | shb_hardware | 2 | variable | no | 728 | shb_os | 3 | variable | no | 729 | shb_userappl | 4 | variable | no | 730 +--------------+------+----------+-------------------+ 732 Table 2: Section Header Block Options 734 shb_hardware: 735 The shb_hardware option is a UTF-8 string containing the 736 description of the hardware used to create this section. The 737 string is not zero-terminated. 739 Examples: "x86 Personal Computer", "Sun Sparc Workstation". 741 shb_os: 742 The shb_os option is a UTF-8 string containing the name of 743 the operating system used to create this section. The string 744 is not zero-terminated. 746 Examples: "Windows XP SP2", "openSUSE 10.2". 748 shb_userappl: 749 The shb_userappl option is a UTF-8 string containing the name 750 of the application used to create this section. The string 751 is not zero-terminated. 753 Examples: "dumpcap V0.99.7". 755 [Open issue: does a program which re-writes a capture file change the 756 original hardware/os/application info?] 758 4.2. Interface Description Block 760 An Interface Description Block (IDB) is the container for information 761 describing an interface on which packet data is captured. 763 Tools that write / read the capture file associate an incrementing 764 unsigned 32-bit number (starting from '0') to each Interface 765 Definition Block, called the Interface ID for the interface in 766 question. This number is unique within each Section and identifies 767 the interface to which the IDB refers; it is only unique inside the 768 current section, so, two Sections can have different interfaces 769 identified by the same Interface ID values. This unique identifier 770 is referenced by other blocks, such as Enhanced Packet Blocks and 771 Interface Statistic Blocks, to indicate the interface to which the 772 block refers (such the interface that was used to capture the packet 773 that an Enhanced Packet Block contains or to which the statistics in 774 an Interface Statistic Block refer). 776 There must be an Interface Description Block for each interface to 777 which another block refers. Blocks such as an Enhanced Packet Block 778 or an Interface Statistics Block contain an Interface ID value 779 referring to a particular interface, and a Simple Packet Block 780 implicitly refers to an interface with an Interface ID of 0. If the 781 file does not contain any blocks that use an Interface ID, then the 782 file does not need to have any IDBs. 784 An Interface Description Block is valid only inside the section to 785 which it belongs. The structure of a Interface Description Block is 786 shown in Figure 10. 788 0 1 2 3 789 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 790 +---------------------------------------------------------------+ 791 0 | Block Type = 0x00000001 | 792 +---------------------------------------------------------------+ 793 4 | Block Total Length | 794 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 795 8 | LinkType | Reserved | 796 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 797 12 | SnapLen | 798 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 799 16 / / 800 / Options (variable) / 801 / / 802 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 803 | Block Total Length | 804 +---------------------------------------------------------------+ 806 Figure 10: Interface Description Block Format 808 The meaning of the fields is: 810 o Block Type: The block type of the Interface Description Block is 811 1. 813 o Block Total Length: total size of this block, as described in 814 Section 3.1. 816 o LinkType (16 bits): an unsigned value that defines the link layer 817 type of this interface. The list of Standardized Link Layer Type 818 codes is available in the tcpdump.org link-layer header types 819 registry [3]. 821 o Reserved (16 bits): not used - MUST be filled with 0 by pcap file 822 writers, and MUST be ignored by pcapng file readers. 824 o SnapLen (32 bits): an unsigned value indicating the maximum number 825 of octets captured from each packet. The portion of each packet 826 that exceeds this value will not be stored in the file. A value 827 of zero indicates no limit. 829 o Options: optionally, a list of options (formatted according to the 830 rules defined in Section 3.5) can be present. 832 In addition to the options defined in Section 3.5, the following 833 options are valid within this block: 835 +----------------+------+---------------------+-------------------+ 836 | Name | Code | Length | Multiple allowed? | 837 +----------------+------+---------------------+-------------------+ 838 | if_name | 2 | variable | no | 839 | if_description | 3 | variable | no | 840 | if_IPv4addr | 4 | 8 | yes | 841 | if_IPv6addr | 5 | 17 | yes | 842 | if_MACaddr | 6 | 6 | no | 843 | if_EUIaddr | 7 | 8 | no | 844 | if_speed | 8 | 8 | no | 845 | if_tsresol | 9 | 1 | no | 846 | if_tzone | 10 | 4 | no | 847 | if_filter | 11 | variable, minimum 1 | no | 848 | if_os | 12 | variable | no | 849 | if_fcslen | 13 | 1 | no | 850 | if_tsoffset | 14 | 8 | no | 851 | if_hardware | 15 | variable | no | 852 +----------------+------+---------------------+-------------------+ 854 Table 3: Interface Description Block Options 856 if_name: 857 The if_name option is a UTF-8 string containing the name of 858 the device used to capture data. The string is not zero- 859 terminated. 861 Examples: "eth0", 862 "\Device\NPF_{AD1CE675-96D0-47C5-ADD0-2504B9126B68}". 864 if_description: 865 The if_description option is a UTF-8 string containing the 866 description of the device used to capture data. The string 867 is not zero-terminated. 869 Examples: "Wi-Fi", "Local Area Connection", "Wireless Network 870 Connection", "First Ethernet Interface". 872 if_IPv4addr: 873 The if_IPv4addr option is an IPv4 network address and 874 corresponding netmask for the interface. The first four 875 octets are the IP address, and the next four octets are the 876 netmask. This option can be repeated multiple times within 877 the same Interface Description Block when multiple IPv4 878 addresses are assigned to the interface. Note that the IP 879 address and netmask are both treated as four octets, one for 880 each octet of the address or mask; they are not 32-bit 881 numbers, and thus the endianness of the SHB does not affect 882 this field's value. 884 Examples: '192 168 1 1 255 255 255 0'. 886 if_IPv6addr: 887 The if_IPv6addr option is an IPv6 network address and 888 corresponding prefix length for the interface. The first 16 889 octets are the IP address and the next octet is the prefix 890 length. This option can be repeated multiple times within 891 the same Interface Description Block when multiple IPv6 892 addresses are assigned to the interface. 894 Example: 2001:0db8:85a3:08d3:1319:8a2e:0370:7344/64 is 895 written (in hex) as '20 01 0d b8 85 a3 08 d3 13 19 8a 2e 03 896 70 73 44 40'. 898 if_MACaddr: 899 The if_MACaddr option is the Interface Hardware MAC address 900 (48 bits), if available. 902 Example: '00 01 02 03 04 05'. 904 if_EUIaddr: 905 The if_EUIaddr option is the Interface Hardware EUI address 906 (64 bits), if available. 908 Example: '02 34 56 FF FE 78 9A BC'. 910 if_speed: 911 The if_speed option is a 64-bit unsigned value indicating the 912 interface speed, in bits per second. 914 Example: the 64-bit decimal number 100000000 for 100Mbps. 916 if_tsresol: 917 The if_tsresol option identifies the resolution of 918 timestamps. If the Most Significant Bit is equal to zero, 919 the remaining bits indicates the resolution of the timestamp 920 as a negative power of 10 (e.g. 6 means microsecond 921 resolution, timestamps are the number of microseconds since 922 1970-01-01 00:00:00 UTC). If the Most Significant Bit is 923 equal to one, the remaining bits indicates the resolution as 924 as negative power of 2 (e.g. 10 means 1/1024 of second). If 925 this option is not present, a resolution of 10^-6 is assumed 926 (i.e. timestamps have the same resolution of the standard 927 'libpcap' timestamps). 929 Example: '6'. 931 if_tzone: 933 The if_tzone option identifies the time zone for GMT support 934 (TODO: specify better). 936 Example: TODO: give a good example. 938 if_filter: 939 The if_filter option identifies the filter (e.g. "capture 940 only TCP traffic") used to capture traffic. The first octet 941 of the Option Data keeps a code of the filter used (e.g. if 942 this is a libpcap string, or BPF bytecode, and more). More 943 details about this format will be presented in Appendix XXX 944 (TODO). (TODO: better use different options for different 945 fields? e.g. if_filter_pcap, if_filter_bpf, ...) 947 Example: '00'"tcp port 23 and host 192.0.2.5". 949 if_os: 950 The if_os option is a UTF-8 string containing the name of the 951 operating system of the machine in which this interface is 952 installed. This can be different from the same information 953 that can be contained by the Section Header Block 954 (Section 4.1) because the capture can have been done on a 955 remote machine. The string is not zero-terminated. 957 Examples: "Windows XP SP2", "openSUSE 10.2". 959 if_fcslen: 960 The if_fcslen option is an 8-bit unsigned integer value that 961 specifies the length of the Frame Check Sequence (in bits) 962 for this interface. For link layers whose FCS length can 963 change during time, the Enhanced Packet Block epb_flags 964 Option can be used in each Enhanced Packet Block (see 965 Section 4.3.1). 967 Example: '4'. 969 if_tsoffset: 970 The if_tsoffset option is a 64-bit signed integer value that 971 specifies an offset (in seconds) that must be added to the 972 timestamp of each packet to obtain the absolute timestamp of 973 a packet. If the option is missing, the timestamps stored in 974 the packet MUST be considered absolute timestamps. The time 975 zone of the offset can be specified with the option if_tzone. 976 TODO: won't a if_tsoffset_low for fractional second offsets 977 be useful for highly synchronized capture systems? 979 Example: '1234'. 981 if_hardware: 982 The if_hardware option is a UTF-8 string containing the 983 description of the interface hardware. The string is not 984 zero-terminated. 986 Examples: "Broadcom NetXtreme", "Intel(R) PRO/1000 MT Network 987 Connection", "NETGEAR WNA1000Mv2 N150 Wireless USB Micro 988 Adapter". 990 4.3. Enhanced Packet Block 992 An Enhanced Packet Block (EPB) is the standard container for storing 993 the packets coming from the network. The Enhanced Packet Block is 994 optional because packets can be stored either by means of this block 995 or the Simple Packet Block, which can be used to speed up capture 996 file generation; or a file may have no packets in it. The format of 997 an Enhanced Packet Block is shown in Figure 11. 999 The Enhanced Packet Block is an improvement over the original, now 1000 obsolete, Packet Block (Appendix A): 1002 o it stores the Interface Identifier as a 32-bit integer value. 1003 This is a requirement when a capture stores packets coming from a 1004 large number of interfaces; 1006 o unlike the Packet Block (Appendix A), the number of packets 1007 dropped by the capture system between this packet and the previous 1008 one is not stored in the header, but rather in an option of the 1009 block itself. 1011 0 1 2 3 1012 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1013 +---------------------------------------------------------------+ 1014 0 | Block Type = 0x00000006 | 1015 +---------------------------------------------------------------+ 1016 4 | Block Total Length | 1017 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1018 8 | Interface ID | 1019 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1020 12 | Timestamp (High) | 1021 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1022 16 | Timestamp (Low) | 1023 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1024 20 | Captured Packet Length | 1025 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1026 24 | Original Packet Length | 1027 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1028 28 / / 1029 / Packet Data / 1030 / variable length, padded to 32 bits / 1031 / / 1032 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1033 / / 1034 / Options (variable) / 1035 / / 1036 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1037 | Block Total Length | 1038 +---------------------------------------------------------------+ 1040 Figure 11: Enhanced Packet Block Format 1042 The Enhanced Packet Block has the following fields: 1044 o Block Type: The block type of the Enhanced Packet Block is 6. 1046 o Block Total Length: total size of this block, as described in 1047 Section 3.1. 1049 o Interface ID (32 bits): an unsigned value that specifies the 1050 interface on which this packet was received or transmitted; the 1051 correct interface will be the one whose Interface Description 1052 Block (within the current Section of the file) is identified by 1053 the same number (see Section 4.2) of this field. The interface ID 1054 MUST be valid, which means that an matching interface description 1055 block MUST exist. 1057 o Timestamp (High) and Timestamp (Low): upper 32 bits and lower 32 1058 bits of a 64-bit timestamp. The timestamp is a single 64-bit 1059 unsigned integer that represents the number of units of time that 1060 have elapsed since 1970-01-01 00:00:00 UTC. The length of a unit 1061 of time is specified by the 'if_tsresol' option (see Figure 10) of 1062 the Interface Description Block referenced by this packet. Note 1063 that, unlike timestamps in the libpcap file format, timestamps in 1064 Enhanced Packet Blocks are not saved as two 32-bit values that 1065 represent the seconds and microseconds that have elapsed since 1066 1970-01-01 00:00:00 UTC. Timestamps in Enhanced Packet Blocks are 1067 saved as two 32-bit words that represent the upper and lower 32 1068 bits of a single 64-bit quantity. 1070 o Captured Packet Length (32 bits): an unsigned value that indicates 1071 the number of octets captured from the packet (i.e. the length of 1072 the Packet Data field). It will be the minimum value among the 1073 Original Packet Length and the snapshot length for the interface 1074 (SnapLen, defined in Figure 10). The value of this field does not 1075 include the padding octets added at the end of the Packet Data 1076 field to align the Packet Data field to a 32-bit boundary. 1078 o Original Packet Length (32 bits): an unsigned value that indicates 1079 the actual length of the packet when it was transmitted on the 1080 network. It can be different from the Captured Packet Length if 1081 the packet has been truncated by the capture process. 1083 o Packet Data: the data coming from the network, including link- 1084 layer headers. The actual length of this field is Captured Packet 1085 Length plus the padding to a 32-bit boundary. The format of the 1086 link-layer headers depends on the LinkType field specified in the 1087 Interface Description Block (see Section 4.2) and it is specified 1088 in the entry for that format in the the tcpdump.org link-layer 1089 header types registry [4]. 1091 o Options: optionally, a list of options (formatted according to the 1092 rules defined in Section 3.5) can be present. 1094 In addition to the options defined in Section 3.5, the following 1095 options are valid within this block: 1097 +---------------+------+----------------------------+---------------+ 1098 | Name | Code | Length | Multiple | 1099 | | | | allowed? | 1100 +---------------+------+----------------------------+---------------+ 1101 | epb_flags | 2 | 4 | no | 1102 | epb_hash | 3 | variable, minimum hash | yes | 1103 | | | type-dependent | | 1104 | epb_dropcount | 4 | 8 | no | 1105 +---------------+------+----------------------------+---------------+ 1107 Table 4: Enhanced Packet Block Options 1109 epb_flags: 1110 The epb_flags option is a 32-bit flags word containing link- 1111 layer information. A complete specification of the allowed 1112 flags can be found in Section 4.3.1. 1114 Example: '0'. 1116 epb_hash: 1117 The epb_hash option contains a hash of the packet. The first 1118 octet specifies the hashing algorithm, while the following 1119 octets contain the actual hash, whose size depends on the 1120 hashing algorithm, and hence from the value in the first 1121 octet. The hashing algorithm can be: 2s complement 1122 (algorithm octet = 0, size = XXX), XOR (algorithm octet = 1, 1123 size=XXX), CRC32 (algorithm octet = 2, size = 4), MD-5 1124 (algorithm octet = 3, size = 16), SHA-1 (algorithm octet = 4, 1125 size = 20), Toeplitz (algorithm octet = 5, size = 4). The 1126 hash covers only the packet, not the header added by the 1127 capture driver: this gives the possibility to calculate it 1128 inside the network card. The hash allows easier comparison/ 1129 merging of different capture files, and reliable data 1130 transfer between the data acquisition system and the capture 1131 library. 1133 Examples: '02 EC 1D 87 97', '03 45 6E C2 17 7C 10 1E 3C 2E 99 1134 6E C2 9A 3D 50 8E'. 1136 epb_dropcount: 1137 The epb_dropcount option is a 64-bit unsigned integer value 1138 specifying the number of packets lost (by the interface and 1139 the operating system) between this packet and the preceding 1140 one for the same interface or, for the first packet for an 1141 interface, between this packet and the start of the capture 1142 process. 1144 Example: '0'. 1146 4.3.1. Enhanced Packet Block Flags Word 1148 The Enhanced Packet Block Flags Word is a 32-bit value that contains 1149 link-layer information about the packet. 1151 The word is encoded as an unsigned 32-bit integer, using the 1152 endianness of the Section Header Block scope it is in. In the 1153 following table, the bits are numbered with 0 being the least- 1154 significant bit and 31 being the most-significant bit of the 32-bit 1155 unsigned integer. The meaning of the bits is the following: 1157 +--------+----------------------------------------------------------+ 1158 | Bit | Description | 1159 | Number | | 1160 +--------+----------------------------------------------------------+ 1161 | 0-1 | Inbound / Outbound packet (00 = information not | 1162 | | available, 01 = inbound, 10 = outbound) | 1163 | 2-4 | Reception type (000 = not specified, 001 = unicast, 010 | 1164 | | = multicast, 011 = broadcast, 100 = | 1165 | | promiscuous). | 1166 | 5-8 | FCS length, in octets (0000 if this information is not | 1167 | | available). This value overrides the | 1168 | | if_fcslen option of the Interface Description | 1169 | | Block, and is used with those link layers (e.g. PPP) | 1170 | | where the length of the FCS can change | 1171 | | during time. | 1172 | 9-15 | Reserved (MUST be set to zero). | 1173 | 16-31 | link-layer-dependent errors (Bit 31 = symbol error, Bit | 1174 | | 30 = preamble error, Bit 29 = Start Frame | 1175 | | Delimiter error, Bit 28 = unaligned frame | 1176 | | error, Bit 27 = wrong Inter Frame Gap error, Bit 26 = | 1177 | | packet too short error, Bit 25 = packet too long error, | 1178 | | Bit 24 = CRC error, other?? are 16 bit | 1179 | | enough?). | 1180 +--------+----------------------------------------------------------+ 1182 NOTE: in earlier versions of this specification, the bits were 1183 specified as being numbered with 0 being the most-significant bit and 1184 31 being the least-significant bit of the 32-bit unsigned integer, 1185 rather than with 0 being the least-significant bit and 31 being the 1186 most-significant bit. Several implementations number the bits with 0 1187 being the least-significant bit, and no known implementations number 1188 them with 0 being the most-significant bit, so the specification was 1189 changed to reflect that reality. 1191 4.4. Simple Packet Block 1193 The Simple Packet Block (SPB) is a lightweight container for storing 1194 the packets coming from the network. Its presence is optional. 1196 A Simple Packet Block is similar to an Enhanced Packet Block (see 1197 Section 4.3), but it is smaller, simpler to process and contains only 1198 a minimal set of information. This block is preferred to the 1199 standard Enhanced Packet Block when performance or space occupation 1200 are critical factors, such as in sustained traffic capture 1201 applications. A capture file can contain both Enhanced Packet Blocks 1202 and Simple Packet Blocks: for example, a capture tool could switch 1203 from Enhanced Packet Blocks to Simple Packet Blocks when the hardware 1204 resources become critical. 1206 The Simple Packet Block does not contain the Interface ID field. 1207 Therefore, it MUST be assumed that all the Simple Packet Blocks have 1208 been captured on the interface previously specified in the first 1209 Interface Description Block. 1211 Figure 12 shows the format of the Simple Packet Block. 1213 0 1 2 3 1214 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1215 +---------------------------------------------------------------+ 1216 0 | Block Type = 0x00000003 | 1217 +---------------------------------------------------------------+ 1218 4 | Block Total Length | 1219 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1220 8 | Original Packet Length | 1221 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1222 12 / / 1223 / Packet Data / 1224 / variable length, padded to 32 bits / 1225 / / 1226 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1227 | Block Total Length | 1228 +---------------------------------------------------------------+ 1230 Figure 12: Simple Packet Block Format 1232 The Simple Packet Block has the following fields: 1234 o Block Type: The block type of the Simple Packet Block is 3. 1236 o Block Total Length: total size of this block, as described in 1237 Section 3.1. 1239 o Original Packet Length (32 bits): an unsigned value indicating the 1240 actual length of the packet when it was transmitted on the 1241 network. It can be different from length of the Packet Data 1242 field's length if the packet has been truncated by the capture 1243 process, in which case the SnapLen value in Section 4.2 will be 1244 less than this Original Packet Length value, and the SnapLen value 1245 MUST be used to determine the size of the Packet Data field 1246 length. 1248 o Packet Data: the data coming from the network, including link- 1249 layer headers. The length of this field can be derived from the 1250 field Block Total Length, present in the Block Header, and it is 1251 the minimum value among the SnapLen (present in the Interface 1252 Description Block) and the Original Packet Length (present in this 1253 header). The format of the data within this Packet Data field 1254 depends on the LinkType field specified in the Interface 1255 Description Block (see Section 4.2) and it is specified in the 1256 entry for that format in the tcpdump.org link-layer header types 1257 registry [5]. 1259 The Simple Packet Block does not contain the timestamp because this 1260 is often one of the most costly operations on PCs. Additionally, 1261 there are applications that do not require it; e.g. an Intrusion 1262 Detection System is interested in packets, not in their timestamp. 1264 A Simple Packet Block cannot be present in a Section that has more 1265 than one interface because of the impossibility to refer to the 1266 correct one (it does not contain any Interface ID field). 1268 The Simple Packet Block is very efficient in term of disk space: a 1269 snapshot whose length is 100 octets requires only 16 octets of 1270 overhead, which corresponds to an efficiency of more than 86%. 1272 4.5. Name Resolution Block 1274 The Name Resolution Block (NRB) is used to support the correlation of 1275 numeric addresses (present in the captured packets) and their 1276 corresponding canonical names and it is optional. Having the literal 1277 names saved in the file prevents the need for performing name 1278 resolution at a later time, when the association between names and 1279 addresses may be different from the one in use at capture time. 1280 Moreover, the NRB avoids the need for issuing a lot of DNS requests 1281 every time the trace capture is opened, and also provides name 1282 resolution when reading the capture with a machine not connected to 1283 the network. 1285 A Name Resolution Block is often placed at the beginning of the file, 1286 but no assumptions can be taken about its position. Multiple NRBs 1287 can exist in a pcapng file, either due to memory constraints or 1288 because additional name resolutions were performed by file processing 1289 tools, like network analyzers. 1291 A Name Resolution Block need not contain any Records, except the 1292 nrb_record_end Record which MUST be the last Record. The addresses 1293 and names in NRB Records MAY be repeated multiple times; i.e., the 1294 same IP address may resolve to multiple names, the same name may 1295 resolve to the multiple IP addresses, and even the same address-to- 1296 name pair may appear multiple times, in the same NRB or across NRBs. 1298 The format of the Name Resolution Block is shown in Figure 13. 1300 0 1 2 3 1301 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1302 +---------------------------------------------------------------+ 1303 0 | Block Type = 0x00000004 | 1304 +---------------------------------------------------------------+ 1305 4 | Block Total Length | 1306 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1307 8 | Record Type | Record Value Length | 1308 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1309 12 / Record Value / 1310 / variable length, padded to 32 bits / 1311 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1312 . . 1313 . . . . other records . . . . 1314 . . 1315 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1316 | Record Type = nrb_record_end | Record Value Length = 0 | 1317 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1318 / / 1319 / Options (variable) / 1320 / / 1321 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1322 | Block Total Length | 1323 +---------------------------------------------------------------+ 1325 Figure 13: Name Resolution Block Format 1327 The Name Resolution Block has the following fields: 1329 o Block Type: The block type of the Name Resolution Block is 4. 1331 o Block Total Length: total size of this block, as described in 1332 Section 3.1. 1334 This is followed by zero or more Name Resolution Records (in the TLV 1335 format), each of which contains an association between a network 1336 address and a name. An nrb_record_end MUST be added after the last 1337 Record, and MUST exist even if there are no other Records in the NRB. 1338 There are currently three possible types of records: 1340 +-----------------+--------+----------+ 1341 | Name | Code | Length | 1342 +-----------------+--------+----------+ 1343 | nrb_record_end | 0x0000 | 0 | 1344 | nrb_record_ipv4 | 0x0001 | variable | 1345 | nrb_record_ipv6 | 0x0002 | variable | 1346 +-----------------+--------+----------+ 1348 Table 5: Name Resolution Block Records 1350 nrb_record_end: 1351 The nrb_record_end record delimits the end of name resolution 1352 records. This record is needed to determine when the list of 1353 name resolution records has ended and some options (if any) 1354 begin. 1356 nrb_record_ipv4: 1357 The nrb_record_ipv4 record specifies an IPv4 address 1358 (contained in the first 4 octets), followed by one or more 1359 zero-terminated UTF-8 strings containing the DNS entries for 1360 that address. The minimum valid Record Length for this 1361 Record Type is thus 6: 4 for the IP octets, 1 character, and 1362 a zero-value octet terminator. Note that the IP address is 1363 treated as four octets, one for each octet of the IP address; 1364 it is not a 32-bit word, and thus the endianness of the SHB 1365 does not affect this field's value. 1367 Example: '127 0 0 1'"localhost". 1369 [Open issue: is an empty string (i.e., just a zero-value 1370 octet) valid?] 1372 nrb_record_ipv6: 1373 The nrb_record_ipv6 record specifies an IPv6 address 1374 (contained in the first 16 octets), followed by one or more 1375 zero-terminated strings containing the DNS entries for that 1376 address. The minimum valid Record Length for this Record 1377 Type is thus 18: 16 for the IP octets, 1 character, and a 1378 zero-value octet terminator. 1380 Example: '20 01 0d b8 00 00 00 00 00 00 00 00 12 34 56 1381 78'"somehost". 1383 [Open issue: is an empty string (i.e., just a zero-value 1384 octet) valid?] 1386 Record Types other than those specified earlier MUST be ignored and 1387 skipped past. More Record Types will likely be defined in the 1388 future, and MUST NOT break backwards compatibility. 1390 Each Record Value is aligned to and padded to a 32-bit boundary. The 1391 corresponding Record Value Length reflects the actual length of the 1392 Record Value; it does not include the lengths of the Record Type 1393 field, the Record Value Length field, any padding for the Record 1394 Value, or anything after the Record Value. For Record Types with 1395 name strings, the Record Length does include the zero-value octet 1396 terminating that string. A Record Length of 0 is valid, unless 1397 indicated otherwise. 1399 After the list of Name Resolution Records, optionally, a list of 1400 options (formatted according to the rules defined in Section 3.5) can 1401 be present. 1403 In addition to the options defined in Section 3.5, the following 1404 options are valid within this block: 1406 +---------------+------+----------+-------------------+ 1407 | Name | Code | Length | Multiple allowed? | 1408 +---------------+------+----------+-------------------+ 1409 | ns_dnsname | 2 | variable | no | 1410 | ns_dnsIP4addr | 3 | 4 | no | 1411 | ns_dnsIP6addr | 4 | 16 | no | 1412 +---------------+------+----------+-------------------+ 1414 Table 6: Name Resolution Block Options 1416 ns_dnsname: 1417 The ns_dnsname option is a UTF-8 string containing the name 1418 of the machine (DNS server) used to perform the name 1419 resolution. The string is not zero-terminated. 1421 Example: "our_nameserver". 1423 ns_dnsIP4addr: 1424 The ns_dnsIP4addr option specifies the IPv4 address of the 1425 DNS server. Note that the IP address is treated as four 1426 octets, one for each octet of the IP address; it is not a 1427 32-bit word, and thus the endianness of the SHB does not 1428 affect this field's value. 1430 Example: '192 168 0 1'. 1432 ns_dnsIP6addr: 1433 The ns_dnsIP6addr option specifies the IPv6 address of the 1434 DNS server. 1436 Example: '20 01 0d b8 00 00 00 00 00 00 00 00 12 34 56 78'. 1438 4.6. Interface Statistics Block 1440 The Interface Statistics Block (ISB) contains the capture statistics 1441 for a given interface and it is optional. The statistics are 1442 referred to the interface defined in the current Section identified 1443 by the Interface ID field. An Interface Statistics Block is normally 1444 placed at the end of the file, but no assumptions can be taken about 1445 its position - it can even appear multiple times for the same 1446 interface. 1448 The format of the Interface Statistics Block is shown in Figure 14. 1450 0 1 2 3 1451 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1452 +---------------------------------------------------------------+ 1453 0 | Block Type = 0x00000005 | 1454 +---------------------------------------------------------------+ 1455 4 | Block Total Length | 1456 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1457 8 | Interface ID | 1458 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1459 12 | Timestamp (High) | 1460 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1461 16 | Timestamp (Low) | 1462 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1463 20 / / 1464 / Options (variable) / 1465 / / 1466 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1467 | Block Total Length | 1468 +---------------------------------------------------------------+ 1470 Figure 14: Interface Statistics Block Format 1472 The fields have the following meaning: 1474 o Block Type: The block type of the Interface Statistics Block is 5. 1476 o Block Total Length: total size of this block, as described in 1477 Section 3.1. 1479 o Interface ID: specifies the interface these statistics refers to; 1480 the correct interface will be the one whose Interface Description 1481 Block (within the current Section of the file) is identified by 1482 same number (see Section 4.2) of this field. 1484 o Timestamp: time this statistics refers to. The format of the 1485 timestamp is the same already defined in the Enhanced Packet Block 1486 (Section 4.3); the length of a unit of time is specified by the 1487 'if_tsresol' option (see Figure 10) of the Interface Description 1488 Block referenced by this packet. 1490 o Options: optionally, a list of options (formatted according to the 1491 rules defined in Section 3.5) can be present. 1493 All the statistic fields are defined as options in order to deal with 1494 systems that do not have a complete set of statistics. Therefore, In 1495 addition to the options defined in Section 3.5, the following options 1496 are valid within this block: 1498 +------------------+------+--------+-------------------+ 1499 | Name | Code | Length | Multiple allowed? | 1500 +------------------+------+--------+-------------------+ 1501 | isb_starttime | 2 | 8 | no | 1502 | isb_endtime | 3 | 8 | no | 1503 | isb_ifrecv | 4 | 8 | no | 1504 | isb_ifdrop | 5 | 8 | no | 1505 | isb_filteraccept | 6 | 8 | no | 1506 | isb_osdrop | 7 | 8 | no | 1507 | isb_usrdeliv | 8 | 8 | no | 1508 +------------------+------+--------+-------------------+ 1510 Table 7: Interface Statistics Block Options 1512 isb_starttime: 1513 The isb_starttime option specifies the time the capture 1514 started; time will be stored in two blocks of four octets 1515 each. The format of the timestamp is the same as the one 1516 defined in the Enhanced Packet Block (Section 4.3); the 1517 length of a unit of time is specified by the 'if_tsresol' 1518 option (see Figure 10) of the Interface Description Block 1519 referenced by this packet. 1521 Example: '96 c3 04 00 73 89 6a 65', in Little Endian, decodes 1522 to 2012-06-29 06:17:00.834163 UTC. 1524 isb_endtime: 1525 The isb_endtime option specifies the time the capture ended; 1526 time will be stored in two blocks of four octets each. The 1527 format of the timestamp is the same as the one defined in the 1528 Enhanced Packet Block (Section 4.3); the length of a unit of 1529 time is specified by the 'if_tsresol' option (see Figure 10) 1530 of the Interface Description Block referenced by this packet. 1532 Example: '97 c3 04 00 aa 47 ca 64', in Little Endian, decodes 1533 to 2012-06-29 07:28:25.298858 UTC. 1535 isb_ifrecv: 1536 The isb_ifrecv option specifies the 64-bit unsigned integer 1537 number of packets received from the physical interface 1538 starting from the beginning of the capture. 1540 Example: the decimal number 100. 1542 isb_ifdrop: 1543 The isb_ifdrop option specifies the 64-bit unsigned integer 1544 number of packets dropped by the interface due to lack of 1545 resources starting from the beginning of the capture. 1547 Example: '0'. 1549 isb_filteraccept: 1550 The isb_filteraccept option specifies the 64-bit unsigned 1551 integer number of packets accepted by filter starting from 1552 the beginning of the capture. 1554 Example: the decimal number 100. 1556 isb_osdrop: 1557 The isb_osdrop option specifies the 64-bit unsigned integer 1558 number of packets dropped by the operating system starting 1559 from the beginning of the capture. 1561 Example: '0'. 1563 isb_usrdeliv: 1564 The isb_usrdeliv option specifies the 64-bit unsigned integer 1565 number of packets delivered to the user starting from the 1566 beginning of the capture. The value contained in this field 1567 can be different from the value 'isb_filteraccept - 1568 isb_osdrop' because some packets could still be in the OS 1569 buffers when the capture ended. 1571 Example: '0'. 1573 All the fields that refer to packet counters are 64-bit values, 1574 represented with the octet order of the current section. Special 1575 care must be taken in accessing these fields: since all the blocks 1576 are aligned to a 32-bit boundary, such fields are not guaranteed to 1577 be aligned on a 64-bit boundary. 1579 4.7. systemd Journal Export Block 1581 The systemd [6] Journal Export Block [7] is a lightweight containter 1582 for systemd Journal Export Format entry data. 1584 One of the primary components of the systemd System and Service 1585 Manager is the "Journal", a message logging system that uses arrays 1586 of key-value pairs. Journal entries are stored in a databse-like 1587 file on disk but can be serialized to easily parseable "Journal 1588 Export Format" data or to a JSON object. The block described here is 1589 limited to Journal Export Format data only. 1591 A systemd Journal Export Block contains a single systemd Journal 1592 Export Format entry. Each entry MUST contain a __REALTIME_TIMESTAMP= 1593 field. If a timestamp for the block is required it can be derived 1594 from this field. Each entry MUST be zero-padded to 32 bits. 1595 Although the primary use of this block is intended for importing data 1596 from systemd, it could potentially be used to include arbitrary key- 1597 value data in a capture file. 1599 Figure 15 shows the format of the Journal Export Block. 1601 0 1 2 3 1602 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1603 +---------------------------------------------------------------+ 1604 0 | Block Type = 0x00000009 | 1605 +---------------------------------------------------------------+ 1606 4 | Block Total Length | 1607 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1608 8 / / 1609 / Journal Entry / 1610 / variable length, padded to 32 bits / 1611 / / 1612 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1613 | Block Total Length | 1614 +---------------------------------------------------------------+ 1616 Figure 15: systemd Journal Export Block Format 1618 The systemd Journal Export Block has the following fields: 1620 o Block Type: The block type of the Journal Export Block is 9. 1622 o Block Total Length: total size of this block, as described in 1623 Section 3.1. 1625 o Journal Entry: A journal entry as described in the Journal Export 1626 Format [8] documentation. Entries consist of a series of field 1627 names followed by text or binary field data. Common field names 1628 can be found in the systemd.journal-fields [9] documentation. The 1629 __REALTIME_TIMESTAMP= field MUST be present and valid as described 1630 above. Entries are not guaranteed to be a multiple of four octets 1631 and must be zero-padded. This allows the length of the entry to 1632 be be determined by finding the last non-zero octet in the Journal 1633 Entry data. An entry may contain an entry separator (trailing 1634 newline) as described in the Journal Export Format specification 1636 4.8. Decryption Secrets Block 1638 A Decryption Secrets Block (DSB) stores (session) secrets that enable 1639 decryption of packets within the capture file. The format of these 1640 secrets is defined by the Secrets Type. 1642 Multiple DSBs can exist in a pcapng file, but they SHOULD be written 1643 before packet blocks that require those secrets. Tools MAY limit 1644 decryption to secrets that appear before packet blocks. 1646 The structure of a Decryption Secrets Block is shown in Figure 16. 1648 0 1 2 3 1649 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1650 +---------------------------------------------------------------+ 1651 0 | Block Type = 0x0000000A | 1652 +---------------------------------------------------------------+ 1653 4 | Block Total Length | 1654 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1655 8 | Secrets Type | 1656 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1657 12 | Secrets Length | 1658 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1659 16 / / 1660 / Secrets Data / 1661 / (variable length, padded to 32 bits) / 1662 / / 1663 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1664 / / 1665 / Options (variable) / 1666 / / 1667 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1668 / Block Total Length / 1669 +---------------------------------------------------------------+ 1671 Figure 16: Decryption Secrets Block Format 1673 The Decryption Secrets Block has the following fields. 1675 o Block Type: The block type of the Decryption Secrets Block is 10. 1677 o Block Total Length: total size of this block, as described in 1678 Section 3.1. 1680 o Secrets Type (32 bits): an unsigned integer identifier that 1681 describes the format of the following Secrets field. Requests for 1682 new Secrets Type codes should be sent to the pcap-ng-format 1683 mailing list [10]. 1685 o Secrets Length (32 bits): an unsigned integer that indicates the 1686 size of the following Secrets field, without any padding octets. 1688 o Secrets Data: binary data containing secrets, padded to a 32 bit 1689 boundary. 1691 o Options: optionally, a list of options (formatted according to the 1692 rules defined in Section 3.5) can be present. No DSB-specific 1693 options are currently defined. 1695 The following is a list of Secrets Types. 1697 0x544c534b: 1698 TLS Key Log. This format is described at NSS Key Log Format 1699 [11]. Every line MUST be properly terminated with either 1700 carriage return and linefeed ('\r\n') or linefeed ('\n'). 1701 Tools MUST be able to handle both line endings. 1703 0x57474b4c: 1704 WireGuard Key Log. Every line consists of the key type, 1705 equals sign ('='), and the base64-encoded 32-byte key with 1706 optional spaces before and in between. The key type is one 1707 of LOCAL_STATIC_PRIVATE_KEY, REMOTE_STATIC_PUBLIC_KEY, 1708 LOCAL_EPHEMERAL_PRIVATE_KEY, or PRESHARED_KEY. This matches 1709 the output of extract-handshakes.sh [12] which is part of the 1710 WireGuard [13] project. A PRESHARED_KEY line is linked to a 1711 session matched by a previous LOCAL_EPHEMERAL_PRIVATE_KEY 1712 line. Every line MUST be properly terminated with either 1713 carriage return and linefeed ('\r\n') or linefeed ('\n'). 1714 Tools MUST be able to handle both line endings. 1716 Warning: LOCAL_STATIC_PRIVATE_KEY and potentially 1717 PRESHARED_KEY are long-term secrets, users SHOULD only store 1718 non-production keys, or ensure proper protection of the 1719 pcapng file. 1721 4.9. Custom Block 1723 A Custom Block (CB) is the container for storing custom data that is 1724 not part of another block; for storing custom data as part of another 1725 block, see Section 3.5.1. The Custom Block is optional, can be 1726 repeated any number of times, and can appear before or after any 1727 other block except the first Section Header Block which must come 1728 first in the file. Different Custom Blocks, of different type codes 1729 and/or different Private Enterprise Numbers, may be used in the same 1730 pcapng file. The format of a Custom Block is shown in Figure 17. 1732 0 1 2 3 1733 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1734 +---------------------------------------------------------------+ 1735 0 | Block Type = 0x00000BAD or 0x40000BAD | 1736 +---------------------------------------------------------------+ 1737 4 | Block Total Length | 1738 +---------------------------------------------------------------+ 1739 8 | Private Enterprise Number (PEN) | 1740 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1741 12 / / 1742 / Custom Data / 1743 / variable length, padded to 32 bits / 1744 / / 1745 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1746 / / 1747 / Options (variable) / 1748 / / 1749 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1750 | Block Total Length | 1751 +---------------------------------------------------------------+ 1753 Figure 17: Custom Block Format 1755 The Custom Block uses the type code 0x00000BAD (2989 in decimal) for 1756 a custom block that pcapng re-writers can copy into new files, and 1757 the type code 0x40000BAD (1073744813 in decimal) for one that should 1758 not be copied. See Section 6.2 for details. 1760 The Custom Block has the following fields: 1762 o Block Type: The block type of the Custom Block is 0x00000BAD or 1763 0x40000BAD, as described previously. 1765 o Block Total Length: total size of this block, as described in 1766 Section 3.1. 1768 o Private Enterprise Number (32 bits): An IANA-assigned Private 1769 Enterprise Number identifying the organization which defined the 1770 Custom Block. See Section 6.1 for details. The PEN MUST be 1771 encoded using the same endianness as the Section Header Block it 1772 is within the scope of. 1774 o Custom Data: the custom data, padded to a 32 bit boundary. 1776 o Options: optionally, a list of options (formatted according to the 1777 rules defined in Section 3.5) can be present. Note that custom 1778 options for the Custom Block still use the custom option format 1779 and type code, as described in Section 3.5.1. 1781 5. Experimental Blocks (deserve further investigation) 1783 5.1. Alternative Packet Blocks (experimental) 1785 Can some other packet blocks (besides the ones described in the 1786 previous paragraphs) be useful? 1788 5.2. Compression Block (experimental) 1790 The Compression Block is optional. A file can contain an arbitrary 1791 number of these blocks. A Compression Block, as the name says, is 1792 used to store compressed data. Its format is shown in Figure 18. 1794 0 1 2 3 1795 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1796 +---------------------------------------------------------------+ 1797 | Block Type = ? | 1798 +---------------------------------------------------------------+ 1799 | Block Total Length | 1800 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1801 | Compr. Type | | 1802 +-+-+-+-+-+-+-+-+ | 1803 | | 1804 | Compressed Data | 1805 | | 1806 | variable length, octet-aligned and padded to end on a 32-bit | 1807 | boundary | 1808 | | 1809 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1810 | Block Total Length | 1811 +---------------------------------------------------------------+ 1813 Figure 18: Compression Block Format 1815 The fields have the following meaning: 1817 o Block Type: The block type of the Compression Block is not yet 1818 assigned. 1820 o Block Total Length: total size of this block, as described in 1821 Section 3.1. 1823 o Compression Type (8 bits): an unsigned value that specifies the 1824 compression algorithm. Possible values for this field are 0 1825 (uncompressed), 1 (Lempel-Ziv), 2 (Gzip), other?? Probably some 1826 kind of dumb and fast compression algorithm could be effective 1827 with some types of traffic (for example web), but which? 1829 o Compressed Data: data of this block. Once decompressed, it is 1830 made of other blocks. 1832 5.3. Encryption Block (experimental) 1834 The Encryption Block is optional. A file can contain an arbitrary 1835 number of these blocks. An Encryption Block is used to store 1836 encrypted data. Its format is shown in Figure 19. 1838 0 1 2 3 1839 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1840 +---------------------------------------------------------------+ 1841 | Block Type = ? | 1842 +---------------------------------------------------------------+ 1843 | Block Total Length | 1844 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1845 | Encr. Type | | 1846 +-+-+-+-+-+-+-+-+ | 1847 | | 1848 | Encrypted Data | 1849 | | 1850 | variable length, octet-aligned | 1851 | | 1852 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1853 | Block Total Length | 1854 +---------------------------------------------------------------+ 1856 Figure 19: Encryption Block Format 1858 The fields have the following meaning: 1860 o Block Type: The block type of the Encryption Block is not yet 1861 assigned. 1863 o Block Total Length: total size of this block, as described in 1864 Section 3.1. 1866 o Encryption Type (8 bits): an unsigned value that specifies the 1867 encryption algorithm. Possible values for this field are ??? 1868 (TODO) NOTE: this block should probably contain other fields, 1869 depending on the encryption algorithm. To be defined precisely. 1871 o Encrypted Data: data of this block. Once decrypted, it originates 1872 other blocks. 1874 5.4. Fixed Length Block (experimental) 1876 The Fixed Length Block is optional. A file can contain an arbitrary 1877 number of these blocks. A Fixed Length Block can be used to optimize 1878 the access to the file. Its format is shown in Figure 20. A Fixed 1879 Length Block stores records with constant size. It contains a set of 1880 Blocks (normally Enhanced Packet Blocks or Simple Packet Blocks), of 1881 which it specifies the size. Knowing this size a priori helps to 1882 scan the file and to load some portions of it without truncating a 1883 block, and is particularly useful with cell-based networks like ATM. 1885 0 1 2 3 1886 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1887 +---------------------------------------------------------------+ 1888 | Block Type = ? | 1889 +---------------------------------------------------------------+ 1890 | Block Total Length | 1891 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1892 | Cell Size | | 1893 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 1894 | | 1895 | Fixed Size Data | 1896 | | 1897 | variable length, octet-aligned | 1898 | | 1899 | | 1900 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1901 | Block Total Length | 1902 +---------------------------------------------------------------+ 1904 Figure 20: Fixed Length Block Format 1906 The fields have the following meaning: 1908 o Block Type: The block type of the Fixed Length Block is not yet 1909 assigned. 1911 o Block Total Length: total size of this block, as described in 1912 Section 3.1. 1914 o Cell size (16 bits): an unsigned value that indicates the size of 1915 the blocks contained in the data field. 1917 o Fixed Size Data: data of this block. 1919 5.5. Directory Block (experimental) 1921 If present, this block contains the following information: 1923 o number of indexed packets (N) 1925 o table with position and length of any indexed packet (N entries) 1927 A directory block MUST be followed by at least N packets, otherwise 1928 it MUST be considered invalid. It can be used to efficiently load 1929 portions of the file to memory and to support operations on memory 1930 mapped files. This block can be added by tools like network 1931 analyzers as a consequence of file processing. 1933 5.6. Traffic Statistics and Monitoring Blocks (experimental) 1935 One or more blocks could be defined to contain network statistics or 1936 traffic monitoring information. They could be use to store data 1937 collected from RMON or Netflow probes, or from other network 1938 monitoring tools. 1940 5.7. Event/Security Block (experimental) 1942 This block could be used to store events. Events could contain 1943 generic information (for example network load over 50%, server 1944 down...) or security alerts. An event could be: 1946 o skipped, if the application doesn't know how to do with it 1948 o processed independently by the packets. In other words, the 1949 applications skips the packets and processes only the alerts 1951 o processed in relation to packets: for example, a security tool 1952 could load only the packets of the file that are near a security 1953 alert; a monitoring tool could skip the packets captured while the 1954 server was down. 1956 6. Vendor-Specific Custom Extensions 1958 This section uses the term "vendor" to describe an organization which 1959 extends the pcapng file with custom, proprietary blocks or options. 1960 It should be noted, however, that the "vendor" is just an abstract 1961 entity that agrees on a custom extension format: for example it may 1962 be a manufacturer, industry association, an individual user, or 1963 collective group of users. 1965 6.1. Supported Use-Cases 1967 There are two different supported use-cases for vendor-specific 1968 custom extensions: local and portable. Local use means the custom 1969 data is only expected to be usable on the same machine, and the same 1970 application, which encoded it into the file. This limitation is due 1971 to the lack of a common registry for the local use number codes (the 1972 block or option type code numbers with the Most Significant Bit set). 1973 Since two different vendors may choose the same number, one vendor's 1974 application reading the other vendor's file would result in decoding 1975 failure. Therefore, vendors SHOULD instead use the portable method, 1976 as described next. 1978 The portable use-case supports vendor-specific custom extensions in 1979 pcapng files which can be shared across systems, organizations, etc. 1980 To avoid number space collisions, an IANA-registered Private 1981 Enterprise Number (PEN) is encoded into the Custom Block or Custom 1982 Option, using the PEN number that belongs to the vendor defining the 1983 extension. Anyone can register a new PEN with IANA, for free, by 1984 filling out the online request form at http://pen.iana.org/pen/ 1985 PenApplication.page [14]. 1987 6.2. Controlling Copy Behavior 1989 Both Custom Blocks and Custom Options support two different codes to 1990 distinguish their "copy" behavior: a code for when the block or 1991 option can be safely copied into a new pcapng file by a pcapng 1992 manipulating application, and a code for when it should not be 1993 copied. A common reason for not copying a Custom Block or Custom 1994 Option is because it depends on other blocks or options in some way 1995 that would invalidate the custom data if the other blocks/options 1996 were removed or re-ordered. For example, if a Custom Block's data 1997 includes an Interface ID number in its Custom Data portion, then it 1998 cannot be safely copied by a pcapng application that merges pcapng 1999 files, because the merging application might re-order or remove one 2000 or more of the Interface Description Blocks, and thereby change the 2001 Interface IDs that the Custom Block depends upon. The same issue 2002 arises if a Custom Block or Custom Option depends on the presence of, 2003 or specific ordering of, other standard-based or custom-defined 2004 blocks or options. 2006 Note that the copy semantics is not related to privacy - there is no 2007 guarantee that a pcapng anonymizer will remove a Custom Block or 2008 Custom Option, even if the appropriate code is used requesting it not 2009 be copied; and the original pcapng file can be shared anyway. If the 2010 Custom Data portion of the Custom Block or Custom Option contains 2011 sensitive information, then it should be encrypted in some fashion. 2013 6.3. Strings vs. Octets 2015 For the Custom Options, there are two Custom Data formats supported: 2016 a UTF-8 string and a binary data payload. The rationale for this 2017 separation is that a pcapng display application which does not 2018 understand the specific PEN's Custom Option can still display the 2019 data as a string if it's a string type code, rather than as hex-ascii 2020 of the octets. 2022 6.4. Endianness Issues 2024 Implementers writing Custom Blocks or Custom Options should be aware 2025 that a pcapng file can be re-written by machines using a different 2026 endianness than the original file, which means all known fields of 2027 the pcapng file will change endianness in the new file. Since the 2028 Custom Data payload of the Custom Block or Custom Option might be an 2029 arbitrary sequence of unknown octets to such machines, they cannot 2030 convert multi-octet values inside the Custom Data into the 2031 appropriate endianness. 2033 For example, a little-endian machine can create a new pcapng file and 2034 add some binary data Custom Options to some Block(s) in the file. 2035 This file can then be sent to a big-endian host, which will convert 2036 it to big-endian format if it re-writes the file. It will, however, 2037 leave the Custom Data payload alone (as little-endian format). If 2038 this file then gets sent back to the little-endian machine, then when 2039 that little-endian machine reads the file it will detect the format 2040 is big- endian, and swap the endianness while it parses the file - 2041 but that will cause the Custom Data payload to be incorrect since it 2042 was already in little-endian format. 2044 Therefore, the vendor should either encode all of their fields in a 2045 consistent manner, such as always in big-endian or always little- 2046 endian format, regardless of the host platform's endianness, or 2047 should encode some flag in the Custom Data payload to indicate in 2048 which endianness the rest of the payload is written. 2050 7. Recommended File Name Extension: .pcapng 2052 The recommended file name extension for the "PCAP Next Generation 2053 Capture File Format" specified in this document is ".pcapng". 2055 On Windows and macOS, files are distinguished by an extension to 2056 their filename. Such an extension is technically not actually 2057 required, as applications should be able to automatically detect the 2058 pcapng file format through the "magic bytes" at the beginning of the 2059 file, as some other UN*X desktop environments do. However, using 2060 name extensions makes it easier to work with files (e.g. visually 2061 distinguish file formats) so it is recommended - though not required 2062 - to use .pcapng as the name extension for files following this 2063 specification. 2065 Please note: To avoid confusion (such as the current usage of .cap 2066 for a plethora of different capture file formats) other file name 2067 extensions than .pcapng should be avoided. 2069 8. Conclusions 2071 The file format proposed in this document should be very versatile 2072 and satisfy a wide range of applications. In the simplest case, it 2073 can contain a raw capture of the network data, made of a series of 2074 Simple Packet Blocks. In the most complex case, it can be used as a 2075 repository for heterogeneous information. In every case, the file 2076 remains easy to parse and an application can always skip the data it 2077 is not interested in; at the same time, different applications can 2078 share the file, and each of them can benefit of the information 2079 produced by the others. Two or more files can be concatenated 2080 obtaining another valid file. 2082 9. Implementations 2084 Some known implementations that read or write the pcapng file format 2085 are listed on the pcapng GitHub wiki [15]. 2087 10. Security Considerations 2089 TBD. 2091 11. IANA Considerations 2093 TBD. 2095 [Open issue: decide whether the block types, option types, NRB Record 2096 types, etc. should be IANA registries. And if so, what the IANA 2097 policy for each should be (see RFC 5226)] 2099 11.1. Standardized Block Type Codes 2101 Every Block is uniquely identified by a 32-bit integer value, stored 2102 in the Block Header. 2104 As pointed out in Section 3.1, Block Type codes whose Most 2105 Significant Bit (bit 31) is set to 1 are reserved for local use by 2106 the application. 2108 All the remaining Block Type codes (0x00000000 to 0x7FFFFFFF) are 2109 standardized by this document. Requests for new Block Type codes 2110 should be sent to the pcap-ng-format mailing list [16]. 2112 The following is a list of the Standardized Block Type Codes: 2114 +-----------------------+-------------------------------------------+ 2115 | Block Type Code | Description | 2116 +-----------------------+-------------------------------------------+ 2117 | 0x00000000 | Reserved ??? | 2118 | 0x00000001 | Interface Description Block | 2119 | | (Section 4.2) | 2120 | 0x00000002 | Packet Block (Appendix A) | 2121 | 0x00000003 | Simple Packet Block | 2122 | | (Section 4.4) | 2123 | 0x00000004 | Name Resolution Block | 2124 | | (Section 4.5) | 2125 | 0x00000005 | Interface Statistics Block | 2126 | | (Section 4.6) | 2127 | 0x00000006 | Enhanced Packet Block | 2128 | | (Section 4.3) | 2129 | 0x00000007 | IRIG Timestamp Block | 2130 | | (requested by Gianluca Varenni | 2131 | | , CACE | 2132 | | Technologies LLC); code also | 2133 | | used for Socket Aggregation | 2134 | | Event Block [17] | 2135 | 0x00000008 | ARINC 429 [18] | 2136 | | in AFDX Encapsulation Information Block | 2137 | | (requested by Gianluca | 2138 | | Varenni , | 2139 | | CACE Technologies LLC) | 2140 | 0x00000009 | systemd Journal Export Block | 2141 | | (Section 4.7) | 2142 | 0x0000000A | Decryption Secrets Block | 2143 | | (Section 4.8) | 2144 | 0x00000101 | Hone Project [19] | 2145 | | Machine Info Block [20] (see | 2146 | | also Google version [21]) | 2147 | 0x00000102 | Hone Project [22] | 2148 | | Connection Event Block [23] | 2149 | | (see also Google version | 2150 | | [24]) | 2151 | 0x00000201 | Sysdig [25] | 2152 | | Machine Info Block | 2153 | 0x00000202 | Sysdig [26] | 2154 | | Process Info Block, version 1 | 2155 | 0x00000203 | Sysdig [27] FD | 2156 | | List Block | 2157 | 0x00000204 | Sysdig [28] | 2158 | | Event Block | 2159 | 0x00000205 | Sysdig [29] | 2160 | | Interface List Block | 2161 | 0x00000206 | Sysdig [30] User | 2162 | | List Block | 2163 | 0x00000207 | Sysdig [31] | 2164 | | Process Info Block, version 2 | 2165 | 0x00000208 | Sysdig [32] | 2166 | | Event Block with flags | 2167 | 0x00000209 | Sysdig [33] | 2168 | | Process Info Block, version 3 | 2169 | 0x00000210 | Sysdig [34] | 2170 | | Process Info Block, version 4 | 2171 | 0x00000211 | Sysdig [35] | 2172 | | Process Info Block, version 5 | 2173 | 0x00000212 | Sysdig [36] | 2174 | | Process Info Block, version 6 | 2175 | 0x00000213 | Sysdig [37] | 2176 | | Process Info Block, version 7 | 2177 | 0x00000BAD | Custom Block that rewriters | 2178 | | can copy into new files | 2179 | | (Section 4.9) | 2180 | 0x40000BAD | Custom Block that rewriters | 2181 | | should not copy into new | 2182 | | files (Section 4.9) | 2183 | 0x0A0D0D0A | Section Header Block | 2184 | | (Section 4.1) | 2185 | 0x0A0D0A00-0x0A0D0AFF | Reserved. Used to detect trace files | 2186 | | corrupted because of file | 2187 | | transfers using the HTTP protocol in text | 2188 | | mode. | 2189 | 0x000A0D0A-0xFF0A0D0A | Reserved. Used to detect trace files | 2190 | | corrupted because of file | 2191 | | transfers using the HTTP protocol in text | 2192 | | mode. | 2193 | 0x000A0D0D-0xFF0A0D0D | Reserved. Used to detect trace files | 2194 | | corrupted because of file | 2195 | | transfers using the HTTP protocol in text | 2196 | | mode. | 2197 | 0x0D0D0A00-0x0D0D0AFF | Reserved. Used to detect trace files | 2198 | | corrupted because of file | 2199 | | transfers using the FTP protocol in text | 2200 | | mode. | 2201 | 0x80000000-0xFFFFFFFF | Reserved for local use. | 2202 +-----------------------+-------------------------------------------+ 2203 Table 8: Standardized Block Type Codes 2205 [Open issue: reserve 0x40000000-0x7FFFFFFF for do-not-copy-bit range 2206 of base types?] 2208 12. Contributors 2210 Loris Degioanni and Gianluca Varenni were coauthoring this document 2211 before it was submitted to the IETF. 2213 13. Acknowledgments 2215 The authors wish to thank Anders Broman, Ulf Lamping, Richard Sharpe 2216 and many others for their invaluable comments. 2218 14. References 2220 14.1. Normative References 2222 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 2223 Requirement Levels", BCP 14, RFC 2119, 2224 DOI 10.17487/RFC2119, March 1997, 2225 . 2227 14.2. URIs 2229 [1] https://www.winpcap.org/mailman/listinfo/pcap-ng-format 2231 [2] https://www.winpcap.org/mailman/listinfo/pcap-ng-format 2233 [3] http://www.tcpdump.org/linktypes.html 2235 [4] http://www.tcpdump.org/linktypes.html 2237 [5] http://www.tcpdump.org/linktypes.html 2239 [6] https://www.freedesktop.org/wiki/Software/systemd/ 2241 [7] https://www.freedesktop.org/wiki/Software/systemd/export/ 2243 [8] https://www.freedesktop.org/wiki/Software/systemd/export/ 2245 [9] https://www.freedesktop.org/software/systemd/man/systemd.journal- 2246 fields.html 2248 [10] https://www.winpcap.org/mailman/listinfo/pcap-ng-format 2250 [11] https://developer.mozilla.org/NSS_Key_Log_Format 2252 [12] https://git.zx2c4.com/WireGuard/tree/contrib/examples/extract- 2253 handshakes/README 2255 [13] https://www.wireguard.com/ 2257 [14] http://pen.iana.org/pen/PenApplication.page 2259 [15] https://github.com/pcapng/pcapng/wiki/Implementations 2261 [16] https://www.winpcap.org/mailman/listinfo/pcap-ng-format 2263 [17] https://github.com/google/linux-sensor/blob/master/hone- 2264 pcapng.txt 2266 [18] https://en.wikipedia.org/wiki/ARINC_429 2268 [19] https://github.com/HoneProject 2270 [20] https://github.com/HoneProject/Linux-Sensor/wiki/Augmented-PCAP- 2271 Next-Generation-Dump-File-Format 2273 [21] https://github.com/google/linux-sensor/blob/master/hone- 2274 pcapng.txt 2276 [22] https://github.com/HoneProject 2278 [23] https://github.com/HoneProject/Linux-Sensor/wiki/Augmented-PCAP- 2279 Next-Generation-Dump-File-Format 2281 [24] https://github.com/google/linux-sensor/blob/master/hone- 2282 pcapng.txt 2284 [25] https://github.com/draios/sysdig 2286 [26] https://github.com/draios/sysdig 2288 [27] https://github.com/draios/sysdig 2290 [28] https://github.com/draios/sysdig 2292 [29] https://github.com/draios/sysdig 2294 [30] https://github.com/draios/sysdig 2296 [31] https://github.com/draios/sysdig 2298 [32] https://github.com/draios/sysdig 2300 [33] https://github.com/draios/sysdig 2302 [34] https://github.com/draios/sysdig 2304 [35] https://github.com/draios/sysdig 2306 [36] https://github.com/draios/sysdig 2308 [37] https://github.com/draios/sysdig 2310 [38] http://www.tcpdump.org/linktypes.html 2312 Appendix A. Packet Block (obsolete!) 2314 The Packet Block is obsolete, and MUST NOT be used in new files. Use 2315 the Enhanced Packet Block or Simple Packet Block instead. This 2316 section is for historical reference only. 2318 A Packet Block was a container for storing packets coming from the 2319 network. 2321 0 1 2 3 2322 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2323 +---------------------------------------------------------------+ 2324 0 | Block Type = 0x00000002 | 2325 +---------------------------------------------------------------+ 2326 4 | Block Total Length | 2327 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2328 8 | Interface ID | Drops Count | 2329 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2330 12 | Timestamp (High) | 2331 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2332 16 | Timestamp (Low) | 2333 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2334 20 | Captured Packet Length | 2335 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2336 24 | Original Packet Length | 2337 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2338 28 / / 2339 / Packet Data / 2340 / variable length, padded to 32 bits / 2341 / / 2342 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2343 / / 2344 / Options (variable) / 2345 / / 2346 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2347 | Block Total Length | 2348 +---------------------------------------------------------------+ 2350 Figure 21: Packet Block Format 2352 The Packet Block has the following fields: 2354 o Block Type: The block type of the Packet Block is 2. 2356 o Block Total Length: total size of this block, as described in 2357 Section 3.1. 2359 o Interface ID: specifies the interface this packet comes from; the 2360 correct interface will be the one whose Interface Description 2361 Block (within the current Section of the file) is identified by 2362 the same number (see Section 4.2) of this field. The interface ID 2363 MUST be valid, which means that an matching interface description 2364 block MUST exist. 2366 o Drops Count: a local drop counter. It specifies the number of 2367 packets lost (by the interface and the operating system) between 2368 this packet and the preceding one. The value xFFFF (in 2369 hexadecimal) is reserved for those systems in which this 2370 information is not available. 2372 o Timestamp (High) and Timestamp (Low): timestamp of the packet. 2373 The format of the timestamp is the same as was already defined for 2374 the Enhanced Packet Block (Section 4.3). 2376 o Captured Packet Length: number of octets captured from the packet 2377 (i.e. the length of the Packet Data field). It will be the 2378 minimum value among the Original Packet Length and the snapshot 2379 length for the interface (SnapLen, defined in Figure 10). The 2380 value of this field does not include the padding octets added at 2381 the end of the Packet Data field to align the Packet Data field to 2382 a 32-bit boundary. 2384 o Original Packet Length: actual length of the packet when it was 2385 transmitted on the network. It can be different from Captured 2386 Packet Length if the packet has been truncated by the capture 2387 process. 2389 o Packet Data: the data coming from the network, including link- 2390 layer headers. The actual length of this field is Captured Packet 2391 Length plus the padding to a 32-bit boundary. The format of the 2392 link-layer headers depends on the LinkType field specified in the 2393 Interface Description Block (see Section 4.2) and it is specified 2394 in the entry for that format in the the tcpdump.org link-layer 2395 header types registry [38]. 2397 o Options: optionally, a list of options (formatted according to the 2398 rules defined in Section 3.5) can be present. 2400 In addition to the options defined in Section 3.5, the following 2401 options were valid within this block: 2403 +------------+------+----------+-------------------+ 2404 | Name | Code | Length | Multiple allowed? | 2405 +------------+------+----------+-------------------+ 2406 | pack_flags | 2 | 4 | no | 2407 | pack_hash | 3 | variable | yes | 2408 +------------+------+----------+-------------------+ 2410 Table 9: Packet Block Options 2412 pack_flags: 2413 The pack_flags option is the same as the epb_flags of the 2414 enhanced packet block. 2416 Example: '0'. 2418 pack_hash: 2419 The pack_hash option is the same as the epb_hash of the 2420 enhanced packet block. 2422 Examples: '02 EC 1D 87 97', '03 45 6E C2 17 7C 10 1E 3C 2E 99 2423 6E C2 9A 3D 50 8E'. 2425 Authors' Addresses 2427 Michael Tuexen (editor) 2428 Muenster University of Applied Sciences 2429 Stegerwaldstrasse 39 2430 Steinfurt 48565 2431 DE 2433 Email: tuexen@fh-muenster.de 2435 Fulvio Risso 2436 Politecnico di Torino 2437 Corso Duca degli Abruzzi, 24 2438 Torino 10129 2439 IT 2441 Email: fulvio.risso@polito.it 2443 Jasper Bongertz 2444 Airbus Defence and Space CyberSecurity 2445 Kanzlei 63c 2446 Meerbusch 40667 2447 DE 2449 Email: jasper@packet-foo.com 2451 Gerald Combs 2452 Wireshark Foundation 2453 339 Madson Pl 2454 Davis, CA 95618 2455 US 2457 Email: gerald@wireshark.org 2459 Guy Harris 2461 Email: gharris@sonic.net 2462 Michael C. Richardson 2463 Sandelman Software Works 2465 Email: mcr+ietf@sandelman.ca 2466 URI: http://www.sandelman.ca/