idnits 2.17.1 draft-richardson-opsawg-pcapng-extras-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (4 October 2021) is 932 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'LINKTYPES' is defined on line 374, but no explicit reference was found in the text == Outdated reference: A later version (-05) exists of draft-tuexen-opsawg-pcapng-03 Summary: 0 errors (**), 0 flaws (~~), 3 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group M. Tuexen, Ed. 3 Internet-Draft Muenster Univ. of Appl. Sciences 4 Intended status: Informational F. Risso 5 Expires: 7 April 2022 Politecnico di Torino 6 J. Bongertz 7 Airbus DS CyberSecurity 8 G. Combs 9 Wireshark 10 G. Harris 12 E. Chaudron 13 Red Hat 14 M. Richardson 15 Sandelman 16 4 October 2021 18 Additional block types for PCAP Next Generation (pcapng) Capture File 19 Format 20 draft-richardson-opsawg-pcapng-extras-00 22 Abstract 24 This document contains a number of extensions to the PCAPng file 25 format which are outside of the IETF networking mandate. 27 Status of This Memo 29 This Internet-Draft is submitted in full conformance with the 30 provisions of BCP 78 and BCP 79. 32 Internet-Drafts are working documents of the Internet Engineering 33 Task Force (IETF). Note that other groups may also distribute 34 working documents as Internet-Drafts. The list of current Internet- 35 Drafts is at https://datatracker.ietf.org/drafts/current/. 37 Internet-Drafts are draft documents valid for a maximum of six months 38 and may be updated, replaced, or obsoleted by other documents at any 39 time. It is inappropriate to use Internet-Drafts as reference 40 material or to cite them other than as "work in progress." 42 This Internet-Draft will expire on 7 April 2022. 44 Copyright Notice 46 Copyright (c) 2021 IETF Trust and the persons identified as the 47 document authors. All rights reserved. 49 This document is subject to BCP 78 and the IETF Trust's Legal 50 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 51 license-info) in effect on the date of publication of this document. 52 Please review these documents carefully, as they describe your rights 53 and restrictions with respect to this document. Code Components 54 extracted from this document must include Simplified BSD License text 55 as described in Section 4.e of the Trust Legal Provisions and are 56 provided without warranty as described in the Simplified BSD License. 58 Table of Contents 60 1. Introduction to Additional Block Types . . . . . . . . . . . 2 61 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 2 62 3. Additional Block Types . . . . . . . . . . . . . . . . . . . 2 63 3.1. systemd Journal Export Block . . . . . . . . . . . . . . 3 64 3.2. Alternative Packet Blocks (experimental) . . . . . . . . 4 65 3.3. Compression Block (experimental) . . . . . . . . . . . . 4 66 3.4. Encryption Block (experimental) . . . . . . . . . . . . . 5 67 3.5. Fixed Length Block (experimental) . . . . . . . . . . . . 6 68 3.6. Directory Block (experimental) . . . . . . . . . . . . . 7 69 3.7. Traffic Statistics and Monitoring Blocks 70 (experimental) . . . . . . . . . . . . . . . . . . . . . 7 71 3.8. Event/Security Block (experimental) . . . . . . . . . . . 7 72 4. Security Considerations . . . . . . . . . . . . . . . . . . . 7 73 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 8 74 6. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 8 75 7. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 8 76 8. References . . . . . . . . . . . . . . . . . . . . . . . . . 8 77 8.1. Normative References . . . . . . . . . . . . . . . . . . 8 78 8.2. Informative References . . . . . . . . . . . . . . . . . 8 79 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 8 81 1. Introduction to Additional Block Types 83 TBD 85 2. Terminology 87 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 88 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 89 "OPTIONAL" in this document are to be interpreted as described in 90 BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all 91 capitals, as shown here. 93 3. Additional Block Types 94 3.1. systemd Journal Export Block 96 The systemd (https://www.freedesktop.org/wiki/Software/systemd/) 97 Journal Export Block 98 (https://www.freedesktop.org/wiki/Software/systemd/export/) is a 99 lightweight container for systemd Journal Export Format entry data. 101 One of the primary components of the systemd System and Service 102 Manager is the "Journal", a message logging system that uses arrays 103 of key-value pairs. Journal entries are stored in a database-like 104 file on disk but can be serialized to easily parseable "Journal 105 Export Format" data or to a JSON object. The block described here is 106 limited to Journal Export Format data only. 108 A systemd Journal Export Block contains a single systemd Journal 109 Export Format entry. Each entry MUST contain a __REALTIME_TIMESTAMP= 110 field. If a timestamp for the block is required it can be derived 111 from this field. Each entry MUST be zero-padded to 32 bits. 112 Although the primary use of this block is intended for importing data 113 from systemd, it could potentially be used to include arbitrary key- 114 value data in a capture file. 116 Figure 1 shows the format of the Journal Export Block. 118 1 2 3 119 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 120 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 121 0 | Block Type = 0x00000009 | 122 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 123 4 | Block Total Length | 124 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 125 8 / / 126 / Journal Entry / 127 / variable length, padded to 32 bits / 128 / / 129 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 130 | Block Total Length | 131 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 133 Figure 1: systemd Journal Export Block Format 135 The systemd Journal Export Block has the following fields: 137 * Block Type: The block type of the Journal Export Block is 9. 139 * Block Total Length: total size of this block, as described in 140 [I-D.tuexen-opsawg-pcapng], section "Section Blocks". 142 * Journal Entry: A journal entry as described in the Journal Export 143 Format (https://www.freedesktop.org/wiki/Software/systemd/export/) 144 documentation. Entries consist of a series of field names 145 followed by text or binary field data. Common field names can be 146 found in the systemd.journal-fields 147 (https://www.freedesktop.org/software/systemd/man/systemd.journal- 148 fields.html) documentation. The __REALTIME_TIMESTAMP= field MUST 149 be present and valid as described above. Entries are not 150 guaranteed to be a multiple of four octets and must be zero- 151 padded. This allows the length of the entry to be determined by 152 finding the last non-zero octet in the Journal Entry data. An 153 entry may contain an entry separator (trailing newline) as 154 described in the Journal Export Format specification 156 3.2. Alternative Packet Blocks (experimental) 158 Can some other packet blocks (besides the ones described in the 159 previous paragraphs) be useful? 161 3.3. Compression Block (experimental) 163 The Compression Block is optional. A file can contain an arbitrary 164 number of these blocks. A Compression Block, as the name says, is 165 used to store compressed data. Its format is shown in Figure 2. 167 1 2 3 168 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 169 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 170 0 | Block Type = ? | 171 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 172 4 | Block Total Length | 173 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 174 8 | Compr. Type | / 175 +-+-+-+-+-+-+-+-+ / 176 / / 177 / Compressed Data / 178 / / 179 / variable length, octet-aligned and padded to end on a 32-bit / 180 / boundary / 181 / / 182 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 183 | Block Total Length | 184 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 186 Figure 2: Compression Block Format 188 The fields have the following meaning: 190 * Block Type: The block type of the Compression Block is not yet 191 assigned. 193 * Block Total Length: total size of this block, as described in 194 [I-D.tuexen-opsawg-pcapng], section "Section Blocks". 196 * Compression Type (8 bits): an unsigned value that specifies the 197 compression algorithm. Possible values for this field are 0 198 (uncompressed), 1 (Lempel-Ziv), 2 (Gzip), other?? Probably some 199 kind of dumb and fast compression algorithm could be effective 200 with some types of traffic (for example web), but which? 202 * Compressed Data: data of this block. Once decompressed, it is 203 made of other blocks. 205 3.4. Encryption Block (experimental) 207 The Encryption Block is optional. A file can contain an arbitrary 208 number of these blocks. An Encryption Block is used to store 209 encrypted data. Its format is shown in Figure 3. 211 1 2 3 212 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 213 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 214 0 | Block Type = ? | 215 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 216 4 | Block Total Length | 217 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 218 8 | Encr. Type | / 219 +-+-+-+-+-+-+-+-+ / 220 / / 221 / Encrypted Data / 222 / / 223 / variable length, octet-aligned / 224 / / 225 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 226 | Block Total Length | 227 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 229 Figure 3: Encryption Block Format 231 The fields have the following meaning: 233 * Block Type: The block type of the Encryption Block is not yet 234 assigned. 236 * Block Total Length: total size of this block, as described in 237 [I-D.tuexen-opsawg-pcapng], section "Section Blocks". 239 * Encryption Type (8 bits): an unsigned value that specifies the 240 encryption algorithm. Possible values for this field are ??? 241 (TODO) NOTE: this block should probably contain other fields, 242 depending on the encryption algorithm. To be defined precisely. 244 * Encrypted Data: data of this block. Once decrypted, it originates 245 other blocks. 247 3.5. Fixed Length Block (experimental) 249 The Fixed Length Block is optional. A file can contain an arbitrary 250 number of these blocks. A Fixed Length Block can be used to optimize 251 the access to the file. Its format is shown in Figure 4. A Fixed 252 Length Block stores records with constant size. It contains a set of 253 Blocks (normally Enhanced Packet Blocks or Simple Packet Blocks), of 254 which it specifies the size. Knowing this size a priori helps to 255 scan the file and to load some portions of it without truncating a 256 block, and is particularly useful with cell-based networks like ATM. 258 1 2 3 259 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 260 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 261 0 | Block Type = ? | 262 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 263 4 | Block Total Length | 264 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 265 8 | Cell Size | | 266 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ / 267 / / 268 / Fixed Size Data / 269 / / 270 / variable length, octet-aligned / 271 / / 272 / / 273 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 274 | Block Total Length | 275 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 277 Figure 4: Fixed Length Block Format 279 The fields have the following meaning: 281 * Block Type: The block type of the Fixed Length Block is not yet 282 assigned. 284 * Block Total Length: total size of this block, as described in 285 [I-D.tuexen-opsawg-pcapng], section "Section Blocks". 287 * Cell size (16 bits): an unsigned value that indicates the size of 288 the blocks contained in the data field. 290 * Fixed Size Data: data of this block. 292 3.6. Directory Block (experimental) 294 If present, this block contains the following information: 296 * number of indexed packets (N) 298 * table with position and length of any indexed packet (N entries) 300 A directory block MUST be followed by at least N packets, otherwise 301 it MUST be considered invalid. It can be used to efficiently load 302 portions of the file to memory and to support operations on memory 303 mapped files. This block can be added by tools like network 304 analyzers as a consequence of file processing. 306 3.7. Traffic Statistics and Monitoring Blocks (experimental) 308 One or more blocks could be defined to contain network statistics or 309 traffic monitoring information. They could be use to store data 310 collected from RMON or Netflow probes, or from other network 311 monitoring tools. 313 3.8. Event/Security Block (experimental) 315 This block could be used to store events. Events could contain 316 generic information (for example network load over 50%, server 317 down...) or security alerts. An event could be: 319 * skipped, if the application doesn't know how to do with it 321 * processed independently by the packets. In other words, the 322 applications skips the packets and processes only the alerts 324 * processed in relation to packets: for example, a security tool 325 could load only the packets of the file that are near a security 326 alert; a monitoring tool could skip the packets captured while the 327 server was down. 329 4. Security Considerations 331 TBD. 333 5. IANA Considerations 335 TBD. 337 [Open issue: decide whether the block types, option types, NRB Record 338 types, etc. should be IANA registries. And if so, what the IANA 339 policy for each should be (see RFC 5226)] 341 6. Contributors 343 Loris Degioanni and Gianluca Varenni were coauthoring this document 344 before it was submitted to the IETF. 346 7. Acknowledgments 348 The authors wish to thank Anders Broman, Ulf Lamping, Richard Sharpe 349 and many others for their invaluable comments. 351 8. References 353 8.1. Normative References 355 [I-D.tuexen-opsawg-pcapng] 356 Tuexen, M., Risso, F., Bongertz, J., Combs, G., Harris, 357 G., Chaudron, E., and M. C. Richardson, "PCAP Next 358 Generation (pcapng) Capture File Format", Work in 359 Progress, Internet-Draft, draft-tuexen-opsawg-pcapng-03, 360 23 June 2021, . 363 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 364 Requirement Levels", BCP 14, RFC 2119, 365 DOI 10.17487/RFC2119, March 1997, 366 . 368 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 369 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 370 May 2017, . 372 8.2. Informative References 374 [LINKTYPES] 375 The Tcpdump Group, "the tcpdump.org link-layer header 376 types registry", . 378 Authors' Addresses 379 Michael Tuexen (editor) 380 Muenster University of Applied Sciences 381 Stegerwaldstrasse 39 382 48565 Steinfurt 383 Germany 385 Email: tuexen@fh-muenster.de 387 Fulvio Risso 388 Politecnico di Torino 389 Corso Duca degli Abruzzi, 24 390 10129 Torino 391 Italy 393 Email: fulvio.risso@polito.it 395 Jasper Bongertz 396 Airbus Defence and Space CyberSecurity 397 Kanzlei 63c 398 40667 Meerbusch 399 Germany 401 Email: jasper@packet-foo.com 403 Gerald Combs 404 Wireshark Foundation 405 339 Madson Pl 406 Davis, CA 95618 407 United States of America 409 Email: gerald@wireshark.org 411 Guy Harris 413 Email: gharris@sonic.net 415 Eelco Chaudron 416 Red Hat 417 De Entree 238 418 1101 EE Amsterdam 419 Netherlands 421 Email: eelco@redhat.com 422 Michael C. Richardson 423 Sandelman Software Works 425 Email: mcr+ietf@sandelman.ca 426 URI: http://www.sandelman.ca/