idnits 2.17.1 draft-dulaunoy-misp-core-format-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** There are 2 instances of too long lines in the document, the longest one being 4 characters in excess of 72. ** The abstract seems to contain references ([MISP-P]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. == There are 1 instance of lines with non-RFC2606-compliant FQDNs in the document. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (October 15, 2016) is 2750 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'MISP-T' is defined on line 1047, but no explicit reference was found in the text ** Obsolete normative reference: RFC 4627 (Obsoleted by RFC 7158, RFC 7159) Summary: 4 errors (**), 0 flaws (~~), 3 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group A. Dulaunoy 3 Internet-Draft A. Iklody 4 Intended status: Informational CIRCL 5 Expires: April 18, 2017 October 15, 2016 7 MISP core format 8 draft-dulaunoy-misp-core-format-00 10 Abstract 12 This document describes the MISP core format used to exchange 13 indicators and threat information between MISP (Malware Information 14 and threat Sharing Platform) instances. The JSON format includes the 15 overall structure along with the semantic associated for each 16 respective key. The format is described to support other 17 implementations which reuse the format and ensuring an 18 interoperability with existing MISP [MISP-P] software and other 19 Threat Intelligence Platforms. 21 Status of This Memo 23 This Internet-Draft is submitted in full conformance with the 24 provisions of BCP 78 and BCP 79. 26 Internet-Drafts are working documents of the Internet Engineering 27 Task Force (IETF). Note that other groups may also distribute 28 working documents as Internet-Drafts. The list of current Internet- 29 Drafts is at http://datatracker.ietf.org/drafts/current/. 31 Internet-Drafts are draft documents valid for a maximum of six months 32 and may be updated, replaced, or obsoleted by other documents at any 33 time. It is inappropriate to use Internet-Drafts as reference 34 material or to cite them other than as "work in progress." 36 This Internet-Draft will expire on April 18, 2017. 38 Copyright Notice 40 Copyright (c) 2016 IETF Trust and the persons identified as the 41 document authors. All rights reserved. 43 This document is subject to BCP 78 and the IETF Trust's Legal 44 Provisions Relating to IETF Documents 45 (http://trustee.ietf.org/license-info) in effect on the date of 46 publication of this document. Please review these documents 47 carefully, as they describe your rights and restrictions with respect 48 to this document. Code Components extracted from this document must 49 include Simplified BSD License text as described in Section 4.e of 50 the Trust Legal Provisions and are provided without warranty as 51 described in the Simplified BSD License. 53 Table of Contents 55 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 56 1.1. Conventions and Terminology . . . . . . . . . . . . . . . 3 57 2. Format . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 58 2.1. Overview . . . . . . . . . . . . . . . . . . . . . . . . 3 59 2.2. Event . . . . . . . . . . . . . . . . . . . . . . . . . . 3 60 2.2.1. Event Attributes . . . . . . . . . . . . . . . . . . 3 61 2.3. Objects . . . . . . . . . . . . . . . . . . . . . . . . . 7 62 2.3.1. Org . . . . . . . . . . . . . . . . . . . . . . . . . 7 63 2.3.2. Orgc . . . . . . . . . . . . . . . . . . . . . . . . 7 64 2.4. Attribute . . . . . . . . . . . . . . . . . . . . . . . . 8 65 2.4.1. Sample Attribute Object . . . . . . . . . . . . . . . 8 66 2.4.2. Attribute Attributes . . . . . . . . . . . . . . . . 8 67 2.5. ShadowAttribute . . . . . . . . . . . . . . . . . . . . . 13 68 2.5.1. Sample Attribute Object . . . . . . . . . . . . . . . 13 69 2.5.2. ShadowAttribute Attributes . . . . . . . . . . . . . 14 70 2.5.3. Org . . . . . . . . . . . . . . . . . . . . . . . . . 18 71 2.6. Tag . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 72 2.6.1. Sample Tag . . . . . . . . . . . . . . . . . . . . . 19 73 3. Manifest . . . . . . . . . . . . . . . . . . . . . . . . . . 19 74 3.1. Format . . . . . . . . . . . . . . . . . . . . . . . . . 20 75 3.1.1. Sample Manifest . . . . . . . . . . . . . . . . . . . 21 76 4. Implementation . . . . . . . . . . . . . . . . . . . . . . . 23 77 5. Security Considerations . . . . . . . . . . . . . . . . . . . 23 78 6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 23 79 7. Sample MISP file . . . . . . . . . . . . . . . . . . . . . . 23 80 8. References . . . . . . . . . . . . . . . . . . . . . . . . . 23 81 8.1. Normative References . . . . . . . . . . . . . . . . . . 23 82 8.2. Informative References . . . . . . . . . . . . . . . . . 24 83 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 24 85 1. Introduction 87 Sharing threat information became a fundamental requirements in the 88 Internet, security and intelligence community at large. Threat 89 information can include indicators of compromise, malicious file 90 indicators, financial fraud indicators or even detailed information 91 about a threat actor. MISP [MISP-P] started as an open source 92 project in late 2011 and the MISP format started to be widely used as 93 an exchange format within the community in the past years. The aim 94 of this document is to describe the specification and the MISP core 95 format. 97 1.1. Conventions and Terminology 99 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 100 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 101 document are to be interpreted as described in RFC 2119 [RFC2119]. 103 2. Format 105 2.1. Overview 107 The MISP core format is in the JSON [RFC4627] format. In MISP, an 108 event is composed of a single JSON object. 110 A capitalized key (like Event, Org) represent a data model and a non- 111 capitalized key is just an attribute. This nomenclature can support 112 an implementation to represent the MISP format in another data 113 structure. 115 2.2. Event 117 An event is a simple meta structure scheme where attributes and meta- 118 data are embedded to compose a coherent set of indicators. An event 119 can be composed from an incident, a security analysis report or a 120 specific threat actor analysis. The meaning of an event only depends 121 of the information embedded in the event. 123 2.2.1. Event Attributes 125 2.2.1.1. uuid 127 uuid represents the Universally Unique IDentifier (UUID) [RFC4122] of 128 the event. The uuid MUST be preserved for any updates or transfer of 129 the same event. UUID version 4 is RECOMMENDED when assigning it to a 130 new event. 132 uuid is represented as a JSON string. uuid MUST be present. 134 2.2.1.2. id 136 id represents the human-readable identifier associated to the event 137 for a specific MISP instance. 139 id is represented as a JSON string. id SHALL be present. 141 2.2.1.3. published 143 published represents the event publication state. If the event was 144 published, the published value MUST be true. In any other 145 publication state, the published value MUST be false. 147 published is represented as a JSON boolean. published MUST be 148 present. 150 2.2.1.4. info 152 info represents the information field of the event. info a free-text 153 value to provide a human-readable summary of the event. info SHOULD 154 NOT be bigger than 256 characters and SHOULD NOT include new-lines. 156 info is represented as a JSON string. info MUST be present. 158 2.2.1.5. threat_level_id 160 threat_level_id represents the threat level. 162 0: 163 Undefined 165 1: 166 Low 168 2: 169 Medium 171 3: 172 High 174 If a higher granularity is required, a MISP taxonomy applied as a Tag 175 SHOULD be preferred. 177 threat_level_id is represented as a JSON string. threat_level_id 178 SHALL be present. 180 2.2.1.6. analysis 182 analysis represents the analysis level. 184 0: 185 Initial 187 1: 188 Ongoing 190 2: 191 Complete 193 If a higher granularity is required, a MISP taxonomy applied as a Tag 194 SHOULD be preferred. 196 analysis is represented as a JSON string. analysis SHALL be present. 198 2.2.1.7. date 200 date represents a reference date to the event in ISO 8601 format 201 (date only: YYYY-MM-DD). This date corresponds to the date the event 202 occured, which may be in the past. 204 date is represented as a JSON string. date MUST be present. 206 2.2.1.8. timestamp 208 timestamp represents a reference time when the event, or one of the 209 attributes within the event was created, or last updated/edited on 210 the instance. timestamp is expressed in seconds (decimal) since 1st 211 of January 1970 (Unix timestamp). The time zone MUST be UTC. 213 timestamp is represented as a JSON string. timestamp MUST be present. 215 2.2.1.9. publish_timestamp 217 publish_timestamp represents a reference time when the event was 218 published on the instance. published_timestamp is expressed in 219 seconds (decimal) since 1st of January 1970 (Unix timestamp). At 220 each publication of an event, publish_timestamp MUST be updated. The 221 time zone MUST be UTC. 223 publish_timestamp is represented as a JSON string. publish_timestamp 224 MUST be present. 226 2.2.1.10. org_id 228 org_id represents a human-readable identifier referencing an Org 229 object of the organization which generated the event. 231 The org_id MUST be updated when the event is generated by a new 232 instance. 234 org_id is represented as a JSON string. org_id MUST be present. 236 2.2.1.11. orgc_id 238 orgc_id represents a human-readable identifier referencing an Orgc 239 object of the organization which created the event. 241 The orgc_id and Orc object MUST be preserved for any updates or 242 transfer of the same event. 244 orgc_id is represented as a JSON string. orgc_id MUST be present. 246 2.2.1.12. attribute_count 248 attribute_count represents the number of attributes in the event. 249 attribute_count is expressed in decimal. 251 attribute_count is represented as a JSON string. attribute_count 252 SHALL be present. 254 2.2.1.13. distribution 256 distribution represents the basic distribution rules of the event. 257 The system must adhere to the distribution setting for access control 258 and for dissemination of the event. 260 distribution is represented by a JSON string. distribution MUST be 261 present and be one of the following options: 263 0 264 Your Organisation Only 266 1 267 This Community Only 269 2 270 Connected Communities 272 3 273 All Communities 275 4 276 Sharing Group 278 2.2.1.14. sharing_group_id 280 sharing_group_id represents a human-readable identifier referencing a 281 Sharing Group object that defines the distribution of the event, if 282 distribution level "4" is set. 284 sharing_group_id is represented by a JSON string and MUST be present. 285 If a distribution level other than "4" is chosen the sharing_group_id 286 MUST be set to "0". 288 2.3. Objects 290 2.3.1. Org 292 An Org object is composed of an uuid, name and id. 294 The uuid represents the Universally Unique IDentifier (UUID) 295 [RFC4122] of the organization. The organization UUID is globally 296 assigned to an organization and SHALL be kept overtime. 298 The name is a readable description of the organization and SHOULD be 299 present. The id is a human-readable identifier generated by the 300 instance and used as reference in the event. 302 uuid, name and id are represented as a JSON string. uuid, name and id 303 MUST be present. 305 2.3.1.1. Sample Org Object 307 "Org": { 308 "id": "2", 309 "name": "CIRCL", 310 "uuid": "55f6ea5e-2c60-40e5-964f-47a8950d210f" 311 } 313 2.3.2. Orgc 315 An Orgc object is composed of an uuid, name and id. 317 The uuid MUST be preserved for any updates or transfer of the same 318 event. UUID version 4 is RECOMMENDED when assigning it to a new 319 event. The organization UUID is globally assigned to an organization 320 and SHALL be kept overtime. 322 The name is a readable description of the organization and SHOULD be 323 present. The id is a human-readable identifier generated by the 324 instance and used as reference in the event. 326 uuid, name and id are represented as a JSON string. uuid, name and id 327 MUST be present. 329 2.4. Attribute 331 Attributes are used to describe the indicators and contextual data of 332 an event. The main information contained in an attribute is made up 333 of a category-type-value triplet, where the category and type give 334 meaning and context to the value. Through the various category-type 335 combinations a wide range of information can be conveyed. 337 A MISP document MUST at least includes category-type-value triplet 338 described in section "Attribute Attributes". 340 2.4.1. Sample Attribute Object 342 "Attribute": { 343 "id": "346056", 344 "type": "comment", 345 "category": "Other", 346 "to_ids": false, 347 "uuid": "57f4f6d9-cd20-458b-84fd-109ec0a83869", 348 "event_id": "3357", 349 "distribution": "5", 350 "timestamp": "1475679332", 351 "comment": "", 352 "sharing_group_id": "0", 353 "deleted": false, 354 "value": "Hello world", 355 "SharingGroup": [], 356 "ShadowAttribute": [], 357 "RelatedAttribute": [] 358 } 360 2.4.2. Attribute Attributes 362 2.4.2.1. uuid 364 uuid represents the Universally Unique IDentifier (UUID) [RFC4122] of 365 the event. The uuid MUST be preserved for any updates or transfer of 366 the same event. UUID version 4 is RECOMMENDED when assigning it to a 367 new event. 369 uuid is represented as a JSON string. uuid MUST be present. 371 2.4.2.2. id 373 id represents the human-readable identifier associated to the event 374 for a specific MISP instance. 376 id is represented as a JSON string. id SHALL be present. 378 2.4.2.3. type 380 type represents the means through which an attribute tries to 381 describe the intent of the attribute creator, using a list of pre- 382 defined attribute types. 384 type is represented as a JSON string. type MUST be present and it 385 MUST be a valid selection for the chosen category. The list of valid 386 category-type combinations is as follows: 388 Internal reference 389 text, link, comment, other 391 Targeting data 392 target-user, target-email, target-machine, target-org, target- 393 location, target-external, comment 395 Antivirus detection 396 link, comment, text, attachment, other 398 Payload delivery 399 md5, sha1, sha224, sha256, sha384, sha512, sha512/224, sha512/256, 400 ssdeep, imphash, authentihash, pehash, tlsh, filename, 401 filename|md5, filename|sha1, filename|sha224, filename|sha256, 402 filename|sha384, filename|sha512, filename|sha512/224, 403 filename|sha512/256, filename|authentihash, filename|ssdeep, 404 filename|tlsh, filename|imphash, filename|pehash, ip-src, ip-dst, 405 hostname, domain, email-src, email-dst, email-subject, email- 406 attachment, url, user-agent, AS, pattern-in-file, pattern-in- 407 traffic, yara, attachment, malware-sample, link, malware-type, 408 comment, text, vulnerability, x509-fingerprint-sha1, other 410 Artifacts dropped 411 md5, sha1, sha224, sha256, sha384, sha512, sha512/224, sha512/256, 412 ssdeep, imphash, authentihash, filename, filename|md5, 413 filename|sha1, filename|sha224, filename|sha256, filename|sha384, 414 filename|sha512, filename|sha512/224, filename|sha512/256, 415 filename|authentihash, filename|ssdeep, filename|tlsh, 416 filename|imphash, filename|pehash, regkey, regkey|value, pattern- 417 in-file, pattern-in-memory, pdb, yara, attachment, malware-sample, 418 named pipe, mutex, windows-scheduled-task, windows-service-name, 419 windows-service-displayname, comment, text, x509-fingerprint-sha1, 420 other 422 Payload installation 423 md5, sha1, sha224, sha256, sha384, sha512, sha512/224, sha512/256, 424 ssdeep, imphash, authentihash, pehash, tlsh, filename, 425 filename|md5, filename|sha1, filename|sha224, filename|sha256, 426 filename|sha384, filename|sha512, filename|sha512/224, 427 filename|sha512/256, filename|authentihash, filename|ssdeep, 428 filename|tlsh, filename|imphash, filename|pehash, pattern-in-file, 429 pattern-in-traffic, pattern-in-memory, yara, vulnerability, 430 attachment, malware-sample, malware-type, comment, text, x509- 431 fingerprint-sha1, other 433 Persistence mechanism 434 filename, regkey, regkey|value, comment, text, other 436 Network activity 437 ip-src, ip-dst, hostname, domain, domain|ip, email-dst, url, uri, 438 user-agent, http-method, AS, snort, pattern-in-file, pattern-in- 439 traffic, attachment, comment, text, x509-fingerprint-sha1, other 441 Payload type 442 comment, text, other 444 Attribution 445 threat-actor, campaign-name, campaign-id, whois-registrant-phone, 446 whois-registrant-email, whois-registrant-name, whois-registrar, 447 whois-creation-date, comment, text, x509-fingerprint-sha1, other 449 External analysis 450 md5, sha1, sha256, filename, filename|md5, filename|sha1, 451 filename|sha256, ip-src, ip-dst, hostname, domain, domain|ip, url, 452 user-agent, regkey, regkey|value, AS, snort, pattern-in-file, 453 pattern-in-traffic, pattern-in-memory, vulnerability, attachment, 454 malware-sample, link, comment, text, x509-fingerprint-sha1, other 456 Financial fraud 457 btc, iban, bic, bank-account-nr, aba-rtn, bin, cc-number, prtn, 458 comment, text, other 460 Other 461 comment, text, other 463 Attributes are based on the usage within their different communities. 464 Attributes can be extended on a regular basis and this reference 465 document is updated accordingly. 467 2.4.2.4. category 469 category represents the intent of what the attribute is describing as 470 selected by the attribute creator, using a list of pre-defined 471 attribute categories. 473 category is represented as a JSON string. category MUST be present 474 and it MUST be a valid selection for the chosen type. The list of 475 valid category-type combinations is mentioned above. 477 2.4.2.5. to_ids 479 to_ids represents whether the attribute is meant to be actionable. 480 Actionable defined attributes that can be used in automated processes 481 as a pattern for detection in Local or Network Intrusion Detection 482 System, log analysis tools or even filtering mechanisms. 484 to_ids is represented as a JSON boolean. to_ids MUST be present. 486 2.4.2.6. event_id 488 event_id represents a human-readable identifier referencing the Event 489 object that the attribute belongs to. 491 The event_id SHOULD be updated when the event is imported to reflect 492 the newly created event's id on the instance. 494 event_id is represented as a JSON string. event_id MUST be present. 496 2.4.2.7. distribution 498 distribution represents the basic distribution rules of the 499 attribute. The system must adhere to the distribution setting for 500 access control and for dissemination of the attribute. 502 distribution is represented by a JSON string. distribution MUST be 503 present and be one of the following options: 505 0 506 Your Organisation Only 508 1 509 This Community Only 511 2 512 Connected Communities 514 3 515 All Communities 517 4 518 Sharing Group 520 5 521 Inherit Event 523 2.4.2.8. timestamp 525 timestamp represents a reference time when the attribute was created 526 or last modified. timestamp is expressed in seconds (decimal) since 527 1st of January 1970 (Unix timestamp). The time zone MUST be UTC. 529 timestamp is represented as a JSON string. timestamp MUST be present. 531 2.4.2.9. comment 533 comment is a contextual comment field. 535 comment is represented by a JSON string. comment MAY be present. 537 2.4.2.10. sharing_group_id 539 sharing_group_id represents a human-readable identifier referencing a 540 Sharing Group object that defines the distribution of the attribute, 541 if distribution level "4" is set. 543 sharing_group_id is represented by a JSON string and MUST be present. 544 If a distribution level other than "4" is chosen the sharing_group_id 545 MUST be set to "0". 547 2.4.2.11. deleted 549 deleted represents a setting that allows attributes to be revoked. 550 Revoked attributes are not actionable and exist merely to inform 551 other instances of a revocation. 553 deleted is represented by a JSON boolean. deleted MUST be present. 555 2.4.2.12. data 557 data contains the base64 encoded contents of an attachment or a 558 malware sample. For malware samples, the sample MUST be encrypted 559 using a password protected zip archive, with the password being 560 "infected". 562 data is represented by a JSON string in base64 encoding. data MUST be 563 set for attributes of type malware-sample and attachment. 565 2.4.2.13. RelatedAttribute 567 RelatedAttribute is an array of attributes correlating with the 568 current attribute. Each element in the array represents an JSON 569 object which contains an Attribute dictionnary with the external 570 attributes who correlate. Each Attribute MUST include the id, 571 org_id, info and a value. Only the correlations found on the local 572 instance are shown in RelatedAttribute. 574 RelatedAttribute MAY be present. 576 2.4.2.14. ShadowAttribute 578 ShadowAttribute is an array of shadow attributes that serve as 579 proposals by third parties to alter the containing attribute. The 580 structure of a ShadowAttribute is similar to that of an Attribute, 581 which can be accepted or discarded by the event creator. If 582 accepted, the original attribute containing the shadow attribute is 583 removed and the shadow attribute is converted into an attribute. 585 Each shadow attribute that references an attribute MUST contain the 586 containing attribute's ID in the old_id field and the event's ID in 587 the event_id field. 589 2.4.2.15. value 591 value represents the payload of an attribute. The format of the 592 value is dependent on the type of the attribute. 594 value is represented by a JSON string. value MUST be present. 596 2.5. ShadowAttribute 598 ShadowAttributes are 3rd party created attributes that either propose 599 to add new information to an event or modify existing information. 600 They are not meant to be actionable until the event creator accepts 601 them - at which point they will be converted into attributes or 602 modify an existing attribute. 604 They are similar in structure to Attributes but additionally carry a 605 reference to the creator of the ShadowAttribute as well as a 606 revocation flag. 608 2.5.1. Sample Attribute Object 609 "ShadowAttribute": { 610 "id": "8", 611 "type": "ip-src", 612 "category": "Network activity", 613 "to_ids": false, 614 "uuid": "57d475f1-da78-4569-89de-1458c0a83869", 615 "event_uuid": "57d475e6-41c4-41ca-b450-145ec0a83869", 616 "event_id": "9", 617 "old_id": "319", 618 "comment": "", 619 "org_id": "1", 620 "proposal_to_delete": false, 621 "value": "5.5.5.5", 622 "deleted": false, 623 "Org": { 624 "id": "1", 625 "name": "MISP", 626 "uuid": "568cce5a-0c80-412b-8fdf-1ffac0a83869" 627 } 628 } 630 2.5.2. ShadowAttribute Attributes 632 2.5.2.1. uuid 634 uuid represents the Universally Unique IDentifier (UUID) [RFC4122] of 635 the event. The uuid MUST be preserved for any updates or transfer of 636 the same event. UUID version 4 is RECOMMENDED when assigning it to a 637 new event. 639 uuid is represented as a JSON string. uuid MUST be present. 641 2.5.2.2. id 643 id represents the human-readable identifier associated to the event 644 for a specific MISP instance. 646 id is represented as a JSON string. id SHALL be present. 648 2.5.2.3. type 650 type represents the means through which an attribute tries to 651 describe the intent of the attribute creator, using a list of pre- 652 defined attribute types. 654 type is represented as a JSON string. type MUST be present and it 655 MUST be a valid selection for the chosen category. The list of valid 656 category-type combinations is as follows: 658 Internal reference 659 text, link, comment, other 661 Targeting data 662 target-user, target-email, target-machine, target-org, target- 663 location, target-external, comment 665 Antivirus detection 666 link, comment, text, attachment, other 668 Payload delivery 669 md5, sha1, sha224, sha256, sha384, sha512, sha512/224, sha512/256, 670 ssdeep, imphash, authentihash, pehash, tlsh, filename, 671 filename|md5, filename|sha1, filename|sha224, filename|sha256, 672 filename|sha384, filename|sha512, filename|sha512/224, 673 filename|sha512/256, filename|authentihash, filename|ssdeep, 674 filename|tlsh, filename|imphash, filename|pehash, ip-src, ip-dst, 675 hostname, domain, email-src, email-dst, email-subject, email- 676 attachment, url, user-agent, AS, pattern-in-file, pattern-in- 677 traffic, yara, attachment, malware-sample, link, malware-type, 678 comment, text, vulnerability, x509-fingerprint-sha1, other 680 Artifacts dropped 681 md5, sha1, sha224, sha256, sha384, sha512, sha512/224, sha512/256, 682 ssdeep, imphash, authentihash, filename, filename|md5, 683 filename|sha1, filename|sha224, filename|sha256, filename|sha384, 684 filename|sha512, filename|sha512/224, filename|sha512/256, 685 filename|authentihash, filename|ssdeep, filename|tlsh, 686 filename|imphash, filename|pehash, regkey, regkey|value, pattern- 687 in-file, pattern-in-memory, pdb, yara, attachment, malware-sample, 688 named pipe, mutex, windows-scheduled-task, windows-service-name, 689 windows-service-displayname, comment, text, x509-fingerprint-sha1, 690 other 692 Payload installation 693 md5, sha1, sha224, sha256, sha384, sha512, sha512/224, sha512/256, 694 ssdeep, imphash, authentihash, pehash, tlsh, filename, 695 filename|md5, filename|sha1, filename|sha224, filename|sha256, 696 filename|sha384, filename|sha512, filename|sha512/224, 697 filename|sha512/256, filename|authentihash, filename|ssdeep, 698 filename|tlsh, filename|imphash, filename|pehash, pattern-in-file, 699 pattern-in-traffic, pattern-in-memory, yara, vulnerability, 700 attachment, malware-sample, malware-type, comment, text, x509- 701 fingerprint-sha1, other 703 Persistence mechanism 704 filename, regkey, regkey|value, comment, text, other 706 Network activity 707 ip-src, ip-dst, hostname, domain, domain|ip, email-dst, url, uri, 708 user-agent, http-method, AS, snort, pattern-in-file, pattern-in- 709 traffic, attachment, comment, text, x509-fingerprint-sha1, other 711 Payload type 712 comment, text, other 714 Attribution 715 threat-actor, campaign-name, campaign-id, whois-registrant-phone, 716 whois-registrant-email, whois-registrant-name, whois-registrar, 717 whois-creation-date, comment, text, x509-fingerprint-sha1, other 719 External analysis 720 md5, sha1, sha256, filename, filename|md5, filename|sha1, 721 filename|sha256, ip-src, ip-dst, hostname, domain, domain|ip, url, 722 user-agent, regkey, regkey|value, AS, snort, pattern-in-file, 723 pattern-in-traffic, pattern-in-memory, vulnerability, attachment, 724 malware-sample, link, comment, text, x509-fingerprint-sha1, other 726 Financial fraud 727 btc, iban, bic, bank-account-nr, aba-rtn, bin, cc-number, prtn, 728 comment, text, other 730 Other 731 comment, text, other 733 Attributes are based on the usage within their different communities. 734 Attributes can be extended on a regular basis and this reference 735 document is updated accordingly. 737 2.5.2.4. category 739 category represents the intent of what the attribute is describing as 740 selected by the attribute creator, using a list of pre-defined 741 attribute categories. 743 category is represented as a JSON string. category MUST be present 744 and it MUST be a valid selection for the chosen type. The list of 745 valid category-type combinations is mentioned above. 747 2.5.2.5. to_ids 749 to_ids represents whether the Attribute to be created if the 750 ShadowAttribute is accepted is meant to be actionable. Actionable 751 defined attributes that can be used in automated processes as a 752 pattern for detection in Local or Network Intrusion Detection System, 753 log analysis tools or even filtering mechanisms. 755 to_ids is represented as a JSON boolean. to_ids MUST be present. 757 2.5.2.6. event_id 759 event_id represents a human-readable identifier referencing the Event 760 object that the ShadowAttribute belongs to. 762 The event_id SHOULD be updated when the event is imported to reflect 763 the newly created event's id on the instance. 765 event_id is represented as a JSON string. event_id MUST be present. 767 2.5.2.7. old_id 769 old_id represents a human-readable identifier referencing the 770 Attribute object that the ShadowAttribute belongs to. A 771 ShadowAttribute can this way target an existing Attribute, implying 772 that it is a proposal to modify an existing Attribute, or 773 alternatively it can be a proposal to create a new Attribute for the 774 containing Event. 776 The old_id SHOULD be updated when the event is imported to reflect 777 the newly created Attribute's id on the instance. Alternatively, if 778 the ShadowAttribute proposes the creation of a new Attribute, it 779 should be set to 0. 781 old_id is represented as a JSON string. old_id MUST be present. 783 2.5.2.8. timestamp 785 timestamp represents a reference time when the attribute was created 786 or last modified. timestamp is expressed in seconds (decimal) since 787 1st of January 1970 (Unix timestamp). The time zone MUST be UTC. 789 timestamp is represented as a JSON string. timestamp MUST be present. 791 2.5.2.9. comment 793 comment is a contextual comment field. 795 comment is represented by a JSON string. comment MAY be present. 797 2.5.2.10. org_id 799 org_id represents a human-readable identifier referencing the 800 proposal creator's Organisation object. 802 Whilst attributes can only be created by the event creator 803 organisation, shadow attributes can be created by third parties. 804 org_id tracks the creator organisation. 806 org_id is represented by a JSON string and MUST be present. 808 2.5.2.11. proposal_to_delete 810 proposal_to_delete is a boolean flag that sets whether the shadow 811 attribute proposes to alter an attribute, or whether it proposes to 812 remove it completely. 814 Accepting a shadow attribute with this flag set will remove the 815 target attribute. 817 proposal_to_delete is a JSON boolean and it MUST be present. If 818 proposal_to_delete is set to true, old_id MUST NOT be 0. 820 2.5.2.12. deleted 822 deleted represents a setting that allows shadow attributes to be 823 revoked. Revoked shadow attributes only serve to inform other 824 instances that the shadow attribute is no longer active. 826 deleted is represented by a JSON boolean. deleted SHOULD be present. 828 2.5.2.13. data 830 data contains the base64 encoded contents of an attachment or a 831 malware sample. For malware samples, the sample MUST be encrypted 832 using a password protected zip archive, with the password being 833 "infected". 835 data is represented by a JSON string in base64 encoding. data MUST be 836 set for shadow attributes of type malware-sample and attachment. 838 2.5.3. Org 840 An Org object is composed of an uuid, name and id. 842 The uuid represents the Universally Unique IDentifier (UUID) 843 [RFC4122] of the organization. The organization UUID is globally 844 assigned to an organization and SHALL be kept overtime. 846 The name is a readable description of the organization and SHOULD be 847 present. The id is a human-readable identifier generated by the 848 instance and used as reference in the event. 850 uuid, name and id are represented as a JSON string. uuid, name and id 851 MUST be present. 853 2.5.3.1. Sample Org Object 855 "Org": { 856 "id": "2", 857 "name": "CIRCL", 858 "uuid": "55f6ea5e-2c60-40e5-964f-47a8950d210f" 859 } 861 2.5.3.2. value 863 value represents the payload of an attribute. The format of the 864 value is dependent on the type of the attribute. 866 value is represented by a JSON string. value MUST be present. 868 2.6. Tag 870 A Tag is a simple method to classify an event with a simple tag name. 871 The tag name can be freely chosen. The tag name can be also chosen 872 from a fixed machine-tag vocabulary called MISP taxonomies[[MISP-T]]. 873 A Tag is represented as a JSON array where each element describes 874 each tag associated. A Tag array SHALL be, at least, at Event level. 875 A tag element is described with a name, id, colour and exportable 876 flag. 878 exportable represents a setting if the tag is kept local or 879 exportable to other MISP instances. exportable is represented by a 880 JSON boolean. id is a human-readable identifier that references the 881 tag on the local instance. colour represents an RGB value of the tag. 883 name MUST be present. colour, id and exportable SHALL be present. 885 2.6.1. Sample Tag 887 "Tag": [{ 888 "exportable": true, 889 "colour": "#ffffff", 890 "name": "tlp:white", 891 "id": "2" }] 893 3. Manifest 895 MISP events can be shared over an HTTP repository, a file package or 896 USB key. A manifest file is used to provide an index of MISP events 897 allowing to only fetch the recently updated files without the need to 898 parse each json file. 900 3.1. Format 902 A manifest file is a simple JSON file named manifest.json in a 903 directory where the MISP events are located. Each MISP event is a 904 file located in the same directory with the event uuid as filename 905 with the json extension. 907 The manifest format is a JSON object composed of a dictionary where 908 the field is the uuid of the event. 910 Each uuid is composed of a JSON object with the following fields 911 which came from the original event referenced by the same uuid: 913 o info (MUST) 915 o Orgc object (MUST) 917 o analysis (SHALL) 919 o timestamp (MUST) 921 o date (MUST) 923 o threat_level_id (SHALL) 925 In addition to the fields originating from the event, the following 926 fields can be added: 928 o integrity:sha256 represents the SHA256 value in hexadecimal 929 representation of the associated MISP event file to ensure 930 integrity of the file. (SHOULD) 932 o integrity:pgp represents a detached PGP signature [RFC4880] of the 933 associated MISP event file to ensure integrity of the file. 934 (SHOULD) 936 If a detached PGP signature is used for each MISP event, a detached 937 PGP signature is a MUST to ensure integrity of the manifest file. A 938 detached PGP signature for a manifest file is a manifest.json.pgp 939 file containing the PGP signature. 941 3.1.1. Sample Manifest 942 { 943 "57c6ac4c-c60c-4f79-a38f-b666950d210f": { 944 "info": "Malspam 2016-08-31 (.wsf in .zip) - campaign: Photo", 945 "Orgc": { 946 "id": "2", 947 "name": "CIRCL" 948 }, 949 "analysis": "0", 950 "Tag": [ 951 { 952 "colour": "#3d7a00", 953 "name": "circl:incident-classification=\"malware\"" 954 }, 955 { 956 "colour": "#ffffff", 957 "name": "tlp:white" 958 } 959 ], 960 "timestamp": "1472638251", 961 "date": "2016-08-31", 962 "threat_level_id": "3" 963 }, 964 "5720accd-dd28-45f8-80e5-4605950d210f": { 965 "info": "Malspam 2016-04-27 - Locky", 966 "Orgc": { 967 "id": "2", 968 "name": "CIRCL" 969 }, 970 "analysis": "2", 971 "Tag": [ 972 { 973 "colour": "#ffffff", 974 "name": "tlp:white" 975 }, 976 { 977 "colour": "#3d7a00", 978 "name": "circl:incident-classification=\"malware\"" 979 }, 980 { 981 "colour": "#2c4f00", 982 "name": "malware_classification:malware-category=\"Ransomware\"" 983 } 984 ], 985 "timestamp": "1461764231", 986 "date": "2016-04-27", 987 "threat_level_id": "3" 988 } 989 } 990 4. Implementation 992 MISP format is implemented by different software including the MISP 993 threat sharing platform and libraries like PyMISP [MISP-P]. 994 Implementations use the format as an export/import mechanism, staging 995 transport format or synchronisation format as used in the MISP core 996 platform. MISP format doesn't impose any restriction on the data 997 representation of the format in data-structure of other 998 implementations. 1000 5. Security Considerations 1002 MISP events might contain sensitive or confidential information. 1003 Adequate access control and encryption measures shall be implemented 1004 to ensure the confidentiality of the MISP events. 1006 Adversaries might include malicious content in MISP events and 1007 attributes. Implementation MUST consider the input of malicious 1008 inputs beside the standard threat information that might already 1009 include malicious intended inputs. 1011 6. Acknowledgements 1013 The authors wish to thank all the MISP community to support the 1014 creation of open standards in threat intelligence sharing. 1016 7. Sample MISP file 1018 8. References 1020 8.1. Normative References 1022 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1023 Requirement Levels", BCP 14, RFC 2119, 1024 DOI 10.17487/RFC2119, March 1997, 1025 . 1027 [RFC4122] Leach, P., Mealling, M., and R. Salz, "A Universally 1028 Unique IDentifier (UUID) URN Namespace", RFC 4122, 1029 DOI 10.17487/RFC4122, July 2005, 1030 . 1032 [RFC4627] Crockford, D., "The application/json Media Type for 1033 JavaScript Object Notation (JSON)", RFC 4627, 1034 DOI 10.17487/RFC4627, July 2006, 1035 . 1037 [RFC4880] Callas, J., Donnerhacke, L., Finney, H., Shaw, D., and R. 1038 Thayer, "OpenPGP Message Format", RFC 4880, 1039 DOI 10.17487/RFC4880, November 2007, 1040 . 1042 8.2. Informative References 1044 [MISP-P] MISP, , "MISP Project - Malware Information Sharing 1045 Platform and Threat Sharing", . 1047 [MISP-T] MISP, , "MISP Taxonomies - shared and common vocabularies 1048 of tags", . 1050 Authors' Addresses 1052 Alexandre Dulaunoy 1053 Computer Incident Response Center Luxembourg 1054 41, avenue de la gare 1055 Luxembourg L-1611 1056 Luxembourg 1058 Phone: +352 247 88444 1059 Email: alexandre.dulaunoy@circl.lu 1061 Andras Iklody 1062 Computer Incident Response Center Luxembourg 1063 41, avenue de la gare 1064 Luxembourg L-1611 1065 Luxembourg 1067 Phone: +352 247 88444 1068 Email: andras.iklody@circl.lu